Merge branch 'main' into dktk-migration

This commit is contained in:
Torben Brenner 2023-08-16 09:14:04 +02:00
commit e69c0ec306
25 changed files with 266 additions and 108 deletions

View File

@ -8,7 +8,6 @@ This repository is the starting point for any information and tools you will nee
- [Hardware](#hardware) - [Hardware](#hardware)
- [Software](#software) - [Software](#software)
- [Network](#network) - [Network](#network)
- [Register with the Directory](#register-with-the-directory)
2. [Deployment](#deployment) 2. [Deployment](#deployment)
- [Site name](#site-name) - [Site name](#site-name)
- [Projects](#projects) - [Projects](#projects)
@ -22,6 +21,7 @@ This repository is the starting point for any information and tools you will nee
- [HTTPS Access](#https-access) - [HTTPS Access](#https-access)
- [TLS terminating proxies](#tls-terminating-proxies) - [TLS terminating proxies](#tls-terminating-proxies)
- [File structure](#file-structure) - [File structure](#file-structure)
- [BBMRI-ERIC Directory](#bbmri-eric-directory)
4. [Things you should know](#things-you-should-know) 4. [Things you should know](#things-you-should-know)
- [Auto-Updates](#auto-updates) - [Auto-Updates](#auto-updates)
- [Auto-Backups](#auto-backups) - [Auto-Backups](#auto-backups)
@ -64,18 +64,10 @@ Additionally, your site might use its own proxy. You should discuss this with yo
Note that git and Docker may also need to be configured to use this proxy. This is a job for your systems administrators. Note that git and Docker may also need to be configured to use this proxy. This is a job for your systems administrators.
If there is a site firewall, this needs to be configured so that git and Docker can reach the outside world. Another job for the systems administrators. If there is a site firewall, this needs to be configured so that outgoing calls to the following URLs are allowed: *.dkfz.de, github.com, docker.io, *.docker.io, *.samply.de.
Note for Ubuntu: Please note that the uncomplicated firewall (ufw) is known to conflict with Docker [here](https://github.com/chaifeng/ufw-docker). Note for Ubuntu: Please note that the uncomplicated firewall (ufw) is known to conflict with Docker [here](https://github.com/chaifeng/ufw-docker).
### Register with the Directory
If you run a biobank, you should register with the [Directory](https://directory.bbmri-eric.eu), a BBMRI project that catalogs biobanks.
To do this, contact the BBMRI national node for the country where your biobank is based, see [the list of nodes](http://www.bbmri-eric.eu/national-nodes/).
Once you have registered, **you should choose one of your sample collections as a default collection for your biobank**. This is the collection that will be automatically used to label any samples that have not been assigned a collection ID in your ETL process. Make a note of this ID, you will need it later on in the installation process.
## Deployment ## Deployment
### Site name ### Site name
@ -255,9 +247,15 @@ All of the Bridgehead's outgoing connections are secured by transport encryption
Your Bridgehead's actual data is not stored in the above directories, but in named docker volumes, see `docker volume ls` and `docker volume inspect <volume_name>`. Your Bridgehead's actual data is not stored in the above directories, but in named docker volumes, see `docker volume ls` and `docker volume inspect <volume_name>`.
### Directory sync ### BBMRI-ERIC Directory
This is an optional feature for bbmri projects. It keeps the [BBMRI Directory](https://directory.bbmri-eric.eu/) up to date with your local data eg. number of samples. It also updates the local FHIR store with the latest contact details etc. from the Directory. You must explicitly set your country specific directory url, username and password to enable this feature. If you run a biobank, you should register with the [Directory](https://directory.bbmri-eric.eu), a BBMRI-ERIC project that catalogs biobanks.
To do this, contact the BBMRI-ERIC national node for the country where your biobank is based, see [the list of nodes](http://www.bbmri-eric.eu/national-nodes/).
Once you have registered, **you should choose one of your sample collections as a default collection for your biobank**. This is the collection that will be automatically used to label any samples that have not been assigned a collection ID in your ETL process. Make a note of this ID, you will need it later on in the installation process.
The Bridgehead's **Directory Sync** is an optional feature that keeps the Directory up to date with your local data, e.g. number of samples. Conversely, it also updates the local FHIR store with the latest contact details etc. from the Directory. You must explicitly set your country specific directory url, username and password to enable this feature.
Full details can be found in [directory_sync_service](https://github.com/samply/directory_sync_service). Full details can be found in [directory_sync_service](https://github.com/samply/directory_sync_service).
@ -274,7 +272,7 @@ You must contact the Directory for your national node to find the URL, and to re
Additionally, you should choose when you want Directory sync to run. In the example above, this is set to happen at 10 pm every evening. You can modify this to suit your requirements. The timer specification should follow the [cron](https://crontab.guru) convention. Additionally, you should choose when you want Directory sync to run. In the example above, this is set to happen at 10 pm every evening. You can modify this to suit your requirements. The timer specification should follow the [cron](https://crontab.guru) convention.
Once you edited the gitlab config. The bridgehead will autoupdate the config with the values and will sync the data. Once you edited the gitlab config, the bridgehead will autoupdate the config with the values and will sync the data.
There will be a delay before the effects of Directory sync become visible. First, you will need to wait until the time you have specified in ```TIMER_CRON```. Second, the information will then be synchronized from your national node with the central European Directory. This can take up to 24 hours. There will be a delay before the effects of Directory sync become visible. First, you will need to wait until the time you have specified in ```TIMER_CRON```. Second, the information will then be synchronized from your national node with the central European Directory. This can take up to 24 hours.

View File

@ -1,8 +0,0 @@
#!/bin/bash
function dirSetup() {
if [ -n "$DS_DIRECTORY_USER_NAME" ]; then
log INFO "Directory sync setup detected -- will start directory sync service."
OVERRIDE+=" -f ./$PROJECT/directory-sync-compose.yml"
fi
}

View File

@ -1,5 +1,7 @@
version: "3.7" version: "3.7"
# This includes only the shared persistence for BBMRI-ERIC and GBN. Federation components are included as modules, see vars.
services: services:
blaze: blaze:
image: docker.verbis.dkfz.de/cache/samply/blaze:latest image: docker.verbis.dkfz.de/cache/samply/blaze:latest
@ -19,42 +21,10 @@ services:
- "traefik.http.routers.blaze_ccp.middlewares=ccp_b_strip,auth" - "traefik.http.routers.blaze_ccp.middlewares=ccp_b_strip,auth"
- "traefik.http.routers.blaze_ccp.tls=true" - "traefik.http.routers.blaze_ccp.tls=true"
focus:
image: docker.verbis.dkfz.de/cache/samply/focus:main
container_name: bridgehead-focus
environment:
API_KEY: ${FOCUS_BEAM_SECRET_SHORT}
BEAM_APP_ID_LONG: focus.${PROXY_ID}
PROXY_ID: ${PROXY_ID}
BLAZE_URL: "http://bridgehead-bbmri-blaze:8080/fhir/"
BEAM_PROXY_URL: http://beam-proxy:8081
RETRY_COUNT: ${FOCUS_RETRY_COUNT}
depends_on:
- "beam-proxy"
- "blaze"
beam-proxy:
image: docker.verbis.dkfz.de/cache/samply/beam-proxy:develop
container_name: bridgehead-beam-proxy
environment:
BROKER_URL: ${BROKER_URL}
PROXY_ID: ${PROXY_ID}
APP_focus_KEY: ${FOCUS_BEAM_SECRET_SHORT}
PRIVKEY_FILE: /run/secrets/proxy.pem
ALL_PROXY: http://forward_proxy:3128
TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
ROOTCERT_FILE: /conf/root.crt.pem
secrets:
- proxy.pem
depends_on:
- "forward_proxy"
volumes:
- /etc/bridgehead/trusted-ca-certs:/conf/trusted-ca-certs:ro
- /srv/docker/bridgehead/bbmri/root.crt.pem:/conf/root.crt.pem:ro
volumes: volumes:
blaze-data: blaze-data:
# used in modules *-locator.yml
secrets: secrets:
proxy.pem: proxy.pem:
file: /etc/bridgehead/pki/${SITE_ID}.priv.pem file: /etc/bridgehead/pki/${SITE_ID}.priv.pem

View File

@ -0,0 +1,6 @@
#!/bin/bash
if [ -n "${DS_DIRECTORY_USER_NAME}" ]; then
log INFO "Directory sync setup detected -- will start directory sync service."
OVERRIDE+=" -f ./$PROJECT/modules/directory-sync-compose.yml"
fi

View File

@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
if [ -n "${ENABLE_DNPM}" ]; then if [ -n "${ENABLE_DNPM}" ]; then
log DEBUG "DNPM setup detected (Beam.Connect) -- will start Beam and Beam.Connect for DNPM." log INFO "DNPM setup detected (Beam.Connect) -- will start Beam and Beam.Connect for DNPM."
OVERRIDE+=" -f ./$PROJECT/modules/dnpm-compose.yml" OVERRIDE+=" -f ./$PROJECT/modules/dnpm-compose.yml"
# Set variables required for Beam-Connect # Set variables required for Beam-Connect

View File

@ -0,0 +1,36 @@
version: "3.7"
services:
focus-eric:
image: docker.verbis.dkfz.de/cache/samply/focus:main
container_name: bridgehead-focus-eric
environment:
API_KEY: ${ERIC_FOCUS_BEAM_SECRET_SHORT}
BEAM_APP_ID_LONG: focus.${ERIC_PROXY_ID}
PROXY_ID: ${ERIC_PROXY_ID}
BLAZE_URL: "http://blaze:8080/fhir/"
BEAM_PROXY_URL: http://beam-proxy-eric:8081
RETRY_COUNT: ${FOCUS_RETRY_COUNT}
depends_on:
- "beam-proxy-eric"
- "blaze"
beam-proxy-eric:
image: docker.verbis.dkfz.de/cache/samply/beam-proxy:develop
container_name: bridgehead-beam-proxy-eric
environment:
BROKER_URL: ${ERIC_BROKER_URL}
PROXY_ID: ${ERIC_PROXY_ID}
APP_focus_KEY: ${ERIC_FOCUS_BEAM_SECRET_SHORT}
PRIVKEY_FILE: /run/secrets/proxy.pem
ALL_PROXY: http://forward_proxy:3128
TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
ROOTCERT_FILE: /conf/root.crt.pem
secrets:
- proxy.pem
depends_on:
- "forward_proxy"
volumes:
- /etc/bridgehead/trusted-ca-certs:/conf/trusted-ca-certs:ro
- /srv/docker/bridgehead/bbmri/modules/eric.root.crt.pem:/conf/root.crt.pem:ro

View File

@ -0,0 +1,13 @@
#!/bin/bash
if [ "${ENABLE_ERIC}" == "true" ]; then
log INFO "BBMRI-ERIC setup detected -- will start services for BBMRI-ERIC."
OVERRIDE+=" -f ./$PROJECT/modules/eric-compose.yml"
# Set required variables
ERIC_BROKER_ID=broker.bbmri.samply.de
ERIC_BROKER_URL=https://${ERIC_BROKER_ID}
ERIC_PROXY_ID=${SITE_ID}.${ERIC_BROKER_ID}
ERIC_FOCUS_BEAM_SECRET_SHORT="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)"
ERIC_SUPPORT_EMAIL=bridgehead@helpdesk.bbmri-eric.eu
fi

View File

@ -0,0 +1,36 @@
version: "3.7"
services:
focus-gbn:
image: docker.verbis.dkfz.de/cache/samply/focus:main
container_name: bridgehead-focus-gbn
environment:
API_KEY: ${GBN_FOCUS_BEAM_SECRET_SHORT}
BEAM_APP_ID_LONG: focus.${GBN_PROXY_ID}
PROXY_ID: ${GBN_PROXY_ID}
BLAZE_URL: "http://blaze:8080/fhir/"
BEAM_PROXY_URL: http://beam-proxy-gbn:8081
RETRY_COUNT: ${FOCUS_RETRY_COUNT}
depends_on:
- "beam-proxy-gbn"
- "blaze"
beam-proxy-gbn:
image: docker.verbis.dkfz.de/cache/samply/beam-proxy:develop
container_name: bridgehead-beam-proxy-gbn
environment:
BROKER_URL: ${GBN_BROKER_URL}
PROXY_ID: ${GBN_PROXY_ID}
APP_focus_KEY: ${GBN_FOCUS_BEAM_SECRET_SHORT}
PRIVKEY_FILE: /run/secrets/proxy.pem
ALL_PROXY: http://forward_proxy:3128
TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
ROOTCERT_FILE: /conf/root.crt.pem
secrets:
- proxy.pem
depends_on:
- "forward_proxy"
volumes:
- /etc/bridgehead/trusted-ca-certs:/conf/trusted-ca-certs:ro
- /srv/docker/bridgehead/bbmri/modules/gbn.root.crt.pem:/conf/root.crt.pem:ro

View File

@ -0,0 +1,13 @@
#!/bin/bash
if [ "${ENABLE_GBN}" == "true" ]; then
log INFO "GBN setup detected -- will start services for German Biobank Node."
OVERRIDE+=" -f ./$PROJECT/modules/gbn-compose.yml"
# Set required variables
GBN_BROKER_ID='#TODO#'
GBN_BROKER_URL=https://${GBN_BROKER_ID}
GBN_PROXY_ID=${SITE_ID}.${GBN_BROKER_ID}
GBN_FOCUS_BEAM_SECRET_SHORT="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)"
GBN_SUPPORT_EMAIL=todo@verbis.dkfz.de
fi

View File

@ -0,0 +1 @@
#TODO#

View File

@ -1,9 +1,10 @@
BROKER_ID=broker.bbmri.samply.de # Makes sense for all European Biobanks
BROKER_URL=https://${BROKER_ID} : ${ENABLE_ERIC:=true}
PROXY_ID=${SITE_ID}.${BROKER_ID}
FOCUS_BEAM_SECRET_SHORT="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)" # Makes only sense for German Biobanks
: ${ENABLE_GBN:=false}
FOCUS_RETRY_COUNT=32 FOCUS_RETRY_COUNT=32
SUPPORT_EMAIL=bridgehead@helpdesk.bbmri-eric.eu
PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem
for module in $PROJECT/modules/*.sh for module in $PROJECT/modules/*.sh
@ -12,6 +13,25 @@ do
source $module source $module
done done
# This will load directory-sync setup. SUPPORT_EMAIL=$ERIC_SUPPORT_EMAIL
source $PROJECT/directory-sync.sh BROKER_URL_FOR_PREREQ=$ERIC_BROKER_URL
dirSetup
if [ -n "$GBN_SUPPORT_EMAIL" ]; then
SUPPORT_EMAIL=$GBN_SUPPORT_EMAIL
fi
function do_enroll {
COUNT=0
if [ "$ENABLE_ERIC" == "true" ]; then
do_enroll_inner $ERIC_PROXY_ID $ERIC_SUPPORT_EMAIL
COUNT=$((COUNT+1))
fi
if [ "$ENABLE_GBN" == "true" ]; then
do_enroll_inner $GBN_PROXY_ID $GBN_SUPPORT_EMAIL
COUNT=$((COUNT+1))
fi
if [ $COUNT -ge 2 ]; then
echo
echo "You just received $COUNT certificate signing requests (CSR). Please send $COUNT e-mails, with 1 CSR each, to the respective e-mail address."
fi
}

View File

@ -73,7 +73,6 @@ case "$ACTION" in
hc_send log "Bridgehead $PROJECT startup: Checking requirements ..." hc_send log "Bridgehead $PROJECT startup: Checking requirements ..."
checkRequirements checkRequirements
hc_send log "Bridgehead $PROJECT startup: Requirements checked out. Now starting bridgehead ..." hc_send log "Bridgehead $PROJECT startup: Requirements checked out. Now starting bridgehead ..."
export LDM_LOGIN=$(getLdmPassword)
exec $COMPOSE -p $PROJECT -f ./minimal/docker-compose.yml -f ./$PROJECT/docker-compose.yml $OVERRIDE up --abort-on-container-exit exec $COMPOSE -p $PROJECT -f ./minimal/docker-compose.yml -f ./$PROJECT/docker-compose.yml $OVERRIDE up --abort-on-container-exit
;; ;;
stop) stop)
@ -103,25 +102,17 @@ case "$ACTION" in
uninstall) uninstall)
exec ./lib/uninstall-bridgehead.sh $PROJECT exec ./lib/uninstall-bridgehead.sh $PROJECT
;; ;;
adduser)
loadVars
log "INFO" "Adding encrypted credentials in /etc/bridgehead/$PROJECT.local.conf"
read -p "Please choose the component (LDM_AUTH|NNGM_AUTH) you want to add a user to : " COMPONENT
read -p "Please enter a username: " USER
read -s -p "Please enter a password (will not be echoed): "$'\n' PASSWORD
add_basic_auth_user $USER $PASSWORD $COMPONENT $PROJECT
;;
enroll) enroll)
loadVars loadVars
do_enroll $PROXY_ID
MANUAL_PROXY_ID="${3:-$PROXY_ID}"
if [ -z "$MANUAL_PROXY_ID" ]; then
log ERROR "No Proxy ID set"
exit 1
else
log INFO "Enrolling Beam Proxy Id $MANUAL_PROXY_ID"
fi
if [ -z "$SUPPORT_EMAIL" ]; then
EMAIL_PARAM=""
else
EMAIL_PARAM="--admin-email $SUPPORT_EMAIL"
fi
docker run --rm -ti -v /etc/bridgehead/pki:/etc/bridgehead/pki docker.verbis.dkfz.de/cache/samply/beam-enroll:latest --output-file $PRIVATEKEYFILENAME --proxy-id $MANUAL_PROXY_ID $EMAIL_PARAM
chmod 600 $PRIVATEKEYFILENAME
;; ;;
preRun | preUpdate) preRun | preUpdate)
fixPermissions fixPermissions

View File

@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
if [ -n "${ENABLE_DNPM}" ]; then if [ -n "${ENABLE_DNPM}" ]; then
log DEBUG "DNPM setup detected (Beam.Connect) -- will start Beam.Connect for DNPM." log INFO "DNPM setup detected (Beam.Connect) -- will start Beam.Connect for DNPM."
OVERRIDE+=" -f ./$PROJECT/modules/dnpm-compose.yml" OVERRIDE+=" -f ./$PROJECT/modules/dnpm-compose.yml"
# Set variables required for Beam-Connect # Set variables required for Beam-Connect

View File

@ -5,6 +5,7 @@ services:
container_name: bridgehead-id-manager container_name: bridgehead-id-manager
environment: environment:
TOMCAT_REVERSEPROXY_FQDN: ${HOST} TOMCAT_REVERSEPROXY_FQDN: ${HOST}
TOMCAT_REVERSEPROXY_SSL: "true"
MAGICPL_SITE: ${IDMANAGEMENT_FRIENDLY_ID} MAGICPL_SITE: ${IDMANAGEMENT_FRIENDLY_ID}
MAGICPL_ALLOWED_ORIGINS: https://${HOST} MAGICPL_ALLOWED_ORIGINS: https://${HOST}
MAGICPL_LOCAL_PATIENTLIST_APIKEY: ${IDMANAGER_LOCAL_PATIENTLIST_APIKEY} MAGICPL_LOCAL_PATIENTLIST_APIKEY: ${IDMANAGER_LOCAL_PATIENTLIST_APIKEY}

View File

@ -18,7 +18,12 @@ services:
- "traefik.http.middlewares.connector_strip.stripprefix.prefixes=/nngm-connector" - "traefik.http.middlewares.connector_strip.stripprefix.prefixes=/nngm-connector"
- "traefik.http.services.connector.loadbalancer.server.port=8080" - "traefik.http.services.connector.loadbalancer.server.port=8080"
- "traefik.http.routers.connector.tls=true" - "traefik.http.routers.connector.tls=true"
- "traefik.http.routers.connector.middlewares=connector_strip,auth" - "traefik.http.routers.connector.middlewares=connector_strip,auth-nngm"
volumes: volumes:
- nngm-rest:/var/log - nngm-rest:/var/log
traefik:
labels:
- "traefik.http.middlewares.auth-nngm.basicauth.users=${NNGM_AUTH}"

View File

@ -1,8 +1,4 @@
#!/bin/bash #!/bin/bash
##nNGM vars:
#NNGM_MAGICPL_APIKEY
#NNGM_CTS_APIKEY
#NNGM_CRYPTKEY
function nngmSetup() { function nngmSetup() {
if [ -n "$NNGM_CTS_APIKEY" ]; then if [ -n "$NNGM_CTS_APIKEY" ]; then

View File

@ -6,6 +6,8 @@ FOCUS_RETRY_COUNT=32
SUPPORT_EMAIL=support-ccp@dkfz-heidelberg.de SUPPORT_EMAIL=support-ccp@dkfz-heidelberg.de
PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem
BROKER_URL_FOR_PREREQ=$BROKER_URL
# This will load id-management setup. Effective only if id-management configuration is defined. # This will load id-management setup. Effective only if id-management configuration is defined.
source $PROJECT/modules/id-management-setup.sh source $PROJECT/modules/id-management-setup.sh
idManagementSetup idManagementSetup

View File

@ -9,14 +9,6 @@ detectCompose() {
fi fi
} }
getLdmPassword() {
if [ -n "$LDM_PASSWORD" ]; then
docker run --rm docker.verbis.dkfz.de/cache/httpd:alpine htpasswd -nb $PROJECT $LDM_PASSWORD | tr -d '\n' | tr -d '\r'
else
echo -n ""
fi
}
exitIfNotRoot() { exitIfNotRoot() {
if [ "$EUID" -ne 0 ]; then if [ "$EUID" -ne 0 ]; then
log "ERROR" "Please run as root" log "ERROR" "Please run as root"
@ -34,7 +26,7 @@ checkOwner(){
} }
printUsage() { printUsage() {
echo "Usage: bridgehead start|stop|is-running|update|install|uninstall|enroll PROJECTNAME" echo "Usage: bridgehead start|stop|is-running|update|install|uninstall|adduser|enroll PROJECTNAME"
echo "PROJECTNAME should be one of ccp|bbmri" echo "PROJECTNAME should be one of ccp|bbmri"
} }
@ -180,6 +172,43 @@ function bk_is_running {
fi fi
} }
##Setting Network properties function do_enroll_inner {
# currently not needed PARAMS=""
#export HOSTIP=$(MSYS_NO_PATHCONV=1 docker run --rm --add-host=host.docker.internal:host-gateway ubuntu cat /etc/hosts | grep 'host.docker.internal' | awk '{print $1}');
MANUAL_PROXY_ID="${1:-$PROXY_ID}"
if [ -z "$MANUAL_PROXY_ID" ]; then
log ERROR "No Proxy ID set"
exit 1
else
log INFO "Enrolling Beam Proxy Id $MANUAL_PROXY_ID"
fi
SUPPORT_EMAIL="${2:-$SUPPORT_EMAIL}"
if [ -n "$SUPPORT_EMAIL" ]; then
PARAMS+="--admin-email $SUPPORT_EMAIL"
fi
docker run --rm -ti -v /etc/bridgehead/pki:/etc/bridgehead/pki samply/beam-enroll:latest --output-file $PRIVATEKEYFILENAME --proxy-id $MANUAL_PROXY_ID $PARAMS
chmod 600 $PRIVATEKEYFILENAME
}
function do_enroll {
do_enroll_inner $@
}
add_basic_auth_user() {
USER="${1}"
PASSWORD="${2}"
NAME="${3}"
PROJECT="${4}"
FILE="/etc/bridgehead/${PROJECT}.local.conf"
ENCRY_CREDENTIALS="$(docker run --rm docker.verbis.dkfz.de/cache/httpd:alpine htpasswd -nb $USER $PASSWORD | tr -d '\n' | tr -d '\r')"
if [ -f $FILE ] && grep -R -q "$NAME=" $FILE # if a specific basic auth user already exists:
then
sed -i "/$NAME/ s|='|='$ENCRY_CREDENTIALS,|" $FILE
else
echo -e "\n## Basic Authentication Credentials for:\n$NAME='$ENCRY_CREDENTIALS'" >> $FILE;
fi
log DEBUG "Saving clear text credentials in $FILE. If wanted, delete them manually."
sed -i "/^$NAME/ s|$|\n# User: $USER\n# Password: $PASSWORD|" $FILE
}

View File

@ -29,12 +29,16 @@ bridgehead ALL= NOPASSWD: BRIDGEHEAD${PROJECT^^}
EOF EOF
# TODO: Determine whether this should be located in setup-bridgehead (triggered through bridgehead install) or in update bridgehead (triggered every hour) # TODO: Determine whether this should be located in setup-bridgehead (triggered through bridgehead install) or in update bridgehead (triggered every hour)
if [ -z "$LDM_PASSWORD" ]; then if [ -z "$LDM_AUTH" ]; then
log "INFO" "Now generating a password for the local data management. Please save the password for your ETL process!" log "INFO" "Now generating basic auth for the local data management (see adduser in bridgehead for more information). "
generated_passwd="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 32)" generated_passwd="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 32)"
add_basic_auth_user $PROJECT $generated_passwd "LDM_AUTH" $PROJECT
fi
log "INFO" "Your generated credentials are:\n user: $PROJECT\n password: $generated_passwd" if [ ! -z "$NNGM_CTS_APIKEY" ] && [ -z "$NNGM_AUTH" ]; then
echo -e "## Local Data Management Basic Authentication\n# User: $PROJECT\nLDM_PASSWORD=$generated_passwd" >> /etc/bridgehead/${PROJECT}.local.conf; log "INFO" "Now generating basic auth for nNGM upload API (see adduser in bridgehead for more information). "
generated_passwd="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 32)"
add_basic_auth_user "nngm" $generated_passwd "NNGM_AUTH" $PROJECT
fi fi
log "INFO" "Registering system units for bridgehead and bridgehead-update" log "INFO" "Registering system units for bridgehead and bridgehead-update"

View File

@ -71,8 +71,12 @@ if [ -d /etc/bridgehead ]; then
fi fi
elif [[ "$DEV_MODE" == "NODEV" ]]; then elif [[ "$DEV_MODE" == "NODEV" ]]; then
log "INFO" "Now cloning your site configuration repository for you." log "INFO" "Now cloning your site configuration repository for you."
read -p "Please enter your site: " site if [ -z "$site" ]; then
read -s -p "Please enter the bridgehead's access token for your site configuration repository (will not be echoed): " access_token read -p "Please enter your site: " site
fi
if [ -z "$access_token" ]; then
read -s -p "Please enter the bridgehead's access token for your site configuration repository (will not be echoed): " access_token
fi
site_configuration_repository_url="https://bytoken:${access_token}@${site_configuration_repository_middle}$(echo $site | tr '[:upper:]' '[:lower:]').git" site_configuration_repository_url="https://bytoken:${access_token}@${site_configuration_repository_middle}$(echo $site | tr '[:upper:]' '[:lower:]').git"
git clone $site_configuration_repository_url /etc/bridgehead git clone $site_configuration_repository_url /etc/bridgehead
if [ $? -gt 0 ]; then if [ $? -gt 0 ]; then

View File

@ -62,6 +62,34 @@ if [ -e /etc/bridgehead/vault.conf ]; then
fi fi
fi fi
log INFO "Checking network access ($BROKER_URL_FOR_PREREQ) ..."
source /etc/bridgehead/${PROJECT}.conf
source ${PROJECT}/vars
set +e
SERVERTIME="$(https_proxy=$HTTPS_PROXY_URL curl -m 5 -s -I $BROKER_URL_FOR_PREREQ 2>&1 | grep -i -e '^Date: ' | sed -e 's/^Date: //i')"
RET=$?
set -e
if [ $RET -ne 0 ]; then
log WARN "Unable to connect to Samply.Beam broker at $BROKER_URL_FOR_PREREQ. Please check your proxy settings.\nThe currently configured proxy was \"$HTTPS_PROXY_URL\". This error is normal when using proxy authentication."
log WARN "Unable to check clock skew due to previous error."
else
log INFO "Checking clock skew ..."
SERVERTIME_AS_TIMESTAMP=$(date --date="$SERVERTIME" +%s)
MYTIME=$(date +%s)
SKEW=$(($SERVERTIME_AS_TIMESTAMP - $MYTIME))
SKEW=$(echo $SKEW | awk -F- '{print $NF}')
SYNCTEXT="For example, consider entering a correct NTP server (e.g. your institution's Active Directory Domain Controller in /etc/systemd/timesyncd.conf (option NTP=) and restart systemd-timesyncd."
if [ $SKEW -ge 300 ]; then
report_error 5 "Your clock is not synchronized (${SKEW}s off). This will cause Samply.Beam's certificate will fail. Please setup time synchronization. $SYNCTEXT"
exit 1
elif [ $SKEW -ge 60 ]; then
log WARN "Your clock is more than a minute off (${SKEW}s). Consider syncing to a time server. $SYNCTEXT"
fi
fi
checkPrivKey() { checkPrivKey() {
if [ -e /etc/bridgehead/pki/${SITE_ID}.priv.pem ]; then if [ -e /etc/bridgehead/pki/${SITE_ID}.priv.pem ]; then
log INFO "Success - private key found." log INFO "Success - private key found."
@ -69,8 +97,6 @@ checkPrivKey() {
log ERROR "Unable to find private key at /etc/bridgehead/pki/${SITE_ID}.priv.pem. To fix, please run\n bridgehead enroll ${PROJECT}\nand follow the instructions." log ERROR "Unable to find private key at /etc/bridgehead/pki/${SITE_ID}.priv.pem. To fix, please run\n bridgehead enroll ${PROJECT}\nand follow the instructions."
return 1 return 1
fi fi
log INFO "Success - all prerequisites are met!"
hc_send log "Success - all prerequisites are met!"
return 0 return 0
} }
@ -80,4 +106,7 @@ else
checkPrivKey || exit 1 checkPrivKey || exit 1
fi fi
log INFO "Success - all prerequisites are met!"
hc_send log "Success - all prerequisites are met!"
exit 0 exit 0

View File

@ -139,6 +139,15 @@ else
log WARN "Automated backups are disabled (variable AUTO_BACKUPS != \"true\")" log WARN "Automated backups are disabled (variable AUTO_BACKUPS != \"true\")"
fi fi
#TODO: the following block can be deleted after successful update at all sites
if [ ! -z "$LDM_PASSWORD" ]; then
FILE="/etc/bridgehead/$PROJECT.local.conf"
log "INFO" "Migrating LDM_PASSWORD to encrypted credentials in $FILE"
add_basic_auth_user $PROJECT $LDM_PASSWORD "LDM_AUTH" $PROJECT
add_basic_auth_user $PROJECT $LDM_PASSWORD "NNGM_AUTH" $PROJECT
sed -i "/LDM_PASSWORD/{d;}" $FILE
fi
exit 0 exit 0
# TODO: Print last commit explicit # TODO: Print last commit explicit

View File

@ -21,7 +21,7 @@ services:
- "traefik.http.routers.dashboard.service=api@internal" - "traefik.http.routers.dashboard.service=api@internal"
- "traefik.http.routers.dashboard.tls=true" - "traefik.http.routers.dashboard.tls=true"
- "traefik.http.routers.dashboard.middlewares=auth" - "traefik.http.routers.dashboard.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=${LDM_LOGIN}" - "traefik.http.middlewares.auth.basicauth.users=${LDM_AUTH}"
ports: ports:
- 80:80 - 80:80
- 443:443 - 443:443
@ -37,6 +37,9 @@ services:
HTTPS_PROXY: ${HTTPS_PROXY_URL} HTTPS_PROXY: ${HTTPS_PROXY_URL}
USERNAME: ${HTTPS_PROXY_USERNAME} USERNAME: ${HTTPS_PROXY_USERNAME}
PASSWORD: ${HTTPS_PROXY_PASSWORD} PASSWORD: ${HTTPS_PROXY_PASSWORD}
tmpfs:
- /var/log/squid
- /var/spool/squid
volumes: volumes:
- /etc/bridgehead/trusted-ca-certs:/docker/custom-certs/:ro - /etc/bridgehead/trusted-ca-certs:/docker/custom-certs/:ro