Merge branch 'main' into dktk-migration

README.md

@@ -8,7 +8,6 @@ This repository is the starting point for any information and tools you will nee
     - [Hardware](#hardware)
     - [Software](#software)
     - [Network](#network)
-    - [Register with the Directory](#register-with-the-directory)
 2. [Deployment](#deployment)
     - [Site name](#site-name)
     - [Projects](#projects)
@@ -22,6 +21,7 @@ This repository is the starting point for any information and tools you will nee
     - [HTTPS Access](#https-access)
     - [TLS terminating proxies](#tls-terminating-proxies)
     - [File structure](#file-structure)
+    - [BBMRI-ERIC Directory](#bbmri-eric-directory)
 4. [Things you should know](#things-you-should-know)
     - [Auto-Updates](#auto-updates)
     - [Auto-Backups](#auto-backups)
@@ -64,18 +64,10 @@ Additionally, your site might use its own proxy. You should discuss this with yo
 
 Note that git and Docker may also need to be configured to use this proxy. This is a job for your systems administrators.
 
-If there is a site firewall, this needs to be configured so that git and Docker can reach the outside world. Another job for the systems administrators.
+If there is a site firewall, this needs to be configured so that outgoing calls to the following URLs are allowed: *.dkfz.de, github.com, docker.io, *.docker.io, *.samply.de.
 
 Note for Ubuntu: Please note that the uncomplicated firewall (ufw) is known to conflict with Docker [here](https://github.com/chaifeng/ufw-docker).
 
-### Register with the Directory
-
-If you run a biobank, you should register with the [Directory](https://directory.bbmri-eric.eu), a BBMRI project that catalogs biobanks.
-
-To do this, contact the BBMRI national node for the country where your biobank is based, see [the list of nodes](http://www.bbmri-eric.eu/national-nodes/).
-
-Once you have registered, **you should choose one of your sample collections as a default collection for your biobank**. This is the collection that will be automatically used to label any samples that have not been assigned a collection ID in your ETL process. Make a note of this ID, you will need it later on in the installation process.
-
 ## Deployment
 
 ### Site name
@@ -255,9 +247,15 @@ All of the Bridgehead's outgoing connections are secured by transport encryption
 
 Your Bridgehead's actual data is not stored in the above directories, but in named docker volumes, see `docker volume ls` and `docker volume inspect <volume_name>`.
 
-### Directory sync
+### BBMRI-ERIC Directory
 
-This is an optional feature for bbmri projects. It keeps the [BBMRI Directory](https://directory.bbmri-eric.eu/) up to date with your local data eg. number of samples. It also updates the local FHIR store with the latest contact details etc. from the Directory.  You must explicitly set your country specific directory url, username and password to enable this feature.
+If you run a biobank, you should register with the [Directory](https://directory.bbmri-eric.eu), a BBMRI-ERIC project that catalogs biobanks.
+
+To do this, contact the BBMRI-ERIC national node for the country where your biobank is based, see [the list of nodes](http://www.bbmri-eric.eu/national-nodes/).
+
+Once you have registered, **you should choose one of your sample collections as a default collection for your biobank**. This is the collection that will be automatically used to label any samples that have not been assigned a collection ID in your ETL process. Make a note of this ID, you will need it later on in the installation process.
+
+The Bridgehead's **Directory Sync** is an optional feature that keeps the Directory up to date with your local data, e.g. number of samples. Conversely, it also updates the local FHIR store with the latest contact details etc. from the Directory. You must explicitly set your country specific directory url, username and password to enable this feature.
 
 Full details can be found in [directory_sync_service](https://github.com/samply/directory_sync_service).
 
@@ -274,7 +272,7 @@ You must contact the Directory for your national node to find the URL, and to re
 
 Additionally, you should choose when you want Directory sync to run. In the example above, this is set to happen at 10 pm every evening. You can modify this to suit your requirements. The timer specification should follow the [cron](https://crontab.guru) convention.
 
-Once you edited the gitlab config. The bridgehead will autoupdate the config with the values and will sync the data.
+Once you edited the gitlab config, the bridgehead will autoupdate the config with the values and will sync the data.
 
 There will be a delay before the effects of Directory sync become visible. First, you will need to wait until the time you have specified in ```TIMER_CRON```. Second, the information will then be synchronized from your national node with the central European Directory. This can take up to 24 hours.
 

bbmri/directory-sync.sh

@@ -1,8 +0,0 @@
-#!/bin/bash
-
-function dirSetup() {
-	if [ -n "$DS_DIRECTORY_USER_NAME" ]; then
-		log INFO "Directory sync setup detected -- will start directory sync service."
-		OVERRIDE+=" -f ./$PROJECT/directory-sync-compose.yml"
-	fi
-}

bbmri/docker-compose.yml

@@ -1,5 +1,7 @@
 version: "3.7"
 
+# This includes only the shared persistence for BBMRI-ERIC and GBN. Federation components are included as modules, see vars.
+
 services:
   blaze:
     image: docker.verbis.dkfz.de/cache/samply/blaze:latest
@@ -19,42 +21,10 @@ services:
       - "traefik.http.routers.blaze_ccp.middlewares=ccp_b_strip,auth"
       - "traefik.http.routers.blaze_ccp.tls=true"
 
-  focus:
-    image: docker.verbis.dkfz.de/cache/samply/focus:main
-    container_name: bridgehead-focus
-    environment:
-      API_KEY: ${FOCUS_BEAM_SECRET_SHORT}
-      BEAM_APP_ID_LONG: focus.${PROXY_ID}
-      PROXY_ID: ${PROXY_ID}
-      BLAZE_URL: "http://bridgehead-bbmri-blaze:8080/fhir/"
-      BEAM_PROXY_URL: http://beam-proxy:8081
-      RETRY_COUNT: ${FOCUS_RETRY_COUNT}
-    depends_on:
-      - "beam-proxy"
-      - "blaze"
-
-  beam-proxy:
-    image: docker.verbis.dkfz.de/cache/samply/beam-proxy:develop
-    container_name: bridgehead-beam-proxy
-    environment:
-      BROKER_URL: ${BROKER_URL}
-      PROXY_ID: ${PROXY_ID}
-      APP_focus_KEY: ${FOCUS_BEAM_SECRET_SHORT}
-      PRIVKEY_FILE: /run/secrets/proxy.pem
-      ALL_PROXY: http://forward_proxy:3128
-      TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
-      ROOTCERT_FILE: /conf/root.crt.pem
-    secrets:
-      - proxy.pem
-    depends_on:
-      - "forward_proxy"
-    volumes:
-      - /etc/bridgehead/trusted-ca-certs:/conf/trusted-ca-certs:ro
-      - /srv/docker/bridgehead/bbmri/root.crt.pem:/conf/root.crt.pem:ro
-
 volumes:
   blaze-data:
 
+# used in modules *-locator.yml
 secrets:
   proxy.pem:
     file: /etc/bridgehead/pki/${SITE_ID}.priv.pem

bbmri/modules/directory-sync.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+if [ -n "${DS_DIRECTORY_USER_NAME}" ]; then
+	log INFO "Directory sync setup detected -- will start directory sync service."
+	OVERRIDE+=" -f ./$PROJECT/modules/directory-sync-compose.yml"
+fi

bbmri/modules/dnpm-setup.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 if [ -n "${ENABLE_DNPM}" ]; then
-	log DEBUG "DNPM setup detected (Beam.Connect) -- will start Beam and Beam.Connect for DNPM."
+	log INFO "DNPM setup detected (Beam.Connect) -- will start Beam and Beam.Connect for DNPM."
 	OVERRIDE+=" -f ./$PROJECT/modules/dnpm-compose.yml"
 
 	# Set variables required for Beam-Connect

bbmri/modules/eric-compose.yml

@@ -0,0 +1,36 @@
+version: "3.7"
+
+services:
+  focus-eric:
+    image: docker.verbis.dkfz.de/cache/samply/focus:main
+    container_name: bridgehead-focus-eric
+    environment:
+      API_KEY: ${ERIC_FOCUS_BEAM_SECRET_SHORT}
+      BEAM_APP_ID_LONG: focus.${ERIC_PROXY_ID}
+      PROXY_ID: ${ERIC_PROXY_ID}
+      BLAZE_URL: "http://blaze:8080/fhir/"
+      BEAM_PROXY_URL: http://beam-proxy-eric:8081
+      RETRY_COUNT: ${FOCUS_RETRY_COUNT}
+    depends_on:
+      - "beam-proxy-eric"
+      - "blaze"
+
+  beam-proxy-eric:
+    image: docker.verbis.dkfz.de/cache/samply/beam-proxy:develop
+    container_name: bridgehead-beam-proxy-eric
+    environment:
+      BROKER_URL: ${ERIC_BROKER_URL}
+      PROXY_ID: ${ERIC_PROXY_ID}
+      APP_focus_KEY: ${ERIC_FOCUS_BEAM_SECRET_SHORT}
+      PRIVKEY_FILE: /run/secrets/proxy.pem
+      ALL_PROXY: http://forward_proxy:3128
+      TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
+      ROOTCERT_FILE: /conf/root.crt.pem
+    secrets:
+      - proxy.pem
+    depends_on:
+      - "forward_proxy"
+    volumes:
+      - /etc/bridgehead/trusted-ca-certs:/conf/trusted-ca-certs:ro
+      - /srv/docker/bridgehead/bbmri/modules/eric.root.crt.pem:/conf/root.crt.pem:ro
+

bbmri/modules/eric-setup.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+
+if [ "${ENABLE_ERIC}" == "true" ]; then
+	log INFO "BBMRI-ERIC setup detected -- will start services for BBMRI-ERIC."
+	OVERRIDE+=" -f ./$PROJECT/modules/eric-compose.yml"
+
+	# Set required variables
+	ERIC_BROKER_ID=broker.bbmri.samply.de
+	ERIC_BROKER_URL=https://${ERIC_BROKER_ID}
+	ERIC_PROXY_ID=${SITE_ID}.${ERIC_BROKER_ID}
+	ERIC_FOCUS_BEAM_SECRET_SHORT="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)"
+	ERIC_SUPPORT_EMAIL=bridgehead@helpdesk.bbmri-eric.eu
+fi

bbmri/modules/gbn-compose.yml

@@ -0,0 +1,36 @@
+version: "3.7"
+
+services:
+  focus-gbn:
+    image: docker.verbis.dkfz.de/cache/samply/focus:main
+    container_name: bridgehead-focus-gbn
+    environment:
+      API_KEY: ${GBN_FOCUS_BEAM_SECRET_SHORT}
+      BEAM_APP_ID_LONG: focus.${GBN_PROXY_ID}
+      PROXY_ID: ${GBN_PROXY_ID}
+      BLAZE_URL: "http://blaze:8080/fhir/"
+      BEAM_PROXY_URL: http://beam-proxy-gbn:8081
+      RETRY_COUNT: ${FOCUS_RETRY_COUNT}
+    depends_on:
+      - "beam-proxy-gbn"
+      - "blaze"
+
+  beam-proxy-gbn:
+    image: docker.verbis.dkfz.de/cache/samply/beam-proxy:develop
+    container_name: bridgehead-beam-proxy-gbn
+    environment:
+      BROKER_URL: ${GBN_BROKER_URL}
+      PROXY_ID: ${GBN_PROXY_ID}
+      APP_focus_KEY: ${GBN_FOCUS_BEAM_SECRET_SHORT}
+      PRIVKEY_FILE: /run/secrets/proxy.pem
+      ALL_PROXY: http://forward_proxy:3128
+      TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
+      ROOTCERT_FILE: /conf/root.crt.pem
+    secrets:
+      - proxy.pem
+    depends_on:
+      - "forward_proxy"
+    volumes:
+      - /etc/bridgehead/trusted-ca-certs:/conf/trusted-ca-certs:ro
+      - /srv/docker/bridgehead/bbmri/modules/gbn.root.crt.pem:/conf/root.crt.pem:ro
+

bbmri/modules/gbn-setup.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+
+if [ "${ENABLE_GBN}" == "true" ]; then
+	log INFO "GBN setup detected -- will start services for German Biobank Node."
+	OVERRIDE+=" -f ./$PROJECT/modules/gbn-compose.yml"
+
+	# Set required variables
+	GBN_BROKER_ID='#TODO#'
+	GBN_BROKER_URL=https://${GBN_BROKER_ID}
+	GBN_PROXY_ID=${SITE_ID}.${GBN_BROKER_ID}
+	GBN_FOCUS_BEAM_SECRET_SHORT="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)"
+	GBN_SUPPORT_EMAIL=todo@verbis.dkfz.de
+fi

bbmri/modules/gbn.root.crt.pem

@@ -0,0 +1 @@
+#TODO#

bbmri/vars

@@ -1,9 +1,10 @@
-BROKER_ID=broker.bbmri.samply.de
+# Makes sense for all European Biobanks
-BROKER_URL=https://${BROKER_ID}
+: ${ENABLE_ERIC:=true}
-PROXY_ID=${SITE_ID}.${BROKER_ID}
+
-FOCUS_BEAM_SECRET_SHORT="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)"
+# Makes only sense for German Biobanks
+: ${ENABLE_GBN:=false}
+
 FOCUS_RETRY_COUNT=32
-SUPPORT_EMAIL=bridgehead@helpdesk.bbmri-eric.eu
 PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem
 
 for module in $PROJECT/modules/*.sh
@@ -12,6 +13,25 @@ do
     source $module
 done
 
-# This will load directory-sync setup.
+SUPPORT_EMAIL=$ERIC_SUPPORT_EMAIL
-source $PROJECT/directory-sync.sh
+BROKER_URL_FOR_PREREQ=$ERIC_BROKER_URL
-dirSetup
+
+if [ -n "$GBN_SUPPORT_EMAIL" ]; then
+    SUPPORT_EMAIL=$GBN_SUPPORT_EMAIL
+fi
+
+function do_enroll {
+    COUNT=0
+    if [ "$ENABLE_ERIC" == "true" ]; then
+        do_enroll_inner $ERIC_PROXY_ID $ERIC_SUPPORT_EMAIL
+        COUNT=$((COUNT+1))
+    fi
+    if [ "$ENABLE_GBN" == "true" ]; then
+        do_enroll_inner $GBN_PROXY_ID $GBN_SUPPORT_EMAIL
+        COUNT=$((COUNT+1))
+    fi
+    if [ $COUNT -ge 2 ]; then
+        echo
+        echo "You just received $COUNT certificate signing requests (CSR). Please send $COUNT e-mails, with 1 CSR each, to the respective e-mail address."
+    fi
+}

bridgehead

@@ -73,7 +73,6 @@ case "$ACTION" in
 		hc_send log "Bridgehead $PROJECT startup: Checking requirements ..."
 		checkRequirements
 		hc_send log "Bridgehead $PROJECT startup: Requirements checked out. Now starting bridgehead ..."
-		export LDM_LOGIN=$(getLdmPassword)
 		exec $COMPOSE -p $PROJECT -f ./minimal/docker-compose.yml -f ./$PROJECT/docker-compose.yml $OVERRIDE up --abort-on-container-exit
 		;;
 	stop)
@@ -103,25 +102,17 @@ case "$ACTION" in
 	uninstall)
 		exec ./lib/uninstall-bridgehead.sh $PROJECT
 		;;
+	adduser)
+		loadVars
+		log "INFO" "Adding encrypted credentials in /etc/bridgehead/$PROJECT.local.conf"
+    		read -p "Please choose the component (LDM_AUTH|NNGM_AUTH) you want to add a user to : " COMPONENT
+    		read -p "Please enter a username: " USER
+    		read -s -p "Please enter a password (will not be echoed): "$'\n' PASSWORD
+    		add_basic_auth_user $USER $PASSWORD $COMPONENT $PROJECT
+		;;
 	enroll)
 		loadVars
-
+		do_enroll $PROXY_ID
-		MANUAL_PROXY_ID="${3:-$PROXY_ID}"
-		if [ -z "$MANUAL_PROXY_ID" ]; then
-			log ERROR "No Proxy ID set"
-			exit 1
-		else
-			log INFO "Enrolling Beam Proxy Id $MANUAL_PROXY_ID"
-		fi
-
-		if [ -z "$SUPPORT_EMAIL" ]; then
-			EMAIL_PARAM=""
-		else
-			EMAIL_PARAM="--admin-email $SUPPORT_EMAIL"
-		fi
-			
-		docker run --rm -ti -v /etc/bridgehead/pki:/etc/bridgehead/pki docker.verbis.dkfz.de/cache/samply/beam-enroll:latest --output-file $PRIVATEKEYFILENAME --proxy-id $MANUAL_PROXY_ID $EMAIL_PARAM
-		chmod 600 $PRIVATEKEYFILENAME
 		;;
 	preRun | preUpdate)
 		fixPermissions

ccp/modules/dnpm-setup.sh

@@ -1,7 +1,7 @@
 #!/bin/bash
 
 if [ -n "${ENABLE_DNPM}" ]; then
-	log DEBUG "DNPM setup detected (Beam.Connect) -- will start Beam.Connect for DNPM."
+	log INFO "DNPM setup detected (Beam.Connect) -- will start Beam.Connect for DNPM."
 	OVERRIDE+=" -f ./$PROJECT/modules/dnpm-compose.yml"
 
 	# Set variables required for Beam-Connect

ccp/modules/id-management-compose.yml

@@ -5,6 +5,7 @@ services:
     container_name: bridgehead-id-manager
     environment:
       TOMCAT_REVERSEPROXY_FQDN: ${HOST}
+      TOMCAT_REVERSEPROXY_SSL: "true"
       MAGICPL_SITE: ${IDMANAGEMENT_FRIENDLY_ID}
       MAGICPL_ALLOWED_ORIGINS: https://${HOST}
       MAGICPL_LOCAL_PATIENTLIST_APIKEY: ${IDMANAGER_LOCAL_PATIENTLIST_APIKEY}

ccp/nngm-compose.yml

@@ -18,7 +18,12 @@ services:
       - "traefik.http.middlewares.connector_strip.stripprefix.prefixes=/nngm-connector"
       - "traefik.http.services.connector.loadbalancer.server.port=8080"
       - "traefik.http.routers.connector.tls=true"
-      - "traefik.http.routers.connector.middlewares=connector_strip,auth"
+      - "traefik.http.routers.connector.middlewares=connector_strip,auth-nngm"
     volumes:
       - nngm-rest:/var/log
 
+  traefik:
+    labels:
+      - "traefik.http.middlewares.auth-nngm.basicauth.users=${NNGM_AUTH}"
+
+

ccp/nngm-setup.sh

@@ -1,8 +1,4 @@
 #!/bin/bash
-##nNGM vars:
-#NNGM_MAGICPL_APIKEY
-#NNGM_CTS_APIKEY
-#NNGM_CRYPTKEY
 
 function nngmSetup() {
 	if [ -n "$NNGM_CTS_APIKEY" ]; then

ccp/vars

@@ -6,6 +6,8 @@ FOCUS_RETRY_COUNT=32
 SUPPORT_EMAIL=support-ccp@dkfz-heidelberg.de
 PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem
 
+BROKER_URL_FOR_PREREQ=$BROKER_URL
+
 # This will load id-management setup. Effective only if id-management configuration is defined.
 source $PROJECT/modules/id-management-setup.sh
 idManagementSetup

lib/functions.sh

@@ -9,14 +9,6 @@ detectCompose() {
 	fi
 }
 
-getLdmPassword() {
-	if [ -n "$LDM_PASSWORD" ]; then
-		docker run --rm docker.verbis.dkfz.de/cache/httpd:alpine htpasswd -nb $PROJECT $LDM_PASSWORD | tr -d '\n' | tr -d '\r'
-	else
-		echo -n ""
-	fi
-}
-
 exitIfNotRoot() {
   if [ "$EUID" -ne 0 ]; then
     log "ERROR" "Please run as root"
@@ -34,7 +26,7 @@ checkOwner(){
 }
 
 printUsage() {
-	echo "Usage: bridgehead start|stop|is-running|update|install|uninstall|enroll PROJECTNAME"
+	echo "Usage: bridgehead start|stop|is-running|update|install|uninstall|adduser|enroll PROJECTNAME"
 	echo "PROJECTNAME should be one of ccp|bbmri"
 }
 
@@ -180,6 +172,43 @@ function bk_is_running {
 	fi
 }
 
-##Setting Network properties
+function do_enroll_inner {
-# currently not needed
+	PARAMS=""
-#export HOSTIP=$(MSYS_NO_PATHCONV=1 docker run --rm --add-host=host.docker.internal:host-gateway ubuntu cat /etc/hosts | grep 'host.docker.internal' | awk '{print $1}');
+	
+	MANUAL_PROXY_ID="${1:-$PROXY_ID}"
+	if [ -z "$MANUAL_PROXY_ID" ]; then
+		log ERROR "No Proxy ID set"
+		exit 1
+	else
+		log INFO "Enrolling Beam Proxy Id $MANUAL_PROXY_ID"
+	fi
+
+	SUPPORT_EMAIL="${2:-$SUPPORT_EMAIL}"
+	if [ -n "$SUPPORT_EMAIL" ]; then
+		PARAMS+="--admin-email $SUPPORT_EMAIL"
+	fi
+
+	docker run --rm -ti -v /etc/bridgehead/pki:/etc/bridgehead/pki samply/beam-enroll:latest --output-file $PRIVATEKEYFILENAME --proxy-id $MANUAL_PROXY_ID $PARAMS
+	chmod 600 $PRIVATEKEYFILENAME
+}
+
+function do_enroll {
+	do_enroll_inner $@
+}
+
+add_basic_auth_user() {
+   USER="${1}"
+   PASSWORD="${2}"
+   NAME="${3}"
+   PROJECT="${4}"
+   FILE="/etc/bridgehead/${PROJECT}.local.conf"
+   ENCRY_CREDENTIALS="$(docker run --rm docker.verbis.dkfz.de/cache/httpd:alpine htpasswd -nb $USER $PASSWORD  | tr -d '\n' | tr -d '\r')"
+   if [ -f $FILE ] && grep -R -q "$NAME=" $FILE # if a specific basic auth user already exists:
+   then
+     sed -i "/$NAME/ s|='|='$ENCRY_CREDENTIALS,|" $FILE
+   else
+     echo -e "\n## Basic Authentication Credentials for:\n$NAME='$ENCRY_CREDENTIALS'" >> $FILE;
+   fi
+ 	log DEBUG "Saving clear text credentials in $FILE. If wanted, delete them manually."
+   sed -i "/^$NAME/ s|$|\n# User: $USER\n# Password: $PASSWORD|" $FILE
+}

lib/install-bridgehead.sh

@@ -29,12 +29,16 @@ bridgehead ALL= NOPASSWD: BRIDGEHEAD${PROJECT^^}
 EOF
 
 # TODO: Determine whether this should be located in setup-bridgehead (triggered through bridgehead install) or in update bridgehead (triggered every hour)
-if [ -z "$LDM_PASSWORD" ]; then
+if [ -z "$LDM_AUTH" ]; then
-  log "INFO" "Now generating a password for the local data management. Please save the password for your ETL process!"
+  log "INFO" "Now generating basic auth for the local data management (see adduser in bridgehead for more information). "
   generated_passwd="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 32)"
+  add_basic_auth_user $PROJECT $generated_passwd "LDM_AUTH" $PROJECT
+fi
 
-  log "INFO" "Your generated credentials are:\n            user: $PROJECT\n            password: $generated_passwd"
+if [ ! -z "$NNGM_CTS_APIKEY" ] && [ -z "$NNGM_AUTH" ]; then
-  echo -e "## Local Data Management Basic Authentication\n# User: $PROJECT\nLDM_PASSWORD=$generated_passwd" >> /etc/bridgehead/${PROJECT}.local.conf;
+  log "INFO" "Now generating basic auth for nNGM upload API (see adduser in bridgehead for more information). "
+  generated_passwd="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 32)"
+  add_basic_auth_user "nngm" $generated_passwd "NNGM_AUTH" $PROJECT
 fi
 
 log "INFO" "Registering system units for bridgehead and bridgehead-update"

lib/prepare-system.sh

@@ -71,8 +71,12 @@ if [ -d /etc/bridgehead ]; then
     fi
 elif [[ "$DEV_MODE" == "NODEV" ]]; then
     log "INFO" "Now cloning your site configuration repository for you."
-    read -p "Please enter your site: " site
+    if [ -z "$site" ]; then
-    read -s -p "Please enter the bridgehead's access token for your site configuration repository (will not be echoed): " access_token
+        read -p "Please enter your site: " site
+    fi
+    if [ -z "$access_token" ]; then
+        read -s -p "Please enter the bridgehead's access token for your site configuration repository (will not be echoed): " access_token
+    fi
     site_configuration_repository_url="https://bytoken:${access_token}@${site_configuration_repository_middle}$(echo $site | tr '[:upper:]' '[:lower:]').git"
     git clone $site_configuration_repository_url /etc/bridgehead
     if [ $? -gt 0 ]; then

lib/prerequisites.sh

@@ -62,6 +62,34 @@ if [ -e /etc/bridgehead/vault.conf ]; then
   fi
 fi
 
+log INFO "Checking network access ($BROKER_URL_FOR_PREREQ) ..."
+
+source /etc/bridgehead/${PROJECT}.conf
+source ${PROJECT}/vars
+
+set +e
+SERVERTIME="$(https_proxy=$HTTPS_PROXY_URL curl -m 5 -s -I $BROKER_URL_FOR_PREREQ 2>&1 | grep -i -e '^Date: ' | sed -e 's/^Date: //i')"
+RET=$?
+set -e
+if [ $RET -ne 0 ]; then
+	log WARN "Unable to connect to Samply.Beam broker at $BROKER_URL_FOR_PREREQ. Please check your proxy settings.\nThe currently configured proxy was \"$HTTPS_PROXY_URL\". This error is normal when using proxy authentication."
+	log WARN "Unable to check clock skew due to previous error."
+else
+	log INFO "Checking clock skew ..."
+
+	SERVERTIME_AS_TIMESTAMP=$(date --date="$SERVERTIME" +%s)
+	MYTIME=$(date +%s)
+	SKEW=$(($SERVERTIME_AS_TIMESTAMP - $MYTIME))
+	SKEW=$(echo $SKEW | awk -F- '{print $NF}')
+	SYNCTEXT="For example, consider entering a correct NTP server (e.g. your institution's Active Directory Domain Controller in /etc/systemd/timesyncd.conf (option NTP=) and restart systemd-timesyncd."
+	if [ $SKEW -ge 300 ]; then
+		report_error 5 "Your clock is not synchronized (${SKEW}s off). This will cause Samply.Beam's certificate will fail. Please setup time synchronization. $SYNCTEXT"
+		exit 1
+	elif [ $SKEW -ge 60 ]; then
+		log WARN "Your clock is more than a minute off (${SKEW}s). Consider syncing to a time server. $SYNCTEXT"
+	fi
+fi
+
 checkPrivKey() {
   if [ -e /etc/bridgehead/pki/${SITE_ID}.priv.pem ]; then
     log INFO "Success - private key found."
@@ -69,8 +97,6 @@ checkPrivKey() {
     log ERROR "Unable to find private key at /etc/bridgehead/pki/${SITE_ID}.priv.pem. To fix, please run\n  bridgehead enroll ${PROJECT}\nand follow the instructions."
     return 1
   fi
-  log INFO "Success - all prerequisites are met!"
-  hc_send log "Success - all prerequisites are met!"
   return 0
 }
 
@@ -80,4 +106,7 @@ else
   checkPrivKey || exit 1
 fi
 
+log INFO "Success - all prerequisites are met!"
+hc_send log "Success - all prerequisites are met!"
+
 exit 0

lib/update-bridgehead.sh

@@ -139,6 +139,15 @@ else
   log WARN "Automated backups are disabled (variable AUTO_BACKUPS != \"true\")"
 fi
 
+#TODO: the following block can be deleted after successful update at all sites
+if [ ! -z "$LDM_PASSWORD" ]; then
+  FILE="/etc/bridgehead/$PROJECT.local.conf"
+  log "INFO" "Migrating LDM_PASSWORD to encrypted credentials in $FILE"
+  add_basic_auth_user $PROJECT $LDM_PASSWORD "LDM_AUTH" $PROJECT
+  add_basic_auth_user $PROJECT $LDM_PASSWORD "NNGM_AUTH" $PROJECT
+  sed -i "/LDM_PASSWORD/{d;}" $FILE
+fi
+
 exit 0
 
 # TODO: Print last commit explicit

minimal/docker-compose.yml

@@ -21,7 +21,7 @@ services:
       - "traefik.http.routers.dashboard.service=api@internal"
       - "traefik.http.routers.dashboard.tls=true"
       - "traefik.http.routers.dashboard.middlewares=auth"
-      - "traefik.http.middlewares.auth.basicauth.users=${LDM_LOGIN}"
+      - "traefik.http.middlewares.auth.basicauth.users=${LDM_AUTH}"
     ports:
       - 80:80
       - 443:443
@@ -37,6 +37,9 @@ services:
       HTTPS_PROXY: ${HTTPS_PROXY_URL}
       USERNAME: ${HTTPS_PROXY_USERNAME}
       PASSWORD: ${HTTPS_PROXY_PASSWORD}
+    tmpfs:
+      - /var/log/squid
+      - /var/spool/squid
     volumes:
       - /etc/bridgehead/trusted-ca-certs:/docker/custom-certs/:ro