Test

README.md

@@ -24,9 +24,6 @@ This repository is the starting point for any information and tools you will nee
     - [BBMRI-ERIC Directory entry needed](#bbmri-eric-directory-entry-needed)
     - [Directory sync tool](#directory-sync-tool)
     - [Loading data](#loading-data)
-    - [Teiler (Frontend)](#teiler-frontend)
-    - [Data Exporter Service](#data-exporter-service)
-      - [Data Quality Report](#data-quality-report)
 4. [Things you should know](#things-you-should-know)
     - [Auto-Updates](#auto-updates)
     - [Auto-Backups](#auto-backups)
@@ -382,36 +379,6 @@ Normally, you will need to build your own ETL to feed the Bridgehead. However, t
 
 You can find the profiles for generating FHIR in [Simplifier](https://simplifier.net/bbmri.de/~resources?category=Profile).
 
-### Teiler (Frontend)
-
-Teiler is the web-based frontend of the Bridgehead, providing access to its various internal, and external services and components.   
-To learn how to integrate your custom module into Teiler, please refer to https://github.com/samply/teiler-dashboard.
-- To activate Teiler, set the following environment variable in your `<PROJECT>.conf` file:  
-
-```bash
-ENABLE_TEILER=true
-```
-
-### Data Exporter Service
-
-The Exporter is a dedicated service for extracting and exporting Bridgehead data in (tabular) formats such as Excel, CSV, Opal, JSON, XML, ...  
-- To enable the Exporter service, set the following environment variable in your `<PROJECT>.conf` file:  
-
-```bash
-ENABLE_EXPORTER=true
-
-#### Data Quality Report
-To assess the quality and plausibility of your imported data, the Reporter component is pre-configured to generate Excel reports with data quality metrics and statistical analyses. Reporter is part of the Exporter and can be enabled by setting the same environment variable in your `<PROJECT>.conf` file:
-```bash
-ENABLE_EXPORTER=true
-```
-
-For convenience, it's recommended to enable the Teiler web frontend alongside the Exporter to access export and quality control features via a web interface: set the following environment varibles in your `<PROJECT>.conf` file:
-```bash
-ENABLE_TEILER=true
-ENABLE_EXPORTER=true
-```
-
 ## Things you should know
 
 ### Auto-Updates

bbmri/modules/eric.acc.root.crt.pem

@@ -1,20 +1,20 @@
 -----BEGIN CERTIFICATE-----
-MIIDNTCCAh2gAwIBAgIUFzdpDi1OLdXyogtCsktHFhCILtMwDQYJKoZIhvcNAQEL
+MIIDNTCCAh2gAwIBAgIUE/wu6FmI+KSMOalI65b+lI3HI4cwDQYJKoZIhvcNAQEL
-BQAwFjEUMBIGA1UEAxMLQnJva2VyLVJvb3QwHhcNMjUwNjEwMTQzNjE1WhcNMzUw
+BQAwFjEUMBIGA1UEAxMLQnJva2VyLVJvb3QwHhcNMjQwOTE2MTUyMzU0WhcNMzQw
-NjA4MTQzNjQ1WjAWMRQwEgYDVQQDEwtCcm9rZXItUm9vdDCCASIwDQYJKoZIhvcN
+OTE0MTUyNDI0WjAWMRQwEgYDVQQDEwtCcm9rZXItUm9vdDCCASIwDQYJKoZIhvcN
-AQEBBQADggEPADCCAQoCggEBALpJCWE9Qe19R9DqotdkPV6jfiuJSKI3UYkCWdWG
+AQEBBQADggEPADCCAQoCggEBAOt1I1FQt2bI4Nnjtg8JBYid29cBIkDT4MMb45Jr
-nRfkKB6OaY5t3JCHDqaEME9FwSd2nFXhTp5F6snG/K7g8MCLIEzGzuSnrdjGqINq
+ays24y4R3WO7VJK9UjNduSq/A1jlA0W0A/szDf8Ojq6bBtg+uL92PTDjYH1QXwX0
-zXLfgqnxvQpPR4ARLNNgnKxZaq7m4Q3T/l+QAshK6CnCUWFQ6q5x3g/pZHFP2USd
+c7eMo2tvvyyrs/cb2/ovDBQ1lpibcxVmVAv042ASmil3SdqKKXpv3ATnF9I7V4cv
-/G2FtDHX6YK4bHbbnigIPG6PdY2RYy60i30XGdIPBNf82XGkAtPUBz731gHOV5Vg
+fwB56FChaGIov5EK+9JOMjTx6oMlBEgUFR6qq/lSqM9my0HYwUFbX2W+nT9EKEIP
-d+jfAqTwZAhYC2CcNmswFw1H9GrvTI/9KZWKcZNUIqemc0A/FyEyONUM18/vjQ7D
+9UP1eyfRZR3E/+oticnm/cS20BGCbjoYrNgLthXKyaASuhGoElKs8EZ3h9MiI+u0
-lUwOcQsgAg44QTOUPgqXv3sJPQM5EnGuv3yYV9u6Y2i78M8CAwEAAaN7MHkwDgYD
+DpR0KpePhAkMLugBrgYWqkMwwD1684LfC4YVQrsLwzo5OW8CAwEAAaN7MHkwDgYD
-VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFPrDeNWgtEyZ
+VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFPbXs3g3lMjH
-VM0yeoRZdK2QGjyvMB8GA1UdIwQYMBaAFPrDeNWgtEyZVM0yeoRZdK2QGjyvMBYG
+1JMe0a5aVbN7lB92MB8GA1UdIwQYMBaAFPbXs3g3lMjH1JMe0a5aVbN7lB92MBYG
-A1UdEQQPMA2CC0Jyb2tlci1Sb290MA0GCSqGSIb3DQEBCwUAA4IBAQAD2S0kqL18
+A1UdEQQPMA2CC0Jyb2tlci1Sb290MA0GCSqGSIb3DQEBCwUAA4IBAQBM5RsXb2HN
-laewh+qnyZ0WMq12mLV/Rwll6ZuShCx2uAu3UZuIGWk3l7gG5zlws+i+zbaNcn4o
+FpC1mYfocXAn20Zu4d603qmc/IqkiOWbp36pWo+jk1AxejyRS9hEpQalgSnvcRPQ
-HsS3WG9kiNLOMKp8LXGkjErl6RaQr+kb8qgYFTPjOr6v0OdVn6ve9RDNYB5Hd+zE
+1hPEhGU+wvI0WWVi/01iNjVbXmJNPQEouXQWAT17dyp9vqQkPw8LNzpSV/qdPgbT
-9jAWmS8PfS2AldE4VAd0C4pWTAinhnKGrKdn1YAX5x+LMq1y0lc1Pd4CDgsjD6SS
+Z9o3sZrjUsSLsK7A7Q5ky4ePkiJBaMsHeAD+wqGwpiJ4D2Xhp8e1v36TWM0qt2EA
-3td7JtenXqCX0mN0XSeck7vvFGa6QpcQoVcN9tRENctHZTwyeGA21IkXylpFPUkE
+gySx9isx/jeGGPBmDqYB9BCal5lrihPN56jd+5pCkyXeZqKWiiXFJKXwcwxctYZc
-LT60k48fNC8TZkBlfvtVGRebpm5krXIKEaVy5LniEpSuOR4hTqsgoQDntBjW4zHA
+ADHIiTLLPXE8LHTUJAO51it1NAZ1S24aMzax4eWDXcWO7/ybbx5pkYkMd6EqlKHd
-GeWQ1wQNTEBX
+8riQJIhY4huX
------END CERTIFICATE-----
+-----END CERTIFICATE-----

bbmri/modules/exporter-compose.yml

@@ -23,18 +23,6 @@ services:
       - "traefik.http.routers.exporter_bbmri.tls=true"
       - "traefik.http.middlewares.exporter_bbmri_strip.stripprefix.prefixes=/bbmri-exporter"
       - "traefik.http.routers.exporter_bbmri.middlewares=exporter_bbmri_strip"
-      # Main router
-      - "traefik.http.routers.exporter_bbmri.priority=20"
-
-      # API router
-      - "traefik.http.routers.exporter_bbmri_api.middlewares=exporter_bbmri_strip,exporter_auth"
-      - "traefik.http.routers.exporter_bbmri_api.rule=PathRegexp(`/bbmri-exporter/.+`)"
-      - "traefik.http.routers.exporter_bbmri_api.tls=true"
-      - "traefik.http.routers.exporter_bbmri_api.priority=25"
-
-      # Shared middlewares
-      - "traefik.http.middlewares.exporter_auth.basicauth.users=${EXPORTER_USER}"
-
     volumes:
       - "/var/cache/bridgehead/bbmri/exporter-files:/app/exporter-files/output"
 
@@ -77,10 +65,3 @@ services:
       - "traefik.http.routers.reporter_bbmri.tls=true"
       - "traefik.http.middlewares.reporter_bbmri_strip.stripprefix.prefixes=/bbmri-reporter"
       - "traefik.http.routers.reporter_bbmri.middlewares=reporter_bbmri_strip"
-      - "traefik.http.routers.reporter_bbmri.priority=20"
-
-      - "traefik.http.routers.reporter_bbmri_api.middlewares=reporter_bbmri_strip,exporter_auth"
-      - "traefik.http.routers.reporter_bbmri_api.rule=PathRegexp(`/bbmri-reporter/.+`)"
-      - "traefik.http.routers.reporter_bbmri_api.tls=true"
-      - "traefik.http.routers.reporter_bbmri_api.priority=25"
-

bbmri/modules/teiler-compose.yml

@@ -68,3 +68,6 @@ services:
       TEILER_DASHBOARD_DE_URL: "https://${HOST}/bbmri-teiler-dashboard/de"
       TEILER_DASHBOARD_EN_URL: "https://${HOST}/bbmri-teiler-dashboard/en"
       HTTP_PROXY: "http://forward_proxy:3128"
+      ENABLE_MTBA: "${ENABLE_MTBA}"
+      ENABLE_DATASHIELD: "${ENABLE_DATASHIELD}"
+      IDMANAGER_UPLOAD_APIKEY: "${IDMANAGER_UPLOAD_APIKEY}" # Only used to check if the ID Manager is active

bbmri/modules/teiler-setup.sh

@@ -5,4 +5,5 @@ if [ "$ENABLE_TEILER" == true ];then
   OVERRIDE+=" -f ./$PROJECT/modules/teiler-compose.yml"
   TEILER_DEFAULT_LANGUAGE=EN
   TEILER_DEFAULT_LANGUAGE_LOWER_CASE=${TEILER_DEFAULT_LANGUAGE,,}
+  # add_public_oidc_redirect_url "/ccp-teiler/*"
 fi

bbmri/vars

@@ -3,6 +3,7 @@ BROKER_URL=https://${BROKER_ID}
 PROXY_ID=${SITE_ID}.${BROKER_ID}
 PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem
 BROKER_URL_FOR_PREREQ=$BROKER_URL
+echo $PRIVATEKEYFILENAME
 
 # Makes sense for all European Biobanks
 : ${ENABLE_ERIC:=true}
@@ -12,6 +13,8 @@ BROKER_URL_FOR_PREREQ=$BROKER_URL
 
 FOCUS_RETRY_COUNT=${FOCUS_RETRY_COUNT:-64}
 
+POSTGRES_TAG=15.6-alpine
+
 for module in $PROJECT/modules/*.sh
 do
     log DEBUG "sourcing $module"

ccp/modules/datashield-compose.yml

@@ -22,7 +22,7 @@ services:
 
   opal:
     container_name: bridgehead-opal
-    image: docker.verbis.dkfz.de/ccp/dktk-opal:latest
+    image: docker.verbis.dkfz.de/ccp/dktk-opal:test
     labels:
       - "traefik.enable=true"
       - "traefik.http.routers.opal_ccp.rule=PathPrefix(`/opal`)"

ccp/modules/teiler-compose.yml

@@ -19,7 +19,8 @@ services:
       HTTP_RELATIVE_PATH: "/ccp-teiler"
 
   teiler-dashboard:
-    image: docker.verbis.dkfz.de/cache/samply/teiler-dashboard:develop
+    #image: docker.verbis.dkfz.de/cache/samply/teiler-dashboard:develop
+    image: samply/teiler-dashboard:develop
     container_name: bridgehead-teiler-dashboard
     labels:
       - "traefik.enable=true"

ccp/vars

@@ -15,6 +15,8 @@ OIDC_PUBLIC_CLIENT_ID=${SITE_ID}-public
 OIDC_URL="https://login.verbis.dkfz.de/realms/test-realm-01"
 OIDC_GROUP_CLAIM="groups"
 
+POSTGRES_TAG=15.6-alpine
+
 for module in $PROJECT/modules/*.sh
 do
     log DEBUG "sourcing $module"

dhki/vars

@@ -8,6 +8,8 @@ PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem
 
 BROKER_URL_FOR_PREREQ=$BROKER_URL
 
+POSTGRES_TAG=15.6-alpine
+
 for module in ccp/modules/*.sh
 do
     log DEBUG "sourcing $module"

lib/functions.sh

@@ -301,19 +301,34 @@ function sync_secrets() {
     if [[ $secret_sync_args == "" ]]; then
         return
     fi
+
+    if [ "$PROJECT" == "bbmri" ]; then
+        # If the project is BBMRI, use the BBMRI-ERIC broker and not the GBN broker
+        proxy_id=$ERIC_PROXY_ID
+        broker_url=$ERIC_BROKER_URL
+        broker_id=$ERIC_BROKER_ID
+        root_crt_file="/srv/docker/bridgehead/bbmri/modules/${ERIC_ROOT_CERT}.root.crt.pem"
+    else
+        proxy_id=$PROXY_ID
+        broker_url=$BROKER_URL
+        broker_id=$BROKER_ID
+        root_crt_file="/srv/docker/bridgehead/$PROJECT/root.crt.pem"
+    fi
+
     mkdir -p /var/cache/bridgehead/secrets/ || fail_and_report 1 "Failed to create '/var/cache/bridgehead/secrets/'. Please run sudo './bridgehead install $PROJECT' again."
     touch /var/cache/bridgehead/secrets/oidc
+    echo $PRIVATEKEYFILENAME
     docker run --rm \
         -v /var/cache/bridgehead/secrets/oidc:/usr/local/cache \
         -v $PRIVATEKEYFILENAME:/run/secrets/privkey.pem:ro \
-        -v /srv/docker/bridgehead/$PROJECT/root.crt.pem:/run/secrets/root.crt.pem:ro \
+        -v $root_crt_file:/run/secrets/root.crt.pem:ro \
         -v /etc/bridgehead/trusted-ca-certs:/conf/trusted-ca-certs:ro \
         -e TLS_CA_CERTIFICATES_DIR=/conf/trusted-ca-certs \
         -e NO_PROXY=localhost,127.0.0.1 \
         -e ALL_PROXY=$HTTPS_PROXY_FULL_URL \
-        -e PROXY_ID=$PROXY_ID \
+        -e PROXY_ID=$proxy_id \
-        -e BROKER_URL=$BROKER_URL \
+        -e BROKER_URL=$broker_url \
-        -e OIDC_PROVIDER=secret-sync-central.central-secret-sync.$BROKER_ID \
+        -e OIDC_PROVIDER=secret-sync-central.central-secret-sync.$broker_id \
         -e SECRET_DEFINITIONS=$secret_sync_args \
         docker.verbis.dkfz.de/cache/samply/secret-sync-local:latest
 

lib/install-bridgehead.sh

@@ -49,12 +49,6 @@ if [ -z "$TRANSFAIR_AUTH" ]; then
   fi
 fi
 
-if [ "$ENABLE_EXPORTER" == "true" ] && [ -z "$EXPORTER_USER" ]; then
-  log "INFO" "Now generating basic auth for the exporter and reporter (see adduser in bridgehead for more information)."
-  generated_passwd="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 32)"
-  add_basic_auth_user $PROJECT $generated_passwd "EXPORTER_USER" $PROJECT
-fi
-
 log "INFO" "Registering system units for bridgehead and bridgehead-update"
 cp -v \
     lib/systemd/bridgehead\@.service \

versions/acceptance

@@ -1,4 +1,3 @@
 FOCUS_TAG=develop
 BEAM_TAG=develop
-BLAZE_TAG=main
+BLAZE_TAG=main
-POSTGRES_TAG=15.13-alpine

versions/prod

@@ -1,4 +1,3 @@
 FOCUS_TAG=main
 BEAM_TAG=main
-BLAZE_TAG=0.32
+BLAZE_TAG=0.32
-POSTGRES_TAG=15.13-alpine

versions/test

@@ -1,4 +1,3 @@
 FOCUS_TAG=develop
 BEAM_TAG=develop
-BLAZE_TAG=main
+BLAZE_TAG=main
-POSTGRES_TAG=15.13-alpine