Compare commits

...

20 Commits

Author SHA1 Message Date
2c79799815 Pin the landing page to a version known to work 2024-08-29 15:05:39 +02:00
ca93451357 Added ECDC Beam Broker site to list of Firewall exceptions 2024-07-26 09:49:13 +02:00
892e2c2cf1 Removed misleading text from stop.sh
This command no longer disables the Bridgehead in systemctl, it
only stops it.
2024-07-18 12:56:42 +02:00
520c560be0 Got systemctl restart to work locally
There were problems on my Ubuntu VM with the target run level that I was
using, so I went for the default. Now, if I restart my VM, the Bridgehead
is successfully started automatically.

Also added some status-generating commands to the run.sh and stop.sh
scripts, so that it looks like they are doing something.
2024-07-18 12:31:42 +02:00
bb81617873 Replaced tabs with spaces 2024-07-17 09:34:46 +02:00
c27536b566 Cherry picked from ehds2 and updated data upload for new scripts 2024-07-16 16:47:57 +02:00
f213e6909a Started adding the new scripts to the README 2024-07-16 16:17:30 +02:00
6887264a5b Added infrastructure for running ECDC-Bridgehead under systemctl
This comprises of the service file to be copied into the systemctl directory,
plus Bridgehead start and stop scripts for both systemctl and for the
administrator on the command line.
2024-07-16 16:08:08 +02:00
5b2c3d7725 Using bridgehead user for performing git pull
Otherwise problems can arise with authentication.
2024-07-16 16:06:10 +02:00
a8a15aaad8 Update repos on start
When Bridgehead is started, make sure that all repositories are up to
date first.
2024-07-12 09:04:08 +02:00
2e8b1dc96c Deal with multiple data files
The original TransFAIR logic was only able to handle a single CSV file
and broke if there were more. This commit fixes that.
2024-07-12 08:45:17 +02:00
105495d6cd Made compatible with ehds2 branch 2024-07-10 14:51:16 +02:00
93e73838c6 Made comments compatible with ehds2 branch 2024-07-10 14:39:46 +02:00
9e72b04824 Added instructions to stop and start Bridgehead 2024-07-10 14:25:21 +02:00
fbe68bc778 Adding more detail to EHDS2-specific installation tasks 2024-07-10 14:18:03 +02:00
17f372b06c Implement a gentler way of handling clock skew 2024-07-10 14:02:04 +02:00
a250d52998 Pinned Blaze version, increased Focus retry count 2024-07-02 14:05:38 +02:00
e309dc495a Improved docker-compose for transFAIR
In particular, corrected the code for detecting CSV files in the data directory.
The old code could not handle more than one CSV file.
2024-06-18 15:28:37 +02:00
53cdd49fd7 Improved docs, turned off obfuscation
ECDC does not want obfuscated results.
2024-03-28 11:04:08 +01:00
36a97ecc20 Removed ECDC URLs and certificates from dev branch 2024-03-27 16:04:25 +01:00
17 changed files with 266 additions and 87 deletions

View File

@ -8,8 +8,8 @@ This repository is the starting point for any information and tools you will nee
- [Hardware](#hardware) - [Hardware](#hardware)
- [Software](#software) - [Software](#software)
- [Network](#network) - [Network](#network)
- [EHDS2/ECDC](#ehds2-ecdc)
2. [Deployment](#deployment) 2. [Deployment](#deployment)
- [EHDS2/ECDC](#ehds2-ecdc)
- [Site name](#site-name) - [Site name](#site-name)
- [Projects](#projects) - [Projects](#projects)
- [GitLab repository](#gitlab-repository) - [GitLab repository](#gitlab-repository)
@ -88,37 +88,43 @@ The following URLs need to be accessible (prefix with `https://`):
* gitlab.bbmri-eric.eu * gitlab.bbmri-eric.eu
* only for German Biobank Node * only for German Biobank Node
* broker.bbmri.de * broker.bbmri.de
* only for EHDS2/ECDC
* ecdc-vm-ehds-test1.swedencentral.cloudapp.azure.com
> 📝 This URL list is subject to change. Instead of the individual names, we highly recommend whitelisting wildcard domains: *.dkfz.de, github.com, *.docker.com, *.docker.io, *.samply.de, *.bbmri.de. > 📝 This URL list is subject to change. Instead of the individual names, we highly recommend whitelisting wildcard domains: *.dkfz.de, github.com, *.docker.com, *.docker.io, *.samply.de, *.bbmri.de.
> 📝 Ubuntu's pre-installed uncomplicated firewall (ufw) is known to conflict with Docker, more info [here](https://github.com/chaifeng/ufw-docker). > 📝 Ubuntu's pre-installed uncomplicated firewall (ufw) is known to conflict with Docker, more info [here](https://github.com/chaifeng/ufw-docker).
## Deployment
### EHDS2/ECDC ### EHDS2/ECDC
ECDC data should be provided as a CSV file and placed in the directory /srv/docker/ecdc/data. The Bridgehead can be started without data, but obviously, any searches run from a Locator will return zero results for this site if you do that. Note that an empty data directory will automatically be inserted on the first start of the Bridgehead if you don't set one up yourself. The ECDC Bridgehead allows you to connect your site/node to the [AMR Explorer](http://ehds2-lens.swedencentral.cloudapp.azure.com/), a non-public central web site that allow certified researchers to search for information relating to antiobiotic resistance, Europe-wide. You can supply the Bridgehead with data from your site in the form of CSV files, which will then be made available to the Explorer for searching purposes.
To get the right Beam certificate for your setup, you will need to edit the following files and comment/uncomment as appropriate: You will need to set up some configuration before you can start a Bridgehead. This can be done as follows:
- bbmri/modules/ehds2.root.crt.pem ```shell
- bbmri/modules/ehds2.test.root.crt.pem sudo mkdir -p /etc/bridgehead
sudo cp /srv/docker/bridgehead/bbmri/modules/bbmri.conf /etc/bridgehead
```
You will also need to edit: Now edit ```/etc/bridgehead/bbmri.conf``` and customize the following variables for your site:
./bbmri/modules/ehds2-setup.sh - SITE_NAME
- SITE_ID
- OPERATOR_FIRST_NAME
- OPERATOR_LAST_NAME
- OPERATOR_EMAIL
and set the broker address correctly for your site. If you run a proxy at your site, you will also need to give values to the ```HTTP*_PROXY*``` variables.
When you first start the Bridgehead, it will clone two extra repositories into /srv/docker, namely, focus and transfair. It will automatically build local images of these repositories for you. These components have the following functionality that has been customized for ECDC: When you first start the Bridgehead, it will clone two extra repositories into /srv/docker, namely, ```focus``` and ```transfair```. It will automatically build local images of these repositories for you. These components have the following functionality that has been customized for ECDC:
- *focus.* This component is responsible for completing the CQL that is used for running queries against the Blaze FHIR store. It uses a set of templates for doing this. Extra templates have been written for the ECDC use case. - *focus.* This component is responsible for completing the CQL that is used for running queries against the Blaze FHIR store. It uses a set of templates for doing this. Extra templates have been written for the ECDC use case. They can be found in /srv/docker/focus/resources/cql/EHDS2*.
- *transfair.* This is an ETL component. It takes the CSV data that you provide, converts it to FHIR, and loads it to Blaze. - *transfair.* This is an ETL component. It takes the CSV data that you provide, converts it to FHIR, and loads it to Blaze. This will be run once, if there is data in /srv/docker/ecdc/data. A lock file in the data directory ensures that it does not get run again. Remove this lock file and restart the Bridgehead if you want to load new data.
These images will normally be rebuilt every time you restart the Bridgehead. This is a workaround to fix a bug: if you don't rebuild these images for every start, then legacy versions will be used and you will lose the new ECDC functionality. The reason for this is still under investigation. These images will normally be rebuilt every time you restart the Bridgehead. This is a workaround to fix a bug: if you don't rebuild these images for every start, then legacy versions will be used and you will lose the new ECDC functionality. The reason for this is still under investigation.
Note that the /srv/docker/ecdc/data directory is also used as the home for a "lock" file, which will be created if your data has been successfully converted to FHIR by transfair. As long as this file exists, transfair will not be run again when you restart the Bridgehead. Hence, if you add or modify data, you will need to delete this file by hand before restarting the Bridgehead.
## Deployment
### Site name ### Site name
You will need to choose a short name for your site. This is not a URL, just a simple identifying string. For the examples below, we will use "your-site-name", but you should obviously choose something that is meaningful to you and which is unique. You will need to choose a short name for your site. This is not a URL, just a simple identifying string. For the examples below, we will use "your-site-name", but you should obviously choose something that is meaningful to you and which is unique.
@ -165,19 +171,14 @@ sudo git clone https://github.com/samply/bridgehead.git /srv/docker/bridgehead
If this is an ECDC/EHDS2 installation, switch to the ```ehds2``` branch and copy the configuration file to the required location: If this is an ECDC/EHDS2 installation, switch to the ```ehds2``` branch and copy the configuration file to the required location:
```shell ```shell
cd
git clone https://github.com/samply/transFAIR.git
cd transFAIR
docker build -t samply/transfair --no-cache .
cd /srv/docker/bridgehead cd /srv/docker/bridgehead
sudo git checkout ehds2 sudo git checkout ehds2
sudo mkdir test data
sudo mkdir -p /etc/bridgehead/ sudo mkdir -p /etc/bridgehead/
sudo cp bbmri/modules/bbmri.conf /etc/bridgehead/ sudo cp bbmri/modules/bbmri.conf /etc/bridgehead/
sudo vi /etc/bridgehead/bbmri.conf # Modify to include national node name and admin contact details sudo vi /etc/bridgehead/bbmri.conf # Modify to include national node name and admin contact details
``` ```
For an ECDC/EHDS2 installation, you will also need to copy your data in a comma-separated value (CSV) formatted file to ```/srv/docker/bridgehead/data```. Make sure it is readable by all. Only files with the ending ```.csv``` will be read in, all other files will be ignored. For an ECDC/EHDS2 installation, you will also need to copy your data in a comma-separated value (CSV) formatted file to ```/srv/docker/ecdc/data```. Make sure it is readable by all. Only files with the ending ```.csv``` will be read in, all other files will be ignored.
If this is not an ECDC/EHDS2 installation, then download your site specific configuration repository: If this is not an ECDC/EHDS2 installation, then download your site specific configuration repository:
```shell ```shell
@ -232,6 +233,22 @@ pki-scripts/managepki list
### Starting and stopping your Bridgehead ### Starting and stopping your Bridgehead
For an ECDC/EHDS2 installation, this is done with the help of specialized scripts:
To start:
```shell
sudo /srv/docker/bridgehead/run.sh
```
To stop (you generally won't need to do this):
```shell
sudo /srv/docker/bridgehead/stop.sh
```
For regular installations, read on.
If you followed the above steps, your Bridgehead should already be configured to autostart (via systemd). If you would like to start/stop manually: If you followed the above steps, your Bridgehead should already be configured to autostart (via systemd). If you would like to start/stop manually:
To start, run To start, run
@ -377,6 +394,19 @@ There will be a delay before the effects of Directory sync become visible. First
The data accessed by the federated search is held in the Bridgehead in a FHIR store (we use Blaze). The data accessed by the federated search is held in the Bridgehead in a FHIR store (we use Blaze).
For an ECDC/EHDS2 installation, you need to provide your data as a table in a CSV (comma-separated value) files and place it in the directory /srv/docker/ecdc/data. You can provide as many data files as you like, and you can add new files incrementally over time.
In order for this new data to be loaded, you will need to execute the ```run.sh``` script with the appropriate arguments:
- To read just the most recently added data files: ```/srv/docker/bridgehead run.sh --upload```.
- To read in all data from scratch: ```/srv/docker/bridgehead run.sh --upload-all```.
These two variants give you the choice between uploading data in an incremental way that preserves the date used for statistics or as a single upload that date stamps everything with the current date.
The Bridgehead can be started without data, but obviously, any searches run from the Explorer will return zero results for your site if you do that. Note that an empty data directory will automatically be inserted on the first start of the Bridgehead if you don't set one up yourself.
For non-ECDC setups, read on.
You can load data into this store by using its FHIR API: You can load data into this store by using its FHIR API:
``` ```

View File

@ -4,7 +4,10 @@ version: "3.7"
services: services:
blaze: blaze:
image: docker.verbis.dkfz.de/cache/samply/blaze:latest #image: docker.verbis.dkfz.de/cache/samply/blaze:latest
# Blaze versions 0.26 and 0.27 do not return anything when you run a
# CQL query, so I am pinning the version at 0.25.
image: samply/blaze:0.25
container_name: bridgehead-bbmri-blaze container_name: bridgehead-bbmri-blaze
environment: environment:
BASE_URL: "http://bridgehead-bbmri-blaze:8080" BASE_URL: "http://bridgehead-bbmri-blaze:8080"

View File

@ -19,18 +19,19 @@
### Common Configuration of all Components ### Common Configuration of all Components
## This is the descriptive humanreadable name of your site (e.g. München LMU) ## This is a descriptive human readable name of your site (e.g. Belgium)
SITE_NAME=ECDC-Bridgehead-<National node> SITE_NAME=<National node>
## This is the id for your site used in machine to machine communication (e.g. muenchen-lmu) ## This is the id for your site used in machine to machine communication (should be
SITE_ID=ecdc-bridgehead-<National node> ## lower-case, e.g. belgium)
SITE_ID=<National node>
## This server's hostname, for access from other computers within your institution ## This server's hostname, for access from other computers within your institution
## (e.g. mybridgehead.intern.myinstitution.org) ## (e.g. mybridgehead.intern.myinstitution.org)
## Optional. If left empty, this is auto-generated via the `hostname` command. ## Optional. If left empty, this is auto-generated via the `hostname` command.
HOST= HOST=
## Proxy Configuration ## Proxy Configuration
# leave empty if not applyable # leave empty if not applicable
# ex.: http://my-proxy-host:my-proxy-port # eg.: http://my-proxy-host:my-proxy-port
HTTP_PROXY_URL= HTTP_PROXY_URL=
HTTP_PROXY_USERNAME= HTTP_PROXY_USERNAME=
HTTP_PROXY_PASSWORD= HTTP_PROXY_PASSWORD=

View File

@ -12,6 +12,7 @@ services:
BLAZE_URL: "http://blaze:8080/fhir/" BLAZE_URL: "http://blaze:8080/fhir/"
BEAM_PROXY_URL: http://beam-proxy-ehds2:8081 BEAM_PROXY_URL: http://beam-proxy-ehds2:8081
RETRY_COUNT: ${FOCUS_RETRY_COUNT} RETRY_COUNT: ${FOCUS_RETRY_COUNT}
OBFUSCATE: "no"
depends_on: depends_on:
- "beam-proxy-ehds2" - "beam-proxy-ehds2"
- "blaze" - "blaze"
@ -55,11 +56,11 @@ services:
command: bash -c " \ command: bash -c " \
echo listing /app/data && \ echo listing /app/data && \
ls -la /app/data && \ ls -la /app/data && \
[ -f /app/data/*.[cC][sS][vV] ] && \ ls /app/data/*.[cC][sS][vV] 1> /dev/null 2>&1 && \
[ ! -f /app/data/lock ] && \ [ ! -f /app/data/lock ] && \
( \ ( \
echo 'Wait for Blaze to finish initializing' ; \ echo 'Wait for Blaze to finish initializing' ; \
sleep 60 ; \ sleep 360 ; \
echo 'Remove old output files' ; \ echo 'Remove old output files' ; \
rm -rf /app/test/* ; \ rm -rf /app/test/* ; \
cd /app ; \ cd /app ; \
@ -77,5 +78,5 @@ services:
test-data-loader: test-data-loader:
container_name: test-data-loader container_name: test-data-loader
image: samply/test-data-loader image: samply/test-data-loader
command: sh -c "sleep 300 && echo Listing all resources in FHIR store && blazectl --server http://bridgehead-bbmri-blaze:8080/fhir count-resources && tail -f /dev/null" command: sh -c "sleep 420 && echo Listing all resources in FHIR store && blazectl --server http://bridgehead-bbmri-blaze:8080/fhir count-resources && tail -f /dev/null"

View File

@ -8,18 +8,15 @@ if [ "${ENABLE_EHDS2}" == "true" ]; then
case "$ENVIRONMENT" in case "$ENVIRONMENT" in
"production") "production")
export EHDS2_BROKER_ID=broker.bbmri.samply.de export EHDS2_BROKER_ID=broker.bbmri.samply.de
#export EHDS2_BROKER_ID=ecdc-vm-ehds-test1.swedencentral.cloudapp.azure.com
export EHDS2_ROOT_CERT=ehds2 export EHDS2_ROOT_CERT=ehds2
;; ;;
"test") "test")
export EHDS2_BROKER_ID=broker.test.bbmri.samply.de export EHDS2_BROKER_ID=broker.test.bbmri.samply.de
#export EHDS2_BROKER_ID=ecdc-vm-ehds-test1.swedencentral.cloudapp.azure.com
export EHDS2_ROOT_CERT=ehds2.test export EHDS2_ROOT_CERT=ehds2.test
;; ;;
*) *)
report_error 6 "Environment \"$ENVIRONMENT\" is unknown. Assuming production. FIX THIS!" report_error 6 "Environment \"$ENVIRONMENT\" is unknown. Assuming production. FIX THIS!"
export EHDS2_BROKER_ID=broker.bbmri.samply.de export EHDS2_BROKER_ID=broker.bbmri.samply.de
#export EHDS2_BROKER_ID=ecdc-vm-ehds-test1.swedencentral.cloudapp.azure.com
export EHDS2_ROOT_CERT=ehds2 export EHDS2_ROOT_CERT=ehds2
;; ;;
esac esac

View File

@ -20,25 +20,3 @@ cBYgkKJTjwdRhc+ehI3kFo7b/a/Z/jl9szKsAPHozMixSi8lGnsYwN80oqeRvT7h
wcMUK+igv3/K wcMUK+igv3/K
-----END CERTIFICATE----- -----END CERTIFICATE-----
# EHDS2/ECDC certificate
#-----BEGIN CERTIFICATE-----
#MIIDNTCCAh2gAwIBAgIUamQJB/Uevzf3+qrDQOqA8mwkwX8wDQYJKoZIhvcNAQEL
#BQAwFjEUMBIGA1UEAxMLQnJva2VyLVJvb3QwHhcNMjQwMjA3MTQ1MTQxWhcNMzQw
#MjA0MTQ1MjExWjAWMRQwEgYDVQQDEwtCcm9rZXItUm9vdDCCASIwDQYJKoZIhvcN
#AQEBBQADggEPADCCAQoCggEBAKvI2XT0os20vG7d8FjRuMq9mALcQgLZvjAbQZbm
#JyeVxRQP5HsSrBlBwaZWgmXWFfIPlwsEsTAUsCneAl4ZL3gQ6831nbmmpgThOEVy
#rGBsbL6qlEWWplD/AGpcdEbAuDmuMSO9tKR6iRm+acTHJcXqsDm2HUUS7U+ManWY
#S0dG10beJE2eVtEv5/fnFlVSn72VgAVgXVl7UA5E77zXK9GUAyQ6Oo7MET9zifZ/
#EifeSoEvLk+U7HRLgQbKz2ti6ABh0wnK7Ec2DjekMBXyeQz0tR+8dS53s5pqti7c
#SUQMHmL4jKxvfQr7fr5wbJpFDu37Vif9GdND9WSDRcufuM8CAwEAAaN7MHkwDgYD
#VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFKMF6QgDV4cV
#WyQZn3EHKLUumXzuMB8GA1UdIwQYMBaAFKMF6QgDV4cVWyQZn3EHKLUumXzuMBYG
#A1UdEQQPMA2CC0Jyb2tlci1Sb290MA0GCSqGSIb3DQEBCwUAA4IBAQAhhE0ht7Sp
#inKLJSVZrdFHbAbRXnYcUz/3oyy/uw0YvF0dJAHz2sgbtCLMHzKU7NwotaEqZedA
#I7688nI42aHggIkwa1CzmFo9ybQ2afgrYNeF+TcTQI6G+fFpeRT5I+FvA9hUI3r4
#Kp63Si7cpQS5uiHs90sv2SNMedo2Gf9VyJKB7NOReQJWsrw32ITi4QLNKHcHciKm
#cpi/OqbR1YTJRoUrcC655G7bSKENzccdTWo8kBJgYLVbe+FohqgheqPqWpdH2zoR
#OxOwviutfJTNvDyhXu8FrDc5tr9f5VjXv1hK2szYuvggwbybUBB9bLsx+fV79oZe
#3zR0j5YwB28G
#-----END CERTIFICATE-----

View File

@ -20,25 +20,3 @@ UNe/254bc0vqHEPT6VI/86c7qAmk1xR0RUfrnKAEqZtUeuoj2fe1L/6yOB16fxt5
vZdXnV1qjlM2 vZdXnV1qjlM2
-----END CERTIFICATE----- -----END CERTIFICATE-----
# EHDS2/ECDC certificate
#-----BEGIN CERTIFICATE-----
#MIIDNTCCAh2gAwIBAgIUamQJB/Uevzf3+qrDQOqA8mwkwX8wDQYJKoZIhvcNAQEL
#BQAwFjEUMBIGA1UEAxMLQnJva2VyLVJvb3QwHhcNMjQwMjA3MTQ1MTQxWhcNMzQw
#MjA0MTQ1MjExWjAWMRQwEgYDVQQDEwtCcm9rZXItUm9vdDCCASIwDQYJKoZIhvcN
#AQEBBQADggEPADCCAQoCggEBAKvI2XT0os20vG7d8FjRuMq9mALcQgLZvjAbQZbm
#JyeVxRQP5HsSrBlBwaZWgmXWFfIPlwsEsTAUsCneAl4ZL3gQ6831nbmmpgThOEVy
#rGBsbL6qlEWWplD/AGpcdEbAuDmuMSO9tKR6iRm+acTHJcXqsDm2HUUS7U+ManWY
#S0dG10beJE2eVtEv5/fnFlVSn72VgAVgXVl7UA5E77zXK9GUAyQ6Oo7MET9zifZ/
#EifeSoEvLk+U7HRLgQbKz2ti6ABh0wnK7Ec2DjekMBXyeQz0tR+8dS53s5pqti7c
#SUQMHmL4jKxvfQr7fr5wbJpFDu37Vif9GdND9WSDRcufuM8CAwEAAaN7MHkwDgYD
#VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFKMF6QgDV4cV
#WyQZn3EHKLUumXzuMB8GA1UdIwQYMBaAFKMF6QgDV4cVWyQZn3EHKLUumXzuMBYG
#A1UdEQQPMA2CC0Jyb2tlci1Sb290MA0GCSqGSIb3DQEBCwUAA4IBAQAhhE0ht7Sp
#inKLJSVZrdFHbAbRXnYcUz/3oyy/uw0YvF0dJAHz2sgbtCLMHzKU7NwotaEqZedA
#I7688nI42aHggIkwa1CzmFo9ybQ2afgrYNeF+TcTQI6G+fFpeRT5I+FvA9hUI3r4
#Kp63Si7cpQS5uiHs90sv2SNMedo2Gf9VyJKB7NOReQJWsrw32ITi4QLNKHcHciKm
#cpi/OqbR1YTJRoUrcC655G7bSKENzccdTWo8kBJgYLVbe+FohqgheqPqWpdH2zoR
#OxOwviutfJTNvDyhXu8FrDc5tr9f5VjXv1hK2szYuvggwbybUBB9bLsx+fV79oZe
#3zR0j5YwB28G
#-----END CERTIFICATE-----

View File

@ -7,7 +7,7 @@
# Makes only sense for EHDS2 project # Makes only sense for EHDS2 project
: ${ENABLE_EHDS2:=false} : ${ENABLE_EHDS2:=false}
FOCUS_RETRY_COUNT=32 FOCUS_RETRY_COUNT=128
PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem PRIVATEKEYFILENAME=/etc/bridgehead/pki/${SITE_ID}.priv.pem
for module in $PROJECT/modules/*.sh for module in $PROJECT/modules/*.sh

View File

@ -88,7 +88,11 @@ case "$ACTION" in
start) start)
loadVars loadVars
hc_send log "Bridgehead $PROJECT startup: Checking requirements ..." hc_send log "Bridgehead $PROJECT startup: Checking requirements ..."
chown -R bridgehead ${BASE}
checkRequirements checkRequirements
# Note: changes to "bridgehead" script will only take effect after next start.
su bridgehead -c "git pull"
chown -R bridgehead ${BASE}
# Local versions of focus and transfair are needed by EHDS2 # Local versions of focus and transfair are needed by EHDS2
clone_focus_if_nonexistent ${BASE}/.. clone_focus_if_nonexistent ${BASE}/..
build_focus ${BASE}/.. build_focus ${BASE}/..

14
ecdc.service Normal file
View File

@ -0,0 +1,14 @@
[Unit]
Description=Start ECDC Bridgehead
[Service]
Type=simple
ExecStart=/srv/docker/bridgehead/restart_service.sh
ExecStop=/srv/docker/bridgehead/shutdown_service.sh
Restart=always
RestartSec=36000
KillMode=mixed
[Install]
WantedBy=default.target

View File

@ -258,8 +258,8 @@ function clone_repo_if_nonexistent() {
cd "$target_dir" cd "$target_dir"
# Checkout the specified branch # Checkout the specified branch
git checkout "$branch_name" chown -R bridgehead .
echo "Checked out branch '$branch_name'." su bridgehead -c "git checkout $branch_name"
cd - cd -
} }
@ -267,7 +267,7 @@ function clone_repo_if_nonexistent() {
function clone_transfair_if_nonexistent() { function clone_transfair_if_nonexistent() {
local base_dir="$1" local base_dir="$1"
clone_repo_if_nonexistent https://github.com/samply/transFAIR.git $base_dir/transfair ehds2 clone_repo_if_nonexistent https://github.com/samply/transFAIR.git $base_dir/transfair ehds2_develop
} }
function clone_focus_if_nonexistent() { function clone_focus_if_nonexistent() {
@ -282,11 +282,13 @@ function build_transfair() {
# We only take the touble to build transfair if: # We only take the touble to build transfair if:
# #
# 1. There is no data lock file (which means that no ETL has yet been run) and # 1. There is data available (any CSV files) and
# 2. There is data available. # 2. There is no data lock file (which means that no ETL has yet been run).
if [ -f ../ecdc/data/*.[cC][sS][vV] ] && [ ! -f ../ecdc/data/lock ]; then if ls ../ecdc/data/*.[cC][sS][vV] 1> /dev/null 2>&1 && [ ! -f ../ecdc/data/lock ]; then
cd $base_dir/transfair cd $base_dir/transfair
su bridgehead -c "git pull"
docker build --progress=plain -t samply/transfair --no-cache . docker build --progress=plain -t samply/transfair --no-cache .
chown -R bridgehead .
cd - cd -
fi fi
} }
@ -295,7 +297,9 @@ function build_focus() {
local base_dir="$1" local base_dir="$1"
cd $base_dir/focus cd $base_dir/focus
su bridgehead -c "git pull"
docker build --progress=plain -f DockerfileWithBuild -t samply/focus --no-cache . docker build --progress=plain -f DockerfileWithBuild -t samply/focus --no-cache .
chown -R bridgehead .
cd - cd -
} }

View File

@ -84,7 +84,7 @@ else
SYNCTEXT="For example, consider entering a correct NTP server (e.g. your institution's Active Directory Domain Controller in /etc/systemd/timesyncd.conf (option NTP=) and restart systemd-timesyncd." SYNCTEXT="For example, consider entering a correct NTP server (e.g. your institution's Active Directory Domain Controller in /etc/systemd/timesyncd.conf (option NTP=) and restart systemd-timesyncd."
if [ $SKEW -ge 300 ]; then if [ $SKEW -ge 300 ]; then
report_error 5 "Your clock is not synchronized (${SKEW}s off). This will cause Samply.Beam's certificate will fail. Please setup time synchronization. $SYNCTEXT" report_error 5 "Your clock is not synchronized (${SKEW}s off). This will cause Samply.Beam's certificate will fail. Please setup time synchronization. $SYNCTEXT"
exit 1 log WARN "Server Time Error"
elif [ $SKEW -ge 60 ]; then elif [ $SKEW -ge 60 ]; then
log WARN "Your clock is more than a minute off (${SKEW}s). Consider syncing to a time server. $SYNCTEXT" log WARN "Your clock is more than a minute off (${SKEW}s). Consider syncing to a time server. $SYNCTEXT"
fi fi

View File

@ -45,7 +45,7 @@ services:
landing: landing:
container_name: bridgehead-landingpage container_name: bridgehead-landingpage
image: docker.verbis.dkfz.de/cache/samply/bridgehead-landingpage:main image: docker.verbis.dkfz.de/cache/samply/bridgehead-landingpage:master
labels: labels:
- "traefik.enable=true" - "traefik.enable=true"
- "traefik.http.routers.landing.rule=PathPrefix(`/`)" - "traefik.http.routers.landing.rule=PathPrefix(`/`)"

30
restart_service.sh Executable file
View File

@ -0,0 +1,30 @@
#!/bin/bash
# Start a running Bridgehead. If there is already a Bridgehead running,
# stop it first.
# This is intended to be used by systemctl.
cd /srv/docker/bridgehead
echo "git status before stop"
git status
echo "Stopping running Bridgehead, if present"
./bridgehead stop bbmri
# If "flush_blaze" is present, delete the Blaze volume before starting
# the Bridgehead again. This allows a user to upload all data, if
# requested.
if [ -f "/srv/docker/ecdc/data/flush_blaze" ]; then
docker volume rm bbmri_blaze-data
rm -f /srv/docker/ecdc/data/flush_blaze
fi
echo "git status before start"
git status | systemd-cat -p info
echo "Start the Bridgehead anew"
./bridgehead start bbmri
echo "Bridgehead has unexpectedly terminated"

83
run.sh Executable file
View File

@ -0,0 +1,83 @@
#!/usr/bin/env bash
# Start a Bridgehead from the command line. Upload data if requested.
# Behind the scenes we use systemctl to do the work.
# Function to print usage
print_usage() {
echo "Start a Bridghead, optionally upload data"
echo "Usage: $0 [--upload | --upload-all | --help | -h]"
echo "Options:"
echo " --upload Run Bridgehead and upload just the new CSV data files."
echo " --upload-all Run Bridgehead and upload all CSV data files."
echo " --help, -h Display this help message."
echo " No options Run Bridgehead only."
}
# Initialize variables
UPLOAD=false
UPLOAD_ALL=false
# Parse arguments
while [[ "$#" -gt 0 ]]; do
case $1 in
--upload)
UPLOAD=true
;;
--upload-all)
UPLOAD_ALL=true
;;
--help|-h)
print_usage
exit 0
;;
*)
echo "Error: Unknown argument '$1'"
print_usage
exit 1
;;
esac
shift
done
# Check for conflicting options
if [ "$UPLOAD" = true ] && [ "$UPLOAD_ALL" = true ]; then
echo "Error: you must specify either --upload or --upload-all, specifying both is not permitted."
print_usage
exit 1
fi
# Disable/stop standard Bridgehead systemctl services, if present
sudo systemctl disable bridgehead@bbmri.service >& /dev/null
sudo systemctl disable system-bridgehead.slice >& /dev/null
sudo systemctl disable bridgehead-update@bbmri.timer >& /dev/null
sudo systemctl stop bridgehead@bbmri.service >& /dev/null
sudo systemctl stop system-bridgehead.slice >& /dev/null
sudo systemctl stop bridgehead-update@bbmri.timer >& /dev/null
# Set up systemctl for EHDS2/ECDC if necessary
cp /srv/docker/bridgehead/ecdc.service /etc/systemd/system
systemctl daemon-reload
systemctl enable ecdc.service
# Use systemctl to stop the Bridgehead if it is running
sudo systemctl stop ecdc.service
# Use files to tell the Bridgehead what to do with any data present
if [ "$UPLOAD" = true ] || [ "$UPLOAD_ALL" = true ]; then
if [ -f /srv/docker/ecdc/data/lock ]; then
rm /srv/docker/ecdc/data/lock
fi
fi
if [ "$UPLOAD_ALL" = true ]; then
echo "All CSV files in /srv/docker/ecdc/data will be uploaded"
touch /srv/docker/ecdc/data/flush_blaze
fi
# Start up the Bridgehead
sudo systemctl start ecdc.service
# Show status of Bridgehead service
sleep 10
systemctl status ecdc.service

13
shutdown_service.sh Executable file
View File

@ -0,0 +1,13 @@
#!/bin/bash
# Shut down a running Bridgehead.
# This is intended to be used by systemctl.
cd /srv/docker/bridgehead
echo "git status before stop"
git status
echo "Stopping running Bridgehead, if present"
./bridgehead stop bbmri

43
stop.sh Executable file
View File

@ -0,0 +1,43 @@
#!/usr/bin/env bash
# Shut down a running Bridgehead.
# Behind the scenes we use systemctl to do the work.
# Function to print usage
print_usage() {
echo "Stop the running Bridgehead"
echo "Usage: $0 [--help | -h]"
echo "Options:"
echo " --help, -h Display this help message."
echo " No options Stop Bridgehead only."
}
# Parse arguments
while [[ "$#" -gt 0 ]]; do
case $1 in
--help|-h)
print_usage
exit 0
;;
*)
echo "Error: Unknown argument '$1'"
print_usage
exit 1
;;
esac
shift
done
# Set up systemctl for EHDS2/ECDC if necessary
cp /srv/docker/bridgehead/ecdc.service /etc/systemd/system
systemctl daemon-reload
systemctl enable ecdc.service
# Use systemctl to stop the Bridgehead if it is running
sudo systemctl stop ecdc.service
# Show status of Bridgehead service
sleep 20
systemctl status ecdc.service
docker ps