verbis-admin
/
bridgehead
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/dnpmconnector' into feature/reportHubAndDNPM

This commit is contained in:
Tobias Kussel 2022-11-09 13:23:55 +00:00
parent fc3198d22c 4a53bb3fb2
commit fd8651bb2c
19 changed files with 367 additions and 192 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

158
README.md
View File

@ -20,6 +20,7 @@ TOC
- [docker](#dockerhttpsdocsdockercomget-docker)
- [systemd](#systemd)
2. [Getting Started](#getting-started)
- [Quick Start](#quick-start)
- [DKTK](#dktkc4)
- [C4](#c4)
- [GBA/BBMRI-ERIC](#gbabbmri-eric)
@ -132,7 +133,8 @@ If systemd is not installed, you can start the bridgehead. However, for producti
## Getting Started
### Installation
### Quick Start
If your system passed all checks from ["Requirements" section], you are now ready to download the bridgehead.
@ -145,10 +147,6 @@ sudo git clone https://github.com/samply/bridgehead.git /srv/docker/bridgehead;
It is recomended to create a user for the bridgehead service. This should be done after clone the repository. Since not all linux distros support ```adduser```, we provide an action for the systemcall ```useradd```. You should try the first one, when the systm can't create the user you should try the second one.
``` shell
adduser --no-create-home --disabled-login --ingroup docker --gecos "" bridgehead
```
``` shell
useradd -M -g docker -N -s /sbin/nologin bridgehead
```
@ -158,6 +156,25 @@ After adding the User you need to change the ownership of the directory to the b
``` shell
chown bridgehead /srv/docker/bridgehead/ -R
```
Download the configuration repository:
``` shell
sudo git clone https://github.com/samply/bridgehead-config.git -b fix/bbmri-config /etc/bridgehead;
```
Change ownership:
``` shell
chown bridgehead /etc/bridgehead/ -R
```
Modify SITE_ID and SITE_NAME in bbmri.conf
RUN:
```shell
sudo /etc/bridgehead/bridgehead enroll bbmri
```
```shell
sudo /srv/docker/bridgehead/bridgehead start bbmri
```
### Configuration
@ -187,141 +204,22 @@ To shutdown the bridgehead just run.
/srv/docker/bridgehead/bridgehead stop <Project>
```
### Systemd service configuration
### Local Datamanagement Security
For a server, we highly recommend that you install the system units for managing the bridgehead, provided by us. You can do this by executing the [bridgehead](./bridgehead) script:
``` shell
sudo /srv/docker/bridgehead/bridgehead install <Project>
```
This will install the systemd units to run and update the bridghead.
Finally, you need to configure your sites secrets. These are places as configuration for each bridgehead system unit. Refer to the section for your specific project:
For Every project you need to set the proxy this way, if you have one. This is done with the ```systemctl edit``` comand.
``` shell
sudo systemctl edit bridgehead@<project>.service;
sudo systemctl edit bridgehead-update@<project>.service;
```
``` conf
[Service]
Environment=http_proxy=<proxy-url>
Environment=https_proxy=<proxy-url>
```
There a further configurations for each project.
#### CCP(DKTK/C4)
For the federate search please follow the basic auth configuration step.
### DKTK/C4
You can create the site specific configuration with:
This will open your default editor allowing you to edit the docker system units configuration. Insert the following lines in the editor and define your machines secrets. You share some of the ID-Management secrets with the central patientlist (Mainz) and controlnumbergenerator (Frankfurt). Refer to the ["Configuration" section](#configuration) for this.
``` conf
[Service]
Environment=http_proxy=
Environment=https_proxy=
```
To make the configuration effective, you need to tell systemd to reload the configuration and restart the docker service:
``` shell
sudo systemctl daemon-reload;
sudo systemctl bridgehead@ccp.service;
```
You can create the site specific configuration with:
``` shell
sudo systemctl edit bridgehead@c4.service;
```
This will open your default editor allowing you to edit the docker system units configuration. Insert the following lines in the editor and define your machines secrets. You share some of the ID-Management secrets with the central patientlist (Mainz) and controlnumbergenerator (Frankfurt). Refer to the ["Configuration" section](#configuration) for this.
``` conf
[Service]
Environment=http_proxy=
Environment=https_proxy=
Environment=HOSTIP=
Environment=HOST=
Environment=HTTP_PROXY_USER=
Environment=HTTP_PROXY_PASSWORD=
Environment=HTTPS_PROXY_USER=
Environment=HTTPS_PROXY_PASSWORD=
Environment=CONNECTOR_POSTGRES_PASS=
Environment=ML_DB_PASS=
Environment=MAGICPL_API_KEY=
Environment=MAGICPL_MAINZELLISTE_API_KEY=
Environment=MAGICPL_API_KEY_CONNECTOR=
Environment=MAGICPL_MAINZELLISTE_CENTRAL_API_KEY=
Environment=MAGICPL_CENTRAL_API_KEY=
Environment=MAGICPL_OIDC_CLIENT_ID=
Environment=MAGICPL_OIDC_CLIENT_SECRET=
```
To make the configuration effective, you need to tell systemd to reload the configuration and restart the docker service:
``` shell
sudo systemctl daemon-reload;
sudo systemctl bridgehead@c4.service;
```
### GBA/BBMRI-ERIC
You can create the site specific configuration with:
``` shell
sudo systemctl edit bridgehead@gbn.service;
```
This will open your default editor allowing you to edit the docker system units configuration. Insert the following lines in the editor and define your machines secrets.
``` conf
[Service]
Environment=HOSTIP=
Environment=HOST=
Environment=HTTP_PROXY_USER=
Environment=HTTP_PROXY_PASSWORD=
Environment=HTTPS_PROXY_USER=
Environment=HTTPS_PROXY_PASSWORD=
Environment=CONNECTOR_POSTGRES_PASS=
```
To make the configuration effective, you need to tell systemd to reload the configuration and restart the docker service:
``` shell
sudo systemctl daemon-reload;
sudo systemctl bridgehead@gbn.service;
```
## Configuration
This will install the systemd units to run and update the bridghead. Also, this will generate a user and password for accessing the LDM. This will be shown only the first time you install the bridgehead.
### Basic Auth
For Data protection we use basic authenfication for some services. To access those services you need an username and password combination. If you start the bridgehead without basic auth, then those services are not accesbile. We provide a script which set the needed config for you, just run the script and follow the instructions.
For Data protection we use basic authentification for some services. To access those services you need an username and password combination.
Caution: If you start the bridgehead without the authentification, then those services are not accessible.
We generate such a combination at the first install (`/etc/bridgehead/<Project>.local.conf`).
``` shell
add_user.sh
```
The result needs to be set in either in the systemd service or in your console.
#### Console
When just running the bridgehead you need to export the auth variable. Be aware that this export is only for the current session in the environment and after exit it will not be accessible anymore.
``` shell
export bc_auth_user=<output>
```
Cation: you need to escape occrring dollar signs.
## Configuration
#### systemd

14
bbmri/docker-compose.yml
View File

@ -8,19 +8,20 @@ services:
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --providers.docker=true
- --providers.file.watch=true
- --providers.docker.exposedbydefault=false
- --providers.file.directory=/configuration/
- --api.dashboard=true
- --accesslog=true # print access-logs
- --accesslog=true
- --entrypoints.web.http.redirections.entrypoint.to=websecure
- --entrypoints.web.http.redirections.entrypoint.scheme=https
labels:
- "traefik.enable=true"
- "traefik.http.routers.dashboard.rule=PathPrefix(`/api`) || PathPrefix(`/dashboard`)"
- "traefik.http.routers.dashboard.entrypoints=websecure"
- "traefik.http.routers.dashboard.service=api@internal"
- "traefik.http.routers.dashboard.tls=true"
- "traefik.http.routers.dashboard.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=${bc_auth_users}"
- "traefik.http.middlewares.auth.basicauth.users=${LDM_LOGIN}"
ports:
- 80:80
- 443:443
@ -64,11 +65,10 @@ services:
- "blaze-data:/app/data"
labels:
- "traefik.enable=true"
- "traefik.http.middlewares.ccp-auth.basicauth.users=${bc_auth_users}"
- "traefik.http.routers.blaze_ccp.rule=PathPrefix(`/bbmri-localdatamanagement`)"
- "traefik.http.middlewares.ccp_b_strip.stripprefix.prefixes=/bbmri-localdatamanagement"
- "traefik.http.services.blaze_ccp.loadbalancer.server.port=8080"
- "traefik.http.routers.blaze_ccp.middlewares=ccp_b_strip,ccp-auth"
- "traefik.http.routers.blaze_ccp.middlewares=ccp_b_strip,auth"
- "traefik.http.routers.blaze_ccp.tls=true"
spot:
@ -83,8 +83,6 @@ services:
depends_on:
- "beam-proxy"
- "blaze"
labels:
- "traefik.enable=false"
beam-proxy:
image: "samply/beam-proxy:develop"
@ -100,8 +98,6 @@ services:
TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
secrets:
- proxy.pem
labels:
- "traefik.enable=false"
depends_on:
- "forward_proxy"
volumes:

8
bridgehead
View File

@ -58,15 +58,19 @@ if [ -f "$PROJECT/docker-compose.override.yml" ]; then
OVERRIDE+=" -f ./$PROJECT/docker-compose.override.yml"
fi
detectCompose
setHostname
case "$ACTION" in
start)
hc_send log "Bridgehead $PROJECT startup: Checking requirements ..."
checkRequirements
hc_send log "Bridgehead $PROJECT startup: Requirements checked out. Now starting bridgehead ..."
exec docker-compose -f ./$PROJECT/docker-compose.yml $OVERRIDE up --abort-on-container-exit
export LDM_LOGIN=$(getLdmPassword)
exec $COMPOSE -f ./$PROJECT/docker-compose.yml $OVERRIDE up --abort-on-container-exit
;;
stop)
exec docker-compose -f ./$PROJECT/docker-compose.yml $OVERRIDE down
exec $COMPOSE -f ./$PROJECT/docker-compose.yml $OVERRIDE down
;;
update)
exec ./lib/update-bridgehead.sh $PROJECT

14
ccp/docker-compose.yml
View File

@ -8,19 +8,20 @@ services:
- --entrypoints.web.address=:80
- --entrypoints.websecure.address=:443
- --providers.docker=true
- --providers.file.watch=true
- --providers.docker.exposedbydefault=false
- --providers.file.directory=/configuration/
- --api.dashboard=true
- --accesslog=true # print access-logs
- --accesslog=true
- --entrypoints.web.http.redirections.entrypoint.to=websecure
- --entrypoints.web.http.redirections.entrypoint.scheme=https
labels:
- "traefik.enable=true"
- "traefik.http.routers.dashboard.rule=PathPrefix(`/api`) || PathPrefix(`/dashboard`)"
- "traefik.http.routers.dashboard.entrypoints=websecure"
- "traefik.http.routers.dashboard.service=api@internal"
- "traefik.http.routers.dashboard.tls=true"
- "traefik.http.routers.dashboard.middlewares=auth"
- "traefik.http.middlewares.auth.basicauth.users=${bc_auth_users}"
- "traefik.http.middlewares.auth.basicauth.users=${LDM_LOGIN}"
ports:
- 80:80
- 443:443
@ -64,11 +65,10 @@ services:
- "blaze-data:/app/data"
labels:
- "traefik.enable=true"
- "traefik.http.middlewares.ccp-auth.basicauth.users=${bc_auth_users}"
- "traefik.http.routers.blaze_ccp.rule=PathPrefix(`/ccp-localdatamanagement`)"
- "traefik.http.middlewares.ccp_b_strip.stripprefix.prefixes=/ccp-localdatamanagement"
- "traefik.http.services.blaze_ccp.loadbalancer.server.port=8080"
- "traefik.http.routers.blaze_ccp.middlewares=ccp_b_strip,ccp-auth"
- "traefik.http.routers.blaze_ccp.middlewares=ccp_b_strip,auth"
- "traefik.http.routers.blaze_ccp.tls=true"
spot:
@ -83,8 +83,6 @@ services:
depends_on:
- "beam-proxy"
- "blaze"
labels:
- "traefik.enable=false"
beam-proxy:
image: "samply/beam-proxy:develop"
@ -102,8 +100,6 @@ services:
TLS_CA_CERTIFICATES_DIR: /conf/trusted-ca-certs
secrets:
- proxy.pem
labels:
- "traefik.enable=false"
depends_on:
- "forward_proxy"
volumes:

3
ccp/nngm-setup.sh
View File

@ -5,6 +5,5 @@ function nngmSetup() {
log INFO "nNGM setup detected -- will start nNGM Connector."
OVERRIDE+="-f ./$PROJECT/nngm-compose.yml"
fi
CONNECTOR_POSTGRES_PASSWORD="$(echo \"This is a salt string to generate one consistent password. It is not required to be secret.\" | openssl rsautl -sign -inkey /etc/bridgehead/pki/${SITE_ID}.priv.pem | base64 | head -c 30)"
}
CONNECTOR_POSTGRES_PASSWORD="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)"

3
ccp/vars
View File

@ -13,3 +13,6 @@ source $PROJECT/nngm-setup.sh
nngmSetup
source $PROJECT/exliquid-setup.sh
exliquidSetup
# This will load DNPM setup. Effective only if DNPM configuration is defined in /etc/bridgehead/dnpm.
source dnpm/dnpm-setup.sh
dnpmSetup

29
dnpm/dnpm-compose-beamconnect.yml Normal file
View File

@ -0,0 +1,29 @@
version: "3.7"
services:
beam-proxy:
environment:
APP_2_ID: dnpm
APP_2_KEY: ${DNPM_BEAM_SECRET_SHORT}
dnpm-beam-connect:
depends_on: [ beam-proxy ]
image: samply/beam-connect:sites-without-auth
environment:
PROXY_URL: http://beam-proxy:8081
PROXY_APIKEY: ${DNPM_BEAM_SECRET_SHORT}
APP_ID: dnpm.${PROXY_ID}
DISCOVERY_URL: ${DNPM_DISCOVERY_URL}
LOCAL_TARGETS_FILE: /run/secrets/connect_targets.json
HTTP_PROXY: http://forward_proxy:3128
HTTPS_PROXY: http://forward_proxy:3128
NO_PROXY: beam-proxy,dnpm-backend
RUST_LOG: ${RUST_LOG:-info}
secrets:
- connect_targets.json
ports:
- 8062:8062
secrets:
connect_targets.json:
file: /etc/bridgehead/dnpm/local_targets.json

51
dnpm/dnpm-compose-bwhc.yml Normal file
View File

@ -0,0 +1,51 @@
version: "3.7"
services:
dnpm-frontend:
depends_on: [ dnpm-backend ]
build:
context: ../dnpm/origin
dockerfile: Frontend.Dockerfile
network: host
args:
NUXT_HOST: 0.0.0.0
NUXT_PORT: 3000
BACKEND_PROTOCOL: ${DNPM_BMHC_BACKEND_PROTOCOL}
BACKEND_HOSTNAME: ${DNPM_BWHC_BACKEND_HOSTNAME}
BACKEND_PORT: 9000
DNPM_BWHC_FRONTEND_ZIP: ${DNPM_BWHC_FRONTEND_ZIP}
HTTP_PROXY: ${http_proxy}
HTTPS_PROXY: ${https_proxy}
ports:
- 3000:3000
environment:
BACKEND_PROTOCOL: ${DNPM_BMHC_BACKEND_PROTOCOL}
BACKEND_HOSTNAME: ${DNPM_BWHC_BACKEND_HOSTNAME}
BACKEND_PORT: 9000
no_proxy: dnpm-backend
dnpm-backend:
build:
context: ../dnpm/origin
dockerfile: Backend.Dockerfile
args:
BWHC_BASE_DIR: /bwhc-backend
DNPM_BWHC_BACKEND_ZIP: ${DNPM_BWHC_BACKEND_ZIP}
ports:
- 9000:9000
environment:
APPLICATION_SECRET: ${DNPM_APPLICATION_SECRET}
ZPM_SITE: ${ZPM_SITE}
noproxy: dnpm-frontend,connect
# PLAY_HTTP_PORT: 9000
# PLAY_HTTP_ADDRESS: 0.0.0.0
volumes:
- ../dnpm/origin/logback.xml:/bwhc-backend/logback.xml:ro
- /etc/bridgehead/dnpm/bwhcConnectorConfig.xml:/bwhc-backend/bwhcConnectorConfig.xml:ro
- /etc/bridgehead/dnpm/production.conf:/bwhc-backend/production.conf:ro
- bwhc_data:/bwhc-backend/data/
- bwhc_hgnc_data:/bwhc-backend/hgnc_data/
volumes:
bwhc_data:
bwhc_hgnc_data:

16
dnpm/dnpm-setup.sh Normal file
View File

@ -0,0 +1,16 @@
#!/bin/bash
function dnpmSetup() {
if [ -e /etc/bridgehead/dnpm/local_targets.json ]; then
log INFO "DNPM setup detected (Beam.Connect) -- will start Beam.Connect for DNPM."
OVERRIDE+=" -f ./dnpm/dnpm-compose-beamconnect.yml"
DNPM_APPLICATION_SECRET="$(echo \"This is a salt string to generate one consistent password. It is not required to be secret.\" | openssl rsautl -sign -inkey /etc/bridgehead/pki/${SITE_ID}.priv.pem | base64 | head -c 30)"
DNPM_BEAM_SECRET_SHORT="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 20)"
source /etc/bridgehead/dnpm/shared-but-secret-vars || fail_and_report 1 "Unable to load /etc/bridgehead/dnpm/shared-but-secret-vars"
export DNPM_DISCOVERY_URL
if [ -e /etc/bridgehead/dnpm/bwhcConnectorConfig.xml ]; then
log INFO "DNPM setup detected (with Frontend/Backend) -- will start BWHC Frontend/Backend."
OVERRIDE+=" -f ./dnpm/dnpm-compose-bwhc.yml"
fi
fi
}

66
dnpm/origin/Backend.Dockerfile Normal file
View File

@ -0,0 +1,66 @@
FROM openjdk:11-jre AS builder
ARG DNPM_BWHC_BACKEND_ZIP
# Change to latest release
ARG VERSION=broker
ARG BWHC_BASE_DIR=/bwhc-backend
ENV BWHC_BASE_DIR=$BWHC_BASE_DIR
ENV BWHC_USER_DB_DIR=$BWHC_BASE_DIR/data/user-db
ENV BWHC_DATA_ENTRY_DIR=$BWHC_BASE_DIR/data/data-entry
ENV BWHC_QUERY_DATA_DIR=$BWHC_BASE_DIR/data/query-data
ADD ${DNPM_BWHC_BACKEND_ZIP} /
RUN unzip $(basename ${DNPM_BWHC_BACKEND_ZIP}) && rm $(basename ${DNPM_BWHC_BACKEND_ZIP})
WORKDIR $BWHC_BASE_DIR
# Prepare config file to use environment variables from docker
RUN sed -i -r "s/APPLICATION_SECRET(.*)/#APPLICATION_SECRET\1/" ./config
RUN sed -i -r "s/ZPM_SITE(.*)/#ZPM_SITE\1/" ./config
# Prepare config file to use fix environment variables for this image
RUN sed -i -r "s~BWHC_DATA_ENTRY_DIR.*~BWHC_DATA_ENTRY_DIR=$BWHC_DATA_ENTRY_DIR~" ./config
RUN sed -i -r "s~BWHC_QUERY_DATA_DIR.*~BWHC_QUERY_DATA_DIR=$BWHC_QUERY_DATA_DIR~" ./config
RUN sed -i -r "s~BWHC_USER_DB_DIR.*~BWHC_USER_DB_DIR=$BWHC_USER_DB_DIR~" ./config
RUN ./install.sh $BWHC_BASE_DIR
RUN mv bwhc-rest-api-gateway-*/ bwhc-rest-api-gateway/
FROM openjdk:11-jre
ARG BWHC_BASE_DIR=/bwhc-backend
ENV BWHC_BASE_DIR=$BWHC_BASE_DIR
ENV BWHC_USER_DB_DIR=$BWHC_BASE_DIR/data/user-db
ENV BWHC_DATA_ENTRY_DIR=$BWHC_BASE_DIR/data/data-entry
ENV BWHC_QUERY_DATA_DIR=$BWHC_BASE_DIR/data/query-data
ENV BWHC_CONNECTOR_CONFIG=$BWHC_BASE_DIR/bwhcConnectorConfig.xml
COPY --from=builder $BWHC_BASE_DIR/config $BWHC_BASE_DIR/
COPY --from=builder $BWHC_BASE_DIR/bwhcConnectorConfig.xml $BWHC_BASE_DIR/
COPY --from=builder $BWHC_BASE_DIR/logback.xml $BWHC_BASE_DIR/
COPY --from=builder $BWHC_BASE_DIR/production.conf $BWHC_BASE_DIR/
COPY --from=builder $BWHC_BASE_DIR/bwhc-rest-api-gateway/ $BWHC_BASE_DIR/bwhc-rest-api-gateway/
VOLUME $BWHC_BASE_DIR/data
VOLUME $BWHC_BASE_DIR/hgnc_data
EXPOSE ${BWHC_BACKEND_PORT}
WORKDIR $BWHC_BASE_DIR
CMD $BWHC_BASE_DIR/bwhc-rest-api-gateway/bin/bwhc-rest-api-gateway \
-Dplay.http.secret.key=$APPLICATION_SECRET \
-Dconfig.file=$BWHC_BASE_DIR/production.conf \
-Dlogger.file=$BWHC_BASE_DIR/logback.xml \
-Dpidfile.path=/dev/null \
-Dbwhc.zpm.site=$ZPM_SITE \
-Dbwhc.data.entry.dir=$BWHC_DATA_ENTRY_DIR \
-Dbwhc.query.data.dir=$BWHC_QUERY_DATA_DIR \
-Dbwhc.user.data.dir=$BWHC_USER_DB_DIR \
-Dbwhc.hgnc.dir=$BWHC_HGNC_DIR \
-Dbwhc.connector.configFile=$BWHC_CONNECTOR_CONFIG

42
dnpm/origin/Frontend.Dockerfile Normal file
View File

@ -0,0 +1,42 @@
FROM node:10-alpine
ARG DNPM_BWHC_FRONTEND_ZIP
# Change to latest release
# Required for image build using local copy of zip file
ARG VERSION=2207
# nuxt host and port to be replaced in package.json. (See 2.3 in bwHCPrototypeManual)
# NUXT_HOST should have a value with public available IP address from within container.
# If changing NUXT_PORT, also change exposed port.
ARG NUXT_HOST=0.0.0.0
ARG NUXT_PORT=3000
# Backend access setup. (See 2.4 in bwHCPrototypeManual)
ARG BACKEND_PROTOCOL=http
ARG BACKEND_HOSTNAME=localhost
ARG BACKEND_PORT=8080
ARG HTTP_PROXY=""
ARG HTTPS_PROXY=""
ADD ${DNPM_BWHC_FRONTEND_ZIP} /
RUN unzip $(basename ${DNPM_BWHC_FRONTEND_ZIP}) && rm $(basename ${DNPM_BWHC_FRONTEND_ZIP})
WORKDIR /bwhc-frontend
RUN npm install
# Prepare package.json
RUN sed -i -r "s/^(\s*)\"host\"[^,]*(,?)/\1\"host\": \"$NUXT_HOST\"\2/" ./package.json
RUN sed -i -r "s/^(\s*)\"port\"[^,]*(,?)/\1\"port\": \"$NUXT_PORT\"\2/" ./package.json
# Prepare nuxt.config.js
RUN sed -i -r "s/^(\s*)baseUrl[^,]*(,?)/\1baseUrl: process.env.BASE_URL || '$BACKEND_PROTOCOL:\/\/$BACKEND_HOSTNAME'\2/" ./nuxt.config.js
RUN sed -i -r "s/^(\s*)port[^,]*(,?)/\1port: process.env.port || ':$BACKEND_PORT'\2/" ./nuxt.config.js
RUN npm run generate
EXPOSE $NUXT_PORT
CMD npm start

37
dnpm/origin/logback.xml Normal file
View File

@ -0,0 +1,37 @@
<?xml version="1.0" encoding="utf-8"?>
<configuration scan="true">
<property name="LOG_DIR" value="./bwhc_logs/"/>
<property name="LOG_FILE" value="bwhealthcloud"/>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<!--
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_DIR}/${LOG_FILE}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_DIR}/${LOG_FILE}-%d{yyyy-MM-dd}.log</fileNamePattern>
<maxHistory>30</maxHistory>
<totalSizeCap>3GB</totalSizeCap>
</rollingPolicy>
<encoder>
<pattern>%d [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
-->
<root level="DEBUG">
<appender-ref ref="STDOUT"/>
<!--
<appender-ref ref="FILE"/>
-->
</root>
</configuration>

10
lib/add_bc_user.sh
View File

@ -1,10 +0,0 @@
#!/bin/bash -e
source lib/functions.sh
log "INFO" "This script add's a user with password to the bridghead"
read -p 'Username: ' bc_user
read -sp 'Password: ' bc_password
log "INFO" "\nPlease export the line in the your environment. Please replace the dollar signs with with \\\$"
docker run --rm -it httpd:latest htpasswd -nb $bc_user $bc_password

37
lib/functions.sh Executable file → Normal file
View File

@ -2,6 +2,23 @@
source lib/log.sh
detectCompose() {
if [[ "$(docker compose version 2>/dev/null)" == *"Docker Compose version"* ]]; then
COMPOSE="docker compose"
else
COMPOSE="docker-compose"
# This is intended to fail on startup in the next prereq check.
fi
}
getLdmPassword() {
if [ -n "$LDM_PASSWORD" ]; then
docker run --rm httpd:alpine htpasswd -nb $PROJECT $LDM_PASSWORD | tr -d '\n' | tr -d '\r'
else
echo -n ""
fi
}
exitIfNotRoot() {
if [ "$EUID" -ne 0 ]; then
log "ERROR" "Please run as root"
@ -102,19 +119,23 @@ fixPermissions() {
source lib/monitoring.sh
fail_and_report() {
report_error() {
log ERROR "$2"
hc_send $1 "$2"
}
fail_and_report() {
report_error $@
exit $1
}
setHostname() {
if [ -z "$HOST" ]; then
export HOST=$(hostname -f)
log DEBUG "Using auto-detected hostname $HOST."
fi
}
##Setting Network properties
# currently not needed
#export HOSTIP=$(MSYS_NO_PATHCONV=1 docker run --rm --add-host=host.docker.internal:host-gateway ubuntu cat /etc/hosts | grep 'host.docker.internal' | awk '{print $1}');
export HOST=$(hostname -f)
export PRODUCTION="false";
if [ "$(git branch --show-current)" == "main" ]; then
export PRODUCTION="true";
fi

11
lib/monitoring.sh
View File

@ -11,6 +11,7 @@ function hc_set_service(){
}
UPTIME=
USER_AGENT=
function hc_send(){
if [ -n "$MONITOR_APIKEY" ]; then
@ -32,10 +33,16 @@ function hc_send(){
UPTIME=$(docker ps -a --format 'table {{.Names}} \t{{.RunningFor}} \t {{.Status}} \t {{.Image}}' --filter name=bridgehead || echo "Unable to get docker statistics")
fi
if [ -z "$USER_AGENT" ]; then
COMMIT_ETC=$(git -C /etc/bridgehead rev-parse HEAD | cut -c -8)
COMMIT_SRV=$(git -C /srv/docker/bridgehead rev-parse HEAD | cut -c -8)
USER_AGENT="srv:$COMMIT_SRV etc:$COMMIT_ETC"
fi
if [ -n "$2" ]; then
MSG="$2\n\nDocker stats:\n$UPTIME"
echo -e "$MSG" | https_proxy=$HTTPS_PROXY_URL curl -s -o /dev/null -X POST --data-binary @- "$HCURL"/"$1" || log WARN "Monitoring failed: Unable to send data to $HCURL/$1"
echo -e "$MSG" | https_proxy=$HTTPS_PROXY_URL curl -A "$USER_AGENT" -s -o /dev/null -X POST --data-binary @- "$HCURL"/"$1" || log WARN "Monitoring failed: Unable to send data to $HCURL/$1"
else
https_proxy=$HTTPS_PROXY_URL curl -s -o /dev/null "$HCURL"/"$1" || log WARN "Monitoring failed: Unable to send data to $HCURL/$1"
https_proxy=$HTTPS_PROXY_URL curl -A "$USER_AGENT" -s -o /dev/null "$HCURL"/"$1" || log WARN "Monitoring failed: Unable to send data to $HCURL/$1"
fi
}

4
lib/prerequisites.sh
View File

@ -2,6 +2,8 @@
source lib/functions.sh
detectCompose
if ! id "bridgehead" &>/dev/null; then
log ERROR "User bridgehead does not exist. Please consult readme for installation."
exit 1
@ -12,7 +14,7 @@ checkOwner /etc/bridgehead bridgehead || exit 1
## Check if user is a su
log INFO "Checking if all prerequisites are met ..."
prerequisites="git docker docker-compose"
prerequisites="git docker"
for prerequisite in $prerequisites; do
$prerequisite --version 2>&1
is_available=$?

9
lib/setup-bridgehead-units.sh
View File

@ -33,6 +33,15 @@ Cmnd_Alias BRIDGEHEAD${PROJECT^^} = \\
bridgehead ALL= NOPASSWD: BRIDGEHEAD${PROJECT^^}
EOF
# TODO: Determine wether this should be located in setup-bridgehead (triggered through bridgehead install) or in update bridgehead (triggered every hour)
if [ -z "$LDM_PASSWORD" ]; then
log "INFO" "Now generating a password for the local data management. Please save the password for your ETL process!"
generated_passwd="$(cat /proc/sys/kernel/random/uuid | sed 's/[-]//g' | head -c 32)"
log "INFO" "Your generated credentials are:\n user: $PROJECT\n password: $generated_passwd"
echo -e "## Local Data Management Basic Authentication\n# User: $PROJECT\nLDM_PASSWORD=$generated_passwd" >> /etc/bridgehead/${PROJECT}.local.conf;
fi
log "INFO" "Register system units for bridgehead and bridgehead-update"
cp -v \
lib/systemd/bridgehead\@.service \

6
lib/traefik-configuration/certificates.yaml
View File

@ -1,4 +1,6 @@
tls:
certificates:
- certFile: /certs/fullchain.pem
stores:
default:
defaultCertificate:
certFile: /certs/fullchain.pem
keyFile: /certs/privkey.pem

15
lib/update-bridgehead.sh
View File

@ -36,6 +36,11 @@ CHANGES=""
git_updated="false"
for DIR in /etc/bridgehead $(pwd); do
log "INFO" "Checking for updates to git repo $DIR ..."
OUT="$(git -C $DIR status --porcelain)"
if [ -n "$OUT" ]; then
log WARN "The working directory $DIR is modified. Changed files: $OUT"
report_error log "The working directory $DIR is modified. Changed files: $OUT"
fi
if [ "$(git -C $DIR config --get credential.helper)" != "$CREDHELPER" ]; then
log "INFO" "Configuring repo to use bridgehead git credential helper."
git -C $DIR config credential.helper "$CREDHELPER"
@ -43,13 +48,15 @@ for DIR in /etc/bridgehead $(pwd); do
old_git_hash="$(git -C $DIR rev-parse --verify HEAD)"
if [ -z "$HTTP_PROXY_URL" ]; then
log "INFO" "Git is using no proxy!"
git -C $DIR fetch 2>&1
git -C $DIR pull 2>&1
OUT=$(git -C $DIR fetch 2>&1 && git -C $DIR pull 2>&1)
else
log "INFO" "Git is using proxy ${HTTP_PROXY_URL} from ${CONFFILE}"
git -c http.proxy=$HTTP_PROXY_URL -c https.proxy=$HTTPS_PROXY_URL -C $DIR fetch 2>&1
git -c http.proxy=$HTTP_PROXY_URL -c https.proxy=$HTTPS_PROXY_URL -C $DIR pull 2>&1
OUT=$(git -c http.proxy=$HTTP_PROXY_URL -c https.proxy=$HTTPS_PROXY_URL -C $DIR fetch 2>&1 && git -c http.proxy=$HTTP_PROXY_URL -c https.proxy=$HTTPS_PROXY_URL -C $DIR pull 2>&1)
fi
if [ $? -ne 0 ]; then
report_error log "Unable to update git $DIR: $OUT"
fi
new_git_hash="$(git -C $DIR rev-parse --verify HEAD)"
if [ "$old_git_hash" != "$new_git_hash" ]; then
CHANGE="Updated git repository in ${DIR} from commit $old_git_hash to $new_git_hash"