Deploy Docker SWAG with docker-compose (beta)

The new version of Docker-Compose deployment for SWAG open-appsec integration provided here is currently in beta status. You find the current General-Availability deployment instructions for SWAG here: Install Docker SWAG with open-appsec (locally managed) Install Docker SWAG with open-appsec (centrally managed)

This new deployment option with Docker-Compose introduces additional containers in case of a standalone deployment (= deployment without any connection to WebUI). These containers introduce some additional functionality, which is relevant for standalone deployments only (already available when connected to central management WebUI): - Local syncing of learning between open-appsec processes and agents - View learning progress on CLI with open-appsec-tuning-tool - Receive configuration recommendations based on the learning progress using the open-appsec-tuning-tool - Receive tuning suggestions and manage them (approve/reject) using the open-appsec-tuning-tool More details on the open-appsec-tuning-tool here: Track Learning and Local Tuning in Standalone Deployments

Prerequisites

• Linux Docker Host with root permission

• Docker-Compose tool installed

• (Optional, Recommended) Sign-Up and Login to WebUI Portal If you want to centrally manage your open-appsec WAF deployment via WebUI (SaaS) OR if you want to locally manage your open-appsec WAF deployment but still connect to central WebUI for viewing the local configuration (in read-only), central monitoring, logging and reporting: Follow the instructions below to sign-up and login to the WebUI available at https://my.openappsec.io:

Sign-Up and Login to Portal

• (Optional, Recommended) Create deployment profile for the open-appsec deployment in WebUI Portal If you signed-up and logged in to the WebUI Portal (see prerequisite above), now follow the instructions below to create a new deployment profile for your open-appsec deployment. Once done, don't forget to copy the profile token after policy installation as this is needed in the installation steps further below.

Create a Profile

Deployment

To deploy open-appsec with docker-compose and optionally connect to the central WebUI available at https://my.openappsec.io follow the steps below:

1. Create a folder for your new open-appsec deployment and switch to that folder, e.g.

mkdir open-appsec-deployment
cd ./open-appsec-deployment

1. Download the docker compose file for your desired open-appsec integration

wget https://raw.githubusercontent.com/openappsec/openappsec/main/deployment/docker-compose/swag/docker-compose.yaml

docker-compose.yaml file content

# Copyright (C) 2022 Check Point Software Technologies Ltd. All rights reserved.

# Licensed under the Apache License, Version 2.0 (the "License");
# You may obtain a copy of the License at

#     http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

##
## Docker compose file for open-appsec integrated with SWAG
##

version: "3.9"
services:
  appsec-agent:
    image: ghcr.io/openappsec/agent:${APPSEC_VERSION}
    container_name: appsec-agent
    restart: unless-stopped
    environment:
      - SHARED_STORAGE_HOST=appsec-shared-storage
      - LEARNING_HOST=appsec-smartsync
      - TUNING_HOST=appsec-tuning-svc
      - https_proxy=${APPSEC_HTTPS_PROXY}
      - user_email=${APPSEC_USER_EMAIL}
      - AGENT_TOKEN=${APPSEC_AGENT_TOKEN}
      - autoPolicyLoad=${APPSEC_AUTO_POLICY_LOAD}
      - registered_server=SWAG Server
    ipc: shareable
    volumes:
      - ${APPSEC_CONFIG}:/etc/cp/conf
      - ${APPSEC_DATA}:/etc/cp/data
      - ${APPSEC_LOGS}:/var/log/nano_agent
      - ${APPSEC_LOCALCONFIG}:/ext/appsec
    command: /cp-nano-agent

  appsec-swag:
    image: ghcr.io/openappsec/swag-attachment:latest
    container_name: appsec-swag
    ipc: service:appsec-agent
    restart: unless-stopped
    cap_add:
      - NET_ADMIN
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=${SWAG_TZ}
      - URL=${SWAG_URL}
      - VALIDATION=${SWAG_VALIDATION}
      - DNSPLUGIN=${SWAG_DNSPLUGIN}
      - AWS_ACCESS_KEY_ID=${SWAG_AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${SWAG_AWS_SECRET_ACCESS_KEY}
      - SUBDOMAINS=${SWAG_SUBDOMAINS}
      - ONLY_SUBDOMAINS=${SWAG_ONLY_SUBDOMAINS}
## see https://docs.linuxserver.io/images/docker-swag/ for
## more cert generation/validation options
      - STAGING=${SWAG_STAGING}
    volumes:
      - ${SWAG_CONFIG}:/config
      - ${SWAG_NGINX_SITE_CONFS}:/config/nginx/site-confs
      - ${SWAG_PROXY_CONFS}:/config/nginx/proxy-confs
    ports:
      - 443:443
      - 80:80 ## optional

  appsec-smartsync:
    profiles:
      - standalone
    image: ghcr.io/openappsec/smartsync:${APPSEC_VERSION}
    container_name: appsec-smartsync
    environment:
      - SHARED_STORAGE_HOST=appsec-shared-storage
    restart: unless-stopped
    depends_on:
      - appsec-shared-storage

  appsec-shared-storage:
    profiles:
      - standalone
    image: ghcr.io/openappsec/smartsync-shared-files:${APPSEC_VERSION}
    container_name: appsec-shared-storage
    ipc: service:appsec-agent
    restart: unless-stopped
## if you do not want to run this container as "root" user you can comment it out and instead run the below command after the deployment
## docker exec -u root appsec-shared-storage chown -R appuser:appuser /db
    user: root
    volumes:
      - ${APPSEC_SMART_SYNC_STORAGE}:/db:z
## instead of using local storage for local learning (see line above)
## you can also configure central nfs storage by configuring nfs volume (uncomment the relevant section at end of this file)
## use a shared nfs storage which is recommended in redundant deployments (uncomment line below, comment out the line above) 
#     - learning_nfs:/db:z

  appsec-tuning-svc:
    profiles:
      - standalone
    image: ghcr.io/openappsec/smartsync-tuning:${APPSEC_VERSION}
    container_name: appsec-tuning-svc
    environment:
      - SHARED_STORAGE_HOST=appsec-shared-storage
      - QUERY_DB_PASSWORD=${APPSEC_DB_PASSWORD}
      - QUERY_DB_HOST=${APPSEC_DB_HOST}
      - QUERY_DB_USER=${APPSEC_DB_USER}
## only relevant when deploying own DB 
#     - SSLMODE:
    restart: unless-stopped
    volumes:
      - ${APPSEC_CONFIG}:/etc/cp/conf
    depends_on:
      - appsec-shared-storage
      - appsec-db
      
  appsec-db:
    profiles:
      - standalone
    image: postgres
    container_name: appsec-db
    restart: unless-stopped
    environment:
      - POSTGRES_PASSWORD=${APPSEC_DB_PASSWORD}
      - POSTGRES_USER=${APPSEC_DB_USER} 
    volumes:
      - ${APPSEC_POSTGRES_STORAGE}:/var/lib/postgresql/data

## example juice-shop backend container (vulnerable webserver, USE ONLY FOR TESTING AND IN LAB ENV)
  juiceshop-backend:
    image: bkimminich/juice-shop:latest
    container_name: juiceshop-backend


## advanced configuration: learning_nfs volume for nfs storage in shared_storage container
##
## when configuring nfs storage in shared_storage container configuration above, make sure to also specify learning_nfs volume (see example below for using AWS EFS storage)
##
#volumes:
# learning_nfs:
#   driver: local
#   driver_opts:
#     type: nfs
#     o: addr=fs-abcdef.efs.eu-west-1.amazonaws.com,rw,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport
#     device: ":/"

Download the .env file for your desired open-appsec integration and adjust the configuration aligned with your own requirements as described below:

Download the default .env file here:

wget https://raw.githubusercontent.com/openappsec/openappsec/main/deployment/docker-compose/swag/.env

.env file content

## .env file for docker-compose deployments of open-appsec integrated with SWAG
## for more info see https://docs.openappsec.io

APPSEC_VERSION=latest
APPSEC_CONFIG=./appsec-config
APPSEC_DATA=./appsec-data
APPSEC_LOGS=./appsec-logs
APPSEC_LOCALCONFIG=./appsec-localconfig

## Make sure the parameter APPSEC_AUTO_POLICY_LOAD is set to false when centrally managing
## open-appsec configuration via open-appsec Web UI.
## You can optionally set it to true when using local, declarative management for open-appsec,
## declarative configuration will then get applied automatically when changed.
APPSEC_AUTO_POLICY_LOAD=false

## Example for configuring HTTPS Proxy:
## APPSEC_HTTPS_PROXY=user:password@proxy_address:port
APPSEC_HTTPS_PROXY=

APPSEC_SMART_SYNC_STORAGE=./appsec-smartsync-storage
APPSEC_USER_EMAIL=[e-mail]
APPSEC_DB_PASSWORD=pass
APPSEC_DB_USER=postgres
APPSEC_DB_HOST=appsec-db
APPSEC_POSTGRES_STORAGE=./appsec-postgres-data

## Most relevant SWAG parameters have been moved here as well allowing configuration via .env file 
SWAG_CONFIG=./swag-config
## Make sure to have a valid nginx config default.conf in SWAG_NGINX_SITE_CONFS folder
SWAG_NGINX_SITE_CONFS=./swag-nginx-site-confs
## Make sure to have valid *.conf proxy configuration in SWAG_NGINX_PROXY_CONFS folder
SWAG_PROXY_CONFS=./swag-proxy-confs
## For deployment of a simple lab testing environment, you can deploy the example configuration provided
## for the vulnerable juice-shop container, see instructions further below.

SWAG_TZ=Etc/UTC
SWAG_VALIDATION=http # configure "http" or "dns" as validation modes 
SWAG_DNSPLUGIN="" # configure e.g. "route53" or some other DNS Plugin supported by SWAG if you set "dns" above

## Examples parameters for "route53" DNS plugin (AWS DNS service), you can add others here as required,
## when you do make sure to also add them to the docker compose file
SWAG_AWS_ACCESS_KEY_ID=""
SWAG_AWS_SECRET_ACCESS_KEY=""
##

SWAG_STAGING=true ## switch to 'false' after successful testing
SWAG_URL=yourdomain.url
SWAG_SUBDOMAINS=""
SWAG_ONLY_SUBDOMAINS=""
## replace yourdomain.url with your own domain
## make sure your domain's public IP resolves to
## the docker host for Let's Encrypt cert generation to succeed

## To connect your deployment to central open-appsec WebUI provide the token for a profile
## which you created in open-appsec WebUI at https://my.openappsec.io
## Example: APPSEC_AGENT_TOKEN=111-22222-111
APPSEC_AGENT_TOKEN=

## Important: When not providing token for connection to central WebUI:
## Make sure to add the value "standalone" to the COMPOSE_PROFILES value, this will enable
## sharing of learning between processes and allow you to perform tuning locally on CLI
COMPOSE_PROFILES=

## JUICE SHOP DEMO CONTAINER:
## In order to deploy the optional, additional, vulnerable juiceshop container (for demo and testing purposes only!):
## Add the value "juiceshop" to the COMPOSE_PROFILES value above.

## Make sure to put a juiceshop.subfolder.conf file in SWAG_PROXY_CONFS folder
## for proxying external traffic to the juiceshop-backend container and also adjust the NGINX default.conf file in SWAG_NGINX_SITE_CONFS folder
## you can use the example files available here:
## https://raw.githubusercontent.com/openappsec/openappsec/examples/juiceshop/swag/juiceshop.subfolder.conf
## https://raw.githubusercontent.com/openappsec/openappsec/examples/juiceshop/swag/default.conf
## note that juiceshop container listens on HTTP port 3000 by default

## Note that COMPOSE_PROFILES can also receive multiple values, e.g. as shown here:
## COMPOSE_PROFILES=standalone,juiceshop

• If you created a deployment profile in the WebUI and copied the Token from it:

  Edit the .env file and add your token to the key APPSEC_AGENT_TOKEN.

• If you did not create a deployment profile in the WebUI and do not want to connect your deployment to central WebUI (SaaS) at all:

  Set the value standalone for the key COMPOSE_PROFILES which will activate the deployment of additional containers which are required only when not connected to the WebUI at all (resulting in standalone, locally, declaratively managed deployment).

• Replace [e-mail] in the .env file with your own email. (More details below.)

Available settings in the .env file allowing further customization of the deployment:

APPSEC_AGENT_TOKEN: For connecting your open-appsec deployment to central WebUI set APPSEC_AGENT_TOKEN to your own deployment profile token as copied from profile settings in the open-appsec central WebUI (see section Prerequisites above).

COMPOSE_PROFILES: Possible values you can set for this key: (you can set multiple values, separated by comma)

standalone : This will activate the deployment of additional containers which are required only when you are not connected to the WebUI at all, resulting in standalone, locally, declaratively managed deployment.

Only activate the standalone profile in case you did not set a WebUI deployment profile token as value for the APPSEC_AGENT_TOKEN key (see above)!

juiceshop : This will deploy an additional, vulnerable juiceshop-backend container that can be used for demo and testing purposes. In the .env file you also find a download link for the proxy-specific configuration allowing you to access the juiceshop backend via the proxy. More info on the OWASP juiceshop project: https://owasp.org/www-project-juice-shop/

Do not activate the juiceshop profile in production environments as the juiceshop container is intentionally highly vulnerable and meant for testing in lab environments only!

USER_EMAIL: (Optional) Associate your email address with your specific deployment by replacing [e-mail] with your own email address.

This allows the open-appsec team to provide you easy assistance in case of any issues you might have with your specific deployment in the future and also to provide you information proactively regarding open-appsec in general or regarding your specific deployment. This is an optional parameter and can be removed. If we send automatic emails there will also be an opt-out option included for receiving similar communication in the future.

APPSEC_HTTPS_PROXY: (Optional) Configure an HTTP(S) proxy server to be used by the agent.

APPSEC_AUTO_POLICY_LOAD: (Optional) When set to true, allows you to set the open-appsec agent to automatically apply any new changes in the local_policy.yaml file without having to restart the agent container or applying the changes with open-appsec-ctl -ap (note that this can take up to 30 seconds). This is useful especially in DevOps scenarios with continuous deployment scenarios.

APPSEC_VERSION: Allows you to specify a specific version for deployment instead of using the default latest version for the containers provided by open-appsec (not relevant for postgres container).

Additional configuration available specifically for this integration type:

SWAG_CONFIG: Set the directory on the docker host for the volume mount of the SWAG configuration.

SWAG_NGINX_SITE_CONFS: Set the directory on the docker host for the volume mount of the NGINX site configuration, this gets mounted to /config/nginx/proxy-confs within the appsec-swag container.

SWAG_PROXY_CONFS: Set the directory on the docker host for the volume mount of the NGINX proxy configurations, this gets mounted to /config/nginx/proxy-confs within the appsec-swag container.

SWAG_TZ: Specify a timezone to use, see this list.

SWAG_VALIDATION: Certbot validation method to use, options are http or dns (dns method also requires DNSPLUGIN variable set) .

SWAG_DNSPLUGIN: Required if VALIDATION is set to dns. Options are acmedns, aliyun, azure, bunny, cloudflare, cpanel, desec, digitalocean, directadmin, dnsimple, dnsmadeeasy, dnspod, do, domeneshop, dreamhost, duckdns, dynu, freedns, gandi, gehirn, glesys, godaddy, google, he, hetzner, infomaniak, inwx, ionos, linode, loopia, luadns, namecheap, netcup, njalla, nsone, ovh, porkbun, rfc2136, route53, sakuracloud, standalone, transip, and vultr. Also need to enter the credentials into the corresponding ini (or json for some plugins) file under /config/dns-conf.

SWAG_STAGING: Set to true to retrieve certs in staging mode. Rate limits will be much higher, but the resulting cert will not pass the browser's security test. Only to be used for testing purposes.

SWAG_URL: Top url you have control over (e.g. example.com if you own it, or customsubdomain.example.com if dynamic dns).

SWAG_SUBDOMAINS: Subdomains you'd like the cert to cover (comma separated, no spaces) ie. www,ftp,cloud. For a wildcard cert, set this exactly to wildcard (wildcard cert is available via dns validation only)

SWAG_ONLY_SUBDOMAINS: If you wish to get certs only for certain subdomains, but not the main domain (main domain may be hosted on another machine and cannot be validated), set this to true

Here you find additional available env variables that you can use with the appsec-swag container. Those, when required, should be configured directly in the docker-compose file, if they are not listed above. https://github.com/linuxserver/docker-swag

For testing purposes in a lab environment you can activate the deployment of the vulnerable juiceshop-backend container via COMPOSE_PROFILES key (see above) and then deploy the available configuration example for exposing it via the proxy, which is provided by the open-appsec team (download link is provided in the .env file).

You find some additional advanced configuration options described within the docker-compose.yaml file as comments.

1. If you decided to locally, declaratively manage open-appsec with local_policy.yaml file: Download the initial declarative configuration file for open-appsec into new subfolder ./appsec-localconfig:

   mkdir ./appsec-localconfig
   wget https://raw.githubusercontent.com/openappsec/openappsec/main/config/linux/v1beta1/prevent/local_policy.yaml -O ./appsec-localconfig/local_policy.yaml

This example configuration file is already set to mode: prevent-learnso that open-appsec will prevent attacks right from the start. Here's the path for an alternative local-config.yaml file set to detect-learn mode. https://raw.githubusercontent.com/openappsec/openappsec/main/config/linux/v1beta1/detect/local_policy.yaml (or simply adjust the setting in the mode setting in the earlier local_policy.yaml file to detect-learn) In production environments it's always recommended to start in detect-learn mode to allow open-appsec to achieve a certain learning level based on traffic observed before moving to prevent-learnfor better detection accuracy and strongly reduced false positives. Read more about this here: Track Learning and Move From Learn/Detect to Prevent

If you migrate from an existing Docker SWAG deployment to Docker SWAG with open-appsec make sure to make all adjustments you did in your existing docker-compose.yaml for SWAG also in the one above and/or in the .env file before deployment. Also make sure to copy your existing SWAG configuration folder contents into ./swag-config on the docker host.

1. Perform the deployment

docker-compose up -d

You will see output similar to the below (example screenshot is from an NGINX deployment). Note that the amount of container will vary between deployments with and without connection to central WebUI.

1. Verify that all containers are up and running by verifying their status in docker ps output. Note that the amount of container will vary based between deployments with and without connection to central WebUI.

docker ps

You will see output similar to the below (screenshot is from NGINX deployment):

docker ps example output with all containers up

1. Configure SWAG’s NGINX reverse proxy to forward traffic to your backend webserver(s). (if you don’t have experience with this, see SWAG docs: SWAG - LinuxServer.io) If you activated deployment of the juiceshop-backend container (not safe for production environments, just use for lab testing purposes) by adding juiceshop to the COMPOSE_PROFILES parameter in the .env file, then you can download and use the following two example files into the mounted folders allowing you to easily configure swag to proxy inbound traffic to the juiceshop-backend container:

wget https://raw.githubusercontent.com/openappsec/openappsec/examples/juiceshop/swag/juiceshop.subfolder.conf -O ./swag-proxy-confs/juiceshop.subfolder.conf

wget https://raw.githubusercontent.com/openappsec/openappsec/examples/juiceshop/swag/default.conf -O ./swag-nginx-site-confs/default.conf

1. To verify if the SSL certificate was successfully issued by Let's Encrypt you can use the following command to show logs of the swag-attachment container:

docker logs swag-attachment

1. If certificate was successfully generated in Let’s Encrypt staging mode (recommended for testing, this will generate a non-trusted certificate only but allows for more failed validations before blocking for certain time) you should now change the following line in the docker-compose.yaml file (in the swag-attachment container specification) to disable staging mode and receive a valid, trusted certificate from Let’s Encrypt: Old value: - STAGING=true New value: - STAGING=false Redeploy to apply the changes:

docker compose down && docker compose up -d

1. SWAG also contains fail2ban as an additional security layer. You can check the fail2ban status and enabled “jails” as follows:

docker exec -it swag-attachment fail2ban-client status

Output should be similar to the following, showing the fail2ban is setup with 5 jails by default:

You can read more about fail2ban here: fail2ban/fail2ban: Daemon to ban hosts that cause multiple authentication errors (github.com)

Congratulations, you successfully deployed open-appsec WAF integrated with Docker SWAG!

Recommended next steps:

• If you connected to central WebUI AND configured your deployment profile in the WebUI to "This management" mode for centrally managing open-appsec configuration: Create one or more assets in the WebUI which represent web applications and/or Web APIs which you want to be protected by open-appsec WAF and allows you to adjust the open-appsec configuration specifically for each of them.

  Make sure to link your assets to the specific WebUI Profile which you created earlier (General -> Profiles) and adjust the Threat Prevention mode to Detect-Learn or Prevent (Threat Prevention -> Mode), the steps are described here: Protect Additional Assets

Don't forget to Enforce policy in the WebUI after you did any changes for those changes to become effective!

• If you decided to locally, declaratively manage open-appsec (with or without connection to central WebUI in "Declarative configuration" mode): Follow the steps described here to configure your open-appsec deployment using the local_policy.yaml file: Configuration Using Local Policy File (Docker) In case you connected your locally managed deployment also to the central WebUI in "Declarative Configuration" mode, you can check security logs and view agent status and configuration also in the central WebUI at https://my.openappsec.io .

Don't forget to apply the policy using open-appsec-ctl -ap in the open-appsec-agent container or by setting APPSEC_AUTO_POLICY_LOAD in the .env file to true for automatic application of any configuration changes done in the local_policy.yaml file for the changes to become effective!

• Verify that open-appsec is preventing attacks: a) Make sure the SWAG NGINX reverse proxy is properly set-up to point to some web application backend (see more details here). The NGINX configuration for the SWAG container is located here (on docker host): ./swag-config/nginx/site-confs/default.conf (and in SWAG container it’s here: /config/nginx/site-confs/default.conf ) Note that for the following simulated attack you can also just run it against the default webpage SWAG returns after deployment and skip this step. b) Run following curl request to simulate an attack (replace with your SWAG container DNS name and Port)

curl -v -k https://<SWAG-CONTAINER-PUBLIC-DNS-NAME>:<PORT>/?shell_cmd=cat/etc/passwd

Note: “-k” was added above so this would work even when still using a Let’s Encrypt “staging” certificate.

You can alternatively use your web browser, just open the following URL:

https://<SWAG-CONTAINER-PUBLIC-DNS-NAME>:<PORT>/?shell_cmd=cat/etc/passwd

If your policy is set to prevent in the local configuration file (default in the file used when following this guide), the request will be blocked with error 403 shown, otherwise it will be detected and you will only see it in the security logs.

c) You can view the resulting open-appsec security logs by running:

docker exec appsec-agent open-appsec-ctl -vl

• If required you can further configure open-appsec WAF’s settings using open-appsec’s local declarative policy, see docs for the local policy file configuration options here:

Configuration Using Local Policy File (Docker)