Links

Using the Advanced Machine Learning Model

About open-appsec's Machine Learning Models

open-appsec uses two machine learning models:
  1. 1.
    A "Supervised Model" that was trained offline based on millions of requests, both malicious and benign.
  2. 2.
    An "Unsupervised Model" that is being built in real time in the protected environment. This model uses traffic patterns specific to the environment.
For the supervised model (see above) there are two alternative options you can chose from:
  • A "Basic Model" is provided as part of the GitHub repository and the default installations. This is recommended for use in Monitor-Only and Test environments.
  • An "Advanced Model" can be downloaded from open-appsec portal. This is more accurate and recommended for Production use.

How to Download the Advanced Machine Learning Model

To download and extract the open-appsec advanced machine learning model follow these steps:
  • Login to the open-appsec portal at my.openappsec.io .
  • Download the advanced machine learning model by going to: User Menu -> Download Advanced ML Model.
This model updates from time to time and you will get an email when these updates happen.

How to Deploy and Use the Advanced Machine Learning Model

To deploy and use open-appsec's advanced machine learning model follow the instructions below to install this Advanced Model in your specific environment:

Deployment and usage of the advanced model inside the docker

  1. 1.
    create a folder called open-appsec-advanced-model
mkdir open-appsec-advance-model
  1. 2.
    Copy the tgz file into the folder.
  2. 3.
    Map this .tgz file into the appsec container to "/advanced-model/open-appsec-advanced-model.tgz" file inside the container.
docker run
docker compose
If you are using docker run command to run the agent container add a volume mount to the command as follows: -v=./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw
See also the following example docker run command for the agent to which the additional volume mount was added accordingly.
docker run --name=open-appsec-agent \
--ipc=host \
-v=<path-to-persistent-location-for-agent-config>:/etc/cp/conf \
-v=<path-to-persistent-location-for-agent-data-files>:/etc/cp/data \
-v=<path-to-persistent-location-for-agent-debugs-and-logs>:/var/log/nano_agent \
-v=<path-to-local-configuration-file>:/ext/appsec \
-v=./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw \
-e user_email=<add-your-email-here> \
-e https_proxy=<user:password@proxy-address:port> \
-it -d ghcr.io/openappsec/agent:latest /cp-nano-agent --standalone
If you are using docker compose to deploy open-appsec add a volume mount under the agent container as follows: - ./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw See also the following example docker-compose file (used for deployment with NGINX Proxy Manager) to which the additional volume mount for the appsec-agent container was added accordingly:
version: '3.3'
# docker compose for npm open-appsec integration
services:
appsec-npm:
container_name: npm-attachment
image: 'ghcr.io/openappsec/nginx-proxy-manager-attachment:latest'
ipc: host
restart: unless-stopped
ports:
- '80:80' # Public HTTP Port
- '443:443' # Public HTTPS Port
- '81:81' # Admin Web Port
volumes:
- ./data:/data
- ./letsencrypt:/etc/letsencrypt
- ./appsec-logs:/ext/appsec-logs
- ./appsec-localconfig:/ext/appsec
appsec-agent:
container_name: appsec-agent
image: 'ghcr.io/openappsec/agent:latest'
network_mode: service:appsec-npm
ipc: host
restart: unless-stopped
environment:
# adjust with your own email below
- nginxproxymanager=true
- autoPolicyLoad=true
volumes:
- ./appsec-config:/etc/cp/conf
- ./appsec-data:/etc/cp/data
- ./appsec-logs:/var/log/nano_agent
- ./appsec-localconfig:/ext/appsec
- ./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw
command: /cp-nano-agent --standalone
  1. 2.
    Run the agent container (if you already have a running agent container make sure to redeploy it)

Deployment and usage of the advanced model in Kubernetes

  1. 1.
    Create a config map from the tgz file in the relevant namespace: kubectl create configmap advanced-model-config --from-file open-appsec-advanced-model.tgz -n <namespace>
  2. 2.
    Restart all open-appsec agent pods in the namespace of your open-appsec deployment, you can restart the pods by following the steps below:
    1. 1.
      get open-appsec deployment name
      kubectl get deployment -n <open-appsec deployment namespace>
    2. 2.
      Restart pods
      kubectl rollout restart deployment <open-appsec deployment name> -n <open-appsec deployment namespace>

Deployment and usage of the advanced model with a Linux-embedded agent

  1. 1.
    Create a folder under the following path in the root directory: /advanced-model mkdir -p /advanced-model
  2. 2.
    Copy the .tgz file into the folder you created cp ./open-appsec-advanced-model.tgz /advanced-model/open-appsec-advanced-model.tgz.
  3. 3.
    Deploy the embedded agent
  4. 4.
    if you already have an open-appsec agent.
    1. 1.
      Complete steps 1-3
    2. 2.
      Run open-appsec-ctl --stop-agent
    3. 3.
      Extract the model to the relevant folder by running tar -xzf /advanced-model/open-appsec-advanced-model.tgz -C /etc/cp/conf/waap
    4. 4.
      Run open-appsec-ctl --start-agent

Adding the advanced model when building new agent code

  1. 1.
    Clone the open-appsec agent GitHub repository (https://github.com/openappsec/openappsec).
  2. 2.
    Extract the open-appsec-advanced-model.tgz file. On Linux you can do this by using the following command: tar -xvf open-appsec-advanced-model.tgz
  3. 3.
    Copy the extracted files to the components/security_apps/waap/resources folder.
  4. 4.
    Build the agent package or Docker by following the instructions in the GitHub repository README (https://github.com/openappsec/openappsec/blob/main/README.md).

Validate the use of the Advanced Model

The machine learning (AI) model version info is shown in the open-appsec-ctl tool from version 1.1.0 and onwards.
The machine learning AI model type and version being used can be validated using the command:
open-appsec-ctl --status
Output of open-appsec-ctl --status
The model being used by each agent can also be validated using the Agents tab in the web UI:
Agents tab in Web UI