Using the Advanced Machine Learning Model

About open-appsec's Machine Learning Models

open-appsec uses two machine learning models:

1. A "Supervised Model" that was trained offline based on millions of requests, both malicious and benign.

2. An "Unsupervised Model" that is being built in real time in the protected environment. This model uses traffic patterns specific to the environment.

For the supervised model (see above) there are two alternative options you can chose from:

• A "Basic Model" is provided as part of the GitHub repository and the default installations. This is recommended for use in Monitor-Only and Test environments.

• An "Advanced Model" can be downloaded from open-appsec portal. This is more accurate and recommended for Production use.

How to Download the Advanced Machine Learning Model

To download and extract the open-appsec advanced machine learning model follow these steps:

• Login to the open-appsec portal at my.openappsec.io .

• Download the advanced machine learning model by going to: User Menu -> Download Advanced ML Model.

This model updates from time to time and you will get an email when these updates happen. When the model is updated please redeploy the Advanced Machine Learning Model by downloading the new file, and following the instructions bellow.

How to Deploy / Update the Advanced Machine Learning Model

To deploy and use open-appsec's advanced machine learning model follow the instructions below to install this Advanced Model in your specific environment:

Deployment and usage of the advanced model inside the docker

1. create a folder called open-appsec-advanced-model

mkdir open-appsec-advance-model

1. Copy the tgz file into the folder.

2. Map this .tgz file into the appsec container to "/advanced-model/open-appsec-advanced-model.tgz" file inside the container.

docker run

If you are using docker run command to run the agent container add a volume mount to the command as follows: -v=./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw

See also the following example docker run command for the agent to which the additional volume mount was added accordingly.

docker run --name=open-appsec-agent \
--ipc=host \
-v=<path-to-persistent-location-for-agent-config>:/etc/cp/conf \
-v=<path-to-persistent-location-for-agent-data-files>:/etc/cp/data \
-v=<path-to-persistent-location-for-agent-debugs-and-logs>:/var/log/nano_agent \
-v=<path-to-local-configuration-file>:/ext/appsec \
-v=./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw \
-e user_email=<add-your-email-here> \
-e https_proxy=<user:password@proxy-address:port> \
-it -d ghcr.io/openappsec/agent:latest /cp-nano-agent --standalone

docker compose

If you are using docker compose to deploy open-appsec add a volume mount under the agent container as follows: - ./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw See also the following example docker-compose file (used for deployment with NGINX Proxy Manager) to which the additional volume mount for the appsec-agent container was added accordingly:

version: '3.3'
# docker compose for npm open-appsec integration

services:
  appsec-npm:
    container_name: npm-attachment
    image: 'ghcr.io/openappsec/nginx-proxy-manager-attachment:latest'
    ipc: host
    restart: unless-stopped
    ports:
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      - '81:81' # Admin Web Port
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
      - ./appsec-logs:/ext/appsec-logs
      - ./appsec-localconfig:/ext/appsec

  appsec-agent:
    container_name: appsec-agent
    image: 'ghcr.io/openappsec/agent:latest'
    network_mode: service:appsec-npm
    ipc: host
    restart: unless-stopped
    environment:
      # adjust with your own email below
      - user_email=[e-mail]
      - nginxproxymanager=true
      - autoPolicyLoad=true
    volumes:
      - ./appsec-config:/etc/cp/conf
      - ./appsec-data:/etc/cp/data
      - ./appsec-logs:/var/log/nano_agent
      - ./appsec-localconfig:/ext/appsec
      - ./open-appsec-advance-model/open-appsec-advanced-model.tgz:/advanced-model/open-appsec-advanced-model.tgz:rw
    command: /cp-nano-agent --standalone

1. Run the agent container (if you already have a running agent container make sure to redeploy it)

Deployment and usage of the advanced model in Kubernetes

1. Create a config map from the tgz file in the relevant namespace: kubectl create configmap advanced-model-config --from-file open-appsec-advanced-model.tgz -n <namespace>

2. Restart all open-appsec agent pods in the namespace of your open-appsec deployment, you can restart the pods by following the steps below:

   1. get open-appsec deployment name

      kubectl get deployment -n <open-appsec deployment namespace>

   2. Restart pods

      kubectl rollout restart deployment <open-appsec deployment name> -n <open-appsec deployment namespace>

Deployment and usage of the advanced model with a Linux-embedded agent

1. Create a folder under the following path in the root directory: /advanced-model mkdir -p /advanced-model

2. Copy the .tgz file into the folder you created cp ./open-appsec-advanced-model.tgz /advanced-model/open-appsec-advanced-model.tgz.

3. Deploy the embedded agent

4. if you already have an open-appsec agent.

   1. Complete steps 1-3

   2. Run open-appsec-ctl --stop-agent

   3. Extract the model to the relevant folder by running tar -xzf /advanced-model/open-appsec-advanced-model.tgz -C /etc/cp/conf/waap

   4. Run open-appsec-ctl --start-agent

Adding the advanced model when building new agent code

1. Clone the open-appsec agent GitHub repository (https://github.com/openappsec/openappsec).

2. Extract the open-appsec-advanced-model.tgz file. On Linux you can do this by using the following command: tar -xvf open-appsec-advanced-model.tgz

3. Copy the extracted files to the components/security_apps/waap/resources folder.

4. Build the agent package or Docker by following the instructions in the GitHub repository README (https://github.com/openappsec/openappsec/blob/main/README.md).

Validate the use of the Advanced Model

The machine learning (AI) model version info is shown in the open-appsec-ctl tool from version 1.1.0 and onwards.

The machine learning AI model type and version being used can be validated using the command:

open-appsec-ctl --status 

Output of open-appsec-ctl --status

The model being used by each agent can also be validated using the Agents tab in the web UI:

Agents tab in Web UI