Track Learning and Move From Learn/Detect to Prevent
PreviousConfigure Contextual Machine Learning for Best AccuracyNextTrack Learning and Local Tuning in Standalone Deployments
Last updated
Last updated
When a new asset is added it is recommended that open-appsec runs in Learn/Detect mode to allow it to create an initial baseline. When the Contextual Machine Learning is properly configured and the application receives a substantial amount of traffic, the learning period takes about 2-3 days.
Depending on amount and variance of traffic the machine learning engine will reach a stage where it has observed a sufficient amount of web requests to understand how the application is used. The faster this stage is reached, the faster detection is accurate and it is recommended to move to Prevent mode.
To speed up the learning period the Contextual Machine Learning engine proposes tuning suggestions. The administrator can review the tuning suggestions and help the engine reach even better accuracy, a Machine Learning process also known as supervised learning.
When a certain maturity level is reached, open-appsec will advice to move into Prevent mode. In this section you will understand how to track the learning level and how to optimize and tune the model for faster learning.
When HTTP requests are inspected the open-appsec Contextual Machine Learning model will reach different learning levels. Each level represents the maturity of the learning model and helps to understand what it needs to reach the next level. It will also indicate when it is time to move from Learn/Detect to Prevent mode. The model progresses through the following learning levels:
When the learning level becomes Graduate, it is recommended to change the asset Mode to Prevent for either High confidence or above or Critical confidence events. Graduate level ensures very good level of accuracy (e.g. low amount of false positives). To reach Master or PhD level is is necessary to configure Trusted Sources. The Phd level is the highest level, which means that more learning is less likely going to improve the model further.
Go to Assets and select the Asset you want to track.
Select the Learn tab. This tab shows the learning statistics of the last 7 days, the Elapsed Time, the Learning Level and the Recommendation at this level.
The example below shows that the machine learning is in Kindergarten level and needs 999 additional HTTP requests and 6 additional learning hours to reach Primary School level.
When an agent enforcing open-appsec first sees a connection to a web application/web API defined URL, a banner will also show at the top of the management web application to denote that "Learning has started":
Keep Learning
No action required. The machine learning model requires additional HTTP requests (and additional time).
Review Tuning Suggestions
Prevent Critical Severity Events
The system is ready to prevent critical severity events. Navigate to the Threat Prevention tab and change the Web Attacks practice Mode to Prevent for Critical Severity events.
Prevent High Severity And Above Events
The system is ready to prevent high severity (and above) events. Navigate to the Threat Prevention tab and change the Web Attacks practice Mode to Prevent for High and above Severity events.
In the example below the Recommendation is Keep Learning and additional HTTP requests are required to reach the next learning level.
Another example shows sufficient learning is achieved and the Recommendation is to Prevent High Severity And Above events. Since the Web Attack Practice is already in Prevent mode the tool tip shows a 'Well done!' message.
The Contextual Machine Learning model may ask to review certain events, also called Tuning Suggestions. Providing feedback to these suggestions is not mandatory as the engine is capable of learning by itself. However doing this, allows the machine learning engine to reach a higher maturity level and therefore a better accuracy faster based on human guidance.
Go to Assets and select the Asset you want to review.
Select the Learn tab. This tab shows Tuning Suggestions and Tuning Decisions.
Review the proposed Tuning Suggestions.
Click on the Malicious or Benign button next to the line of the Tuning Suggestion. Your Tuning Suggestion now moves to the Tuning Decisions list.
Go to Step 2: Learn the recommended action of the previous section to learn what to do next to improve the learning process.
Follow these actions when it is time to change the Practice mode from Learn/Detect to Prevent.
Go to Assets and select the Asset you wish to Protect.
Select the Learn tab and examine the recommendation.
Select the Events tab and examine Critical and High Events for the asset from the last 1-2 days.
If needed, right-click on event and add Exceptions for traffic that the machine learning engine may have misclassified (exceptions can be added based on the uri, source identifier, parameter, and more). See more about this here.
Select the Threat Prevention tab and change the mode to Prevent
You can further tune the sensitivity to block either High or above or just Critical events.
Set the level based on the recommendation in the Learn tab as well as your impression when looking at the events during the last day of the learning period.
You can also start with Critical for few days, examine the events and then move to High or above.
Enforce policy.
Hover over the Learning Level tooltip to learn the current learning level and the next level. It will also indicate what is required to reach the next level in the 'Watch next?' section. Positive contributing factors to the learning process are: number of trusted sources defined by the admin, time elapsed, amount of traffic inspected, amount of supervised learning suggestions and some other model parameters.
Hover over the Recommendation tooltip to learn what the current recommended action is for the asset. Recommendations include:
The learning mechanism generated tuning suggestions. Go to the section in this documentation to learn how to review them and decide whether the events are malicious or benign.
