Boon Amber Model Filter¶

The foglamp-filter-amber filter allows data to be passed to an instance of the Amber from Boon Logic for creation and execution of a Boon model. Amber detects anomalies in complex assets and environments by autonomously training for the normal instead of the abnormal.

Operational Principle¶

Each FogLAMP asset that is received by the filter will create a sensor within Amber, the datapoints of the asset become the features in the Amber sensor. Therefore a model will be created on a per asset basis, if multiple items need to be correlated within a model they should be placed into the same asset before being passed to this filter. Any items that are to be excluded from the model should be removed from the asset using an upstream filter such as the Asset filter. Alternatively, a filter such as the Single Asset Model filter can be used to create readings select values from one or more readings to create the required asset model.

The model will add two datapoints to the readings it receives and pass those readings onward. The two items are names with a prefix of amber: and are the state of the model and the anomaly_index created by running the data through the model.

If the following reading is received:

{
  "reading": {
    "rpm": 486,
    "x": 0,
    "depth": 40,
    "state": "Spining Down",
  },
  "timestamp": "2024-10-15 09:57:30.362620"
}

The reading that will be sent onward by the filter will be:

{
  "reading": {
    "rpm": 486,
    "x": 0,
    "depth": 40,
    "state": "Spining Down",
    "amber:state": "Monitoring",
    "amber:progress": 14,
    "amber:clusterCount": 0,
    "amber:ID": 0,
    "amber:SI": 0,
    "amber:AD": 0,
    "amber:AH": 0,
    "amber:AM": 0,
    "amber:AW": 0,
    "amber:NI": 0,
    "amber:NS": 0,
    "amber:NW": 0,
    "amber:OM": 0,
    "amber:CS": 100,
    "amber:RCA_rpm": 0,
    "amber:RCA_x": 0,
    "amber:RCA_depth": 0
  },
  "timestamp": "2024-10-15 09:57:30.362620"
}

Datapoint	Description
amber:state	The state represents the state of the model. Initially, when a new model is created, it will be in the state Buffering. In this state it will ingest the values and use them to build the clusters with Amber that will comprise the model. When sufficient data has been read the model will transition to the state Auto Tuning and once that stage is complete it will transition to Monitoring. In the monitoring state it will make inferences and return anomaly index values based on the model it has built up and the current values.
amber:progress	Progress as a percentage value (applicable for “Buffering” and “Autotuning” states)
amber:clusterCount	Number of clusters created so far
amber:ID	list of cluster IDs. The values in this list correspond one-to-one with input samples, indicating the cluster to which each input pattern was assigned.
amber:SI	Smoothed anomaly index. The values in this list correspond one-for-one with input samples and range between 0 and 1000. Values closer to 0 represent input patterns which are ordinary given the data seen so far on this sensor. Values closer to 1000 represent novel patterns which are anomalous with respect to data seen before.
amber:AD	Binary anomaly detection value. This produced by thresholding the smoothed anomaly index (SI). The threshold is determined automatically from the SI values. A value of 0 indicates that the SI has not exceeded the anomaly detection threshold. A value of 1 indicates it has, signaling an anomaly at the corresponding input sample.
amber:AH	Anomaly History value. This is a moving-window sum of the AD value, giving the number of anomaly detections (1’s) present in the AD signal over a “recent history” window whose length is the buffer size.
amber:AM	Amber Metric value. This is a floating point values between 0.0 and 1.0 indicating the extent to which each corresponding AH value shows an unusually high number of anomalies in recent history. The value is derived statistically from a Poisson model, with values close to 0.0 signaling a lower, and values close to 1.0 signaling a higher, frequency of anomalies than usual.
amber:AW	Amber Warning Level value. This index is produced by thresholding the Amber Metric (AM) and takes on the values 0, 1 or 2 representing a discrete “warning level” for an asset based on the frequency of anomalies within recent history. 0 = normal, 1 = asset changing, 2 = asset critical. The default thresholds for the two warning levels are the standard statistical values of 0.95 (outlier, asset changing) and 0.997 (extreme outlier, asset critical).
amber:NI	Novelty Index value. These are values that show how different a new cluster actually is from the model. If the cluster is already in the model, it returns a 0. New clusters return an RI type value ranging from 0 to 1000 based on the L2 distance it is from the model’s clusters.
amber:NS	Smoothed Novelty Index value. This is just a weighted average of the new NI and the previous NI. These values range from 0 to 1000.
amber:NW	Novelty Warning Level value. This is just a scaled version of the NS values that now range from 0 to 2 to give a warning level for the asset.
amber:OM	Operational mode value this is a sliding average of the cluster IDs to give a basic representation of the different states
amber:CS	Compliance Score. A percentage of compliance of the values with normal operation.
amber:RCA_datapoint	Root cause analysis contribution score. Indicates the contribution of the specific data point to the anomaly.

A filter may also be placed upstream of the Amber filter to reduce the data passed onwards, either removing the inouts to the model or any unwanted metrics from the result of the model. Again, the Asset and Single Asset Model filters are good candidates for this.

Configuration¶

Amber filters are created in much the same way that any other filter is created, in FogLAMP. A configuration screen will be displayed during the creation process that allows a small number of parameters to be set to tailor the model.

Naming¶

amber_1

Model Name The Model Name is simply a string to identify the model with and to prevent the possibility of multiple filters sharing the same sensor. Since the filter Will use the name of the asset to create a sensor, and each sensor name must be unique within an Amber instance, the Model Name is used as a prefix when creating an Amber sensor. If we have an asset called pump1 and we name our model Cavitation, our Amber sensor will be called Cavitation:pump1.

If we change the model name for a running filter we will create a new sensor and the model will go back to its initial learning state. The previous sensor will not be deleted when we change the model name and therefore if we change the model name back to the previous nae we will again use the previous sensor and the learning process that was done with that previous sensor will have been retained.

Model Tuning¶

amber_2

Max Learning Samples The Max Learning Samples sets the parameter within Amber that tells it to move to Monitoring state after it has observed this number of samples sent to the sensor.

Auto Tuning Samples

Learning Rate Numerator A model transitions from learning to monitoring mode if fewer than this number of new clusters are opened in the last Learning Rate Denominator samples.

Learning Rate Denominator The number of data samples used in conjunction with Learning Rate Numerator when deciding if the model can transition.

Streaming Window Configuration Set the size of the streaming window either by applying the default value for single and multiple feature models or by allowing it to be manually set.

Streaming Window Size Set the streaming window to a manual value. The streaming window size is used to determine how many consecutive values are considered when evaluating a model. Effectively controlling the influence of history on the model. If multiple data points are used in a model then this would normally be set to 1. If a single data point is used then the history if the value is important, so the value should be greater than 1.

License¶

amber_3

Username Amber API username. This is used to authenticate with the Amber API and is stored in ~/.Amber.license.

Password Amber API password. This is used to authenticate with the Amber API and is stored in ~/.Amber.license.

Server Amber API server URL. This specifies the URL of the Amber API server and is stored in ~/.Amber.license.

See Also¶

foglamp-filter-ADM_LD_prediction - Filter to detect whether a large discharge is required for a centrifuge

foglamp-filter-asset - A FogLAMP processing filter that is used to block or allow certain assets to pass onwards in the data stream

foglamp-filter-asset-validation - A plugin for performing basic sanity checking on the data flowing in the pipeline.

foglamp-filter-batch-label - A filter to attach batch labels to the data. Batch numbers are updated based on conditions seen in the data stream.

foglamp-filter-breakover - Filter to forecast the a pending breakover event in a centrifuge.

foglamp-filter-change - A FogLAMP processing filter plugin that only forwards data that changes by more than a configurable amount

foglamp-filter-conditional-labeling - Attach labels to the reading data based on a set of expressions matched against the data stream.

foglamp-filter-delta - A FogLAMP processing filter plugin that removes duplicates from the stream of data and only forwards new values that differ from previous values by more than a given tolerance

foglamp-filter-edgeml - Filter which takes image data, calls out to ML process, and forwards the inference from ML as asset contents.

foglamp-filter-enumeration - A filter to map between symbolic names and numeric values in a datapoint.

foglamp-filter-inventory - A plugin that can inventory the data that flows through a FogLAMP pipeline.

foglamp-filter-metadata - A FogLAMP processing filter plugin that adds metadata to the readings in the data stream

foglamp-filter-normalise - Normalise the timestamps of all readings that pass through the filter. This allows data collected at different rate or with skewed timestamps to be directly compared.

foglamp-filter-rate - A FogLAMP processing filter plugin that sends reduced rate data until an expression triggers sending full rate data

foglamp-filter-sigmacleanse - A data cleansing plugin that removes data that differs from the mean value by more than x sigma

foglamp-filter-vibration_features - A filter plugin that takes a stream of vibration data and generates a set of features that characterise that data