Delta Filter

The foglamp-filter-delta is a filter that only forwards data that changes by more than a configurable tolerance value/percentage. It is used to remove duplicate data values from an asset stream. The definition of duplicate however allows for some noise in the reading value by requiring a tolerance value/percentage.

By defining a minimum rate it is possible to force readings to be sent at that defined rate when there is no change in the value of the reading. Rates may be defined as per second, per minute, per hour or per day.

Delta filters are added in the same way as any other filters.

• Click on the Applications add icon for your service or task.

• Select the delta plugin from the list of available plugins.

• Name your delta filter.

• Click Next and you will be presented with the following configuration page

• Configure the parameters of the delta filter

  • Tolerance Measure: Defines if the Tolerance Value represents a percentage change or an absolute change.

  • Tolerance Value: The tolerance percentage/value when comparing reading data. Only values that differ by more than this percentage/value will be considered as different from each other.

  • Reading Processing Mode: Controls when the delta filter outputs a reading and what to include in this output reading. This is an enumeration with following options:

    1. Include full reading if any Datapoint exceeds tolerance

    2. Include full reading if all Datapoints exceed tolerance

    3. Include only the Datapoints that exceed tolerance

  • Minimum Rate: The minimum rate at which readings should be sent. This is the rate at which readings will appear if there is no change in value.

  • Minimum Rate Units: The units in which minimum rate is defined (per second, minute, hour or day)

  

  • Individual Tolerances: A JSON document that can be used to define specific tolerance values for an asset. This is defined as a set of name/value pairs for those assets that should use a tolerance percentage/value other than the global tolerance value specified above. The following example would set the tolerance for the temperature asset to 15 and for the pressure asset to 5, in the same units as specified in Tolerance Measure. All other assets would use the tolerance specified in Tolerance Value.

    {
        "temperature" : 15,
        "pressure" : 5
    }
    

• Enable the filter and click Done to complete the process of adding the new filter.

Important notes:

Due to inherently limited precision of double/high-precision floating point values in programming languages like C++, the absolute tolerance value that can be reliably checked by this filter depends on the scale of the data being processed.

Following combinations have been tested successfully.

Scale of data	Absolute tolerance value
10^9	10^-03
10^6	10^-06
10^3	10^-06
10^0	10^-06
10^-03	10^-06
10^-06	10^-06

See Also

foglamp-filter-asset - A FogLAMP processing filter that is used to block or allow certain assets to pass onwards in the data stream

foglamp-filter-change - A FogLAMP processing filter plugin that only forwards data that changes by more than a configurable amount

foglamp-filter-normalise - Normalise the timestamps of all readings that pass through the filter. This allows data collected at different rate or with skewed timestamps to be directly compared.

foglamp-filter-rate - A FogLAMP processing filter plugin that sends reduced rate data until an expression triggers sending full rate data