As an alternative to injecting decoded data into Agents to form Agent Events, stack probes can also:

  • Publish the data as statistics.

  • Publish the data to the Core Data Feed as a message (CDF-M).

  • Make the data available to the P3 process.

Example value-added metrics

Examples of self-contained calculations which can be performed within a stack probe:

  • App-to-wire

  • Gap detection

Examples of non self-contained calculations which are more suitable to perform in VMX-Analysis:

  • North/south latency (e.g. gateway latency) where it is a different protocol either side of the gateway

Some types of north/south latency calculation can be performed at extremely high volume within VMX-Capture using the pre correlation within the P3 process. However, even though the P3 process can process data from multiple stack probes, it still has limited capability to track state or preserve information from message to message. Where possible, this type of analysis is better performed by passing the individual Agent Events to VMX-Analysis.

Statistic output: Pre-aggregated stats

In Beeks Analytics, Aggregators define how data is “sliced and diced“ to produce statistics - or stats - used in the analytics views in VMX-Explorer. Most aggregated statistics are calculated in VMX-Analysis. However, some stats are aggregated in VMX-Capture and are then passed to VMX-Analysis as pre-aggregated stats.

Why do we aggregate some stats in VMX-Capture?

It makes sense for some stats to be calculated as close to source as possible. This is because:

a) Some aggregation requires every single packet, which is a lot of data to pass to VMX-Analysis.
For example, TCP stats.

b) Passing a lot of data to VMX-Analysis takes time, which can affect precision timings and therefore requires an extra burden to calculate and check the timings to ensure they are accurate.
For example, microburst detection.

Which stats are pre-aggregated?

Usually, only data measured at a single visibility point is pre-aggregated. However, VMX-Capture does support multi-point measurement for pre-aggregation if required.

P3 Process

The P3 process is a separate process from the stack probe. It is called P3 because it performs three functions:

  • Pre Filtering

  • Pre Correlation

  • Pre Aggregation

P3 Pre Filtering

Pre Filtering can be used to dynamically filter the output of one probe using input from another probe. An example might be where a beacon published market data IDs which need to be forwarded to VMX-Analysis for further analysis; market data messages with IDs that didn’t match an ID from the beacon would be ignored. This can be used effectively in situations where complex or dynamic filtering logic is required.

P3 Pre Correlation

The P3 pre correlation mechanism allows simple correlation across a network hop e.g. correlation between two sides of a firewall that is applying Network Address Translation.

P3 Pre Aggregation

P3 pre aggregation differs from the pre aggregation within a stack. P3 pre aggregation can be used in situations where there is an exceptionally high volume of messages. The message load can be split between multiple stack probes, and these probe outputs can then be split between multiple P3 processes. P3 can then rollup buckets of distribution to VMX-Analysis, to get effective HDR distributions for high message volume calculations.