Data streams
The Core Data Feed is made available at different points in the Beeks Analytics architecture to expose the following data:
CDF-T (Telemetry)
CDF-M (Messages)
CDF-I (Items, Intervals, and richer information)
These are summarised in the next sections.
CDF-T (Telemetry)
CDF-T provides a stream of Telemetry metrics about the data we observe at a single measuring point, such as:
Information about Packet Capture files.
Layer 1 to 4 telemetry.
Microburst.
Market data analytics - for example, gap statistics, wiretime statistics.
CDF-T context is limited to a specific source decoder stack probe. Multiple stack probes can publish CDF-T output concurrently to the same topic, but each individual message can only contain data from the context of an individual decoder stack probe.
CDF-M (Messages)
A stream of decoded Messages that we observe at single measuring points, such as:
Decoded messages, including nanosecond wire receive time.
Simple latency calculations.
Like the CDF-T, CDF-M context is limited to a specific source decoder stack probe.
Each Message can be enriched with additional attributes to allow it to be considered equivalent to an OpenTelemetry span.
CDF-I (Items, Intervals, and richer information)
CDF-I is a second-level analysis that brings together CDF-T and CDF-M data from multiple points to create traces comprised of Items, Intervals, and richer Information, such as:
Agent Events. I.e. normalised messages.
Latency Intervals between Agent Events.
Associations between one Agent Event and many others.
Alerts.
Aggregated Statistics.
CDF-I can combine information from multiple stacks, including doing calculations and enrichment that involve information that is provided by different stacks. It also provides support for rich correlations and associations.
CDF-T and CDF-M context is limited to the specific source stack i.e. each topic can have multiple stack probes publishing messages concurrently to it., however each individual message in these streams can only contain data from the context of an individual decoder stack probe.
See the VMX-Analysis section of the Analytics Concepts Guide for more information about latency intervals between Agent Events, Associations and Alerts.
See the Configuration Guide for VMX-Capture for more information on the P3 process and the pre aggregation and pre correlation that can be performed at extremely high volumes within VMX-Capture. The P3 process is the part of the VMX-Capture software which is capable of publishing a CDF-I stream.