Worked Example: TCP Round-trip time stats in Beeks Analytics for Markets
This is a relatively simple example of using P3, and explains how P3 is used to calculate one specific statistic that is provided as part of the Beeks Analytics for Markets deployments.
Aim of the TCP RTT P3 stack
The Beeks Analytics for Markets configuration uses different stacks for each direction of traffic. This means we need a mechanism for calculating round-trips, since, by their nature, round-trips require you to measure the timing difference between a message in one direction and a message in the opposite direction.
For orders, these round-trips can be performed in VMX-Analysis. We have seen already, in the Worked Example: Order Entry Stack Probes in Beeks Analytics for Markets , how agent events in each different direction are passed by different stacks to VMX-Analysis, and how VMX-Analysis correlates the two messages together.
For TCP round-trips, it would not be efficient to pass every TCP SYN and every TCP ACK packet as a message to VMX-Analysis, and we do not need to store each round-trip measurement - we only need to see the aggregated calculations. As such, it is the type of latency measurement which is better performed in the P3 process.
The TCP aggregator in VMX-Analysis can be passed the aggregated statistics from these calculations.
TCP round trip statistics can also be sent out via the Core Data Feed. See the CDF-T sections of the Core Data Feed Guide for further information.
P3 stack configuration overview for TCP RTT
The probe configuration is defined in the <confdir>/agent/p3/<VP_NAME>_TCP_rtt.p3.json file. Here are some excerpts from that configuration.
In the following example, we will assume that the probe is configured for the ‘VP’ pmux/visibility point, which is the default for a Beeks Analytics for Markets deployment.
P3 Transports and Input Probe Configuration
In the <confdir>/agent/p3/<VP_NAME>_TCP_rtt.p3.json file:
"transports": [ "udp:233.54.12.111:8210:127.0.0.1:256000", "udp:233.54.12.111:8211:127.0.0.1:256000", "udp:233.54.12.111:8212:127.0.0.1:256000", "udp:233.54.12.111:8213:127.0.0.1:256000" ],These are the UDP ports that the P3 configuration is listening on. In the BAM configuration, these are receiving inputs from four separate stack probes.
Note how the syn probes and the ack probes switch around the destination host/port and source host/port for their correlation - this allows the messages in each direction to be correlated with each other.
TCP Acknowledgment Probes
The configuration of the TCP acknowledgment probes is relatively simple. They use the tcp.ack field as their correlation id (cid).
In the <confdir>/agent/pmux/<VP_NAME>/ directory:
Probe Config Filename | platency3 msg_collector transport value | platency3 node_id | platency3_cid | platency3_tag |
|---|---|---|---|---|
VP_TCP_rtt_Tx_ack.stack.json | udp:233.54.12.111:8211:127.0.0.1 | 2 |
|
|
VP_TCP_rtt_Rx_ack.stack.json | udp:233.54.12.111:8213:127.0.0.1 | 4 |
|
|
TCP Syn Probes
The TCP syn probes are more complicated. As well as providing the correlation IDs to the P3 probe, they also provide the other fields that will be used in the aggregation. The following table just lists the correlation ID that these probes emit (the specific TCP next sequence number), and the first tag (which contains the correlation ID value).
In the <confdir>/agent/pmux/<VP_NAME>/ directory:
Probe Config Filename | platency3 msg_collector transport value | platency3 node_id | platency3_cid | platency3_tag 1 |
|---|---|---|---|---|
VP_TCP_rtt_Tx_syn.stack.json | udp:233.54.12.111:8210:127.0.0.1 | 1 |
|
|
VP_TCP_rtt_Rx_syn.stack.json | udp:233.54.12.111:8212:127.0.0.1 | 3 |
|
|
TCP Probe Transport Parameters
All of the P3 TCP probes have the following, non-default, parameters set for the IP decoder:
"dec_ip": { "parameters": { "tcp_on_receive": false, "tcp_on_ack": false, "tcp_on_transmit": true } }See IP Decoder (dec_ip) parameters for the background on why these parameters are set in this way for this use case.
P3 Model Configuration
The model section of the configuration defines the nodes that will be required for the correlation (ingess_syn, ingress_ack, egress_syn and egress_ack), and the extra fields that will be captured in each direction. The flows array defines the from and to mappings for these correlations:
"flows": [ { "from": "egress_syn", "to": "ingress_ack" }, { "from": "ingress_syn", "to": "egress_ack" } ]This definition (and the hops section of the configuration, see below) is where the ingress and the egress directions are tied together.
P3 Aggregator Configuration
The aggregators section defines the pre-aggregators that the P3 process will publish statistics into. The aggregator definitions are referenced in different files, which are contained in the <confdir>/agent/global/preagg directory:
"aggregators": [ { "name": "egress_ExtGrDs", "nodePathStatsJsonFilename": "$VMX_HOME/../../server/config/agent/global/preagg/TCP_rtt_Tx_ExtGrpDS.p3.agg.json", "publishIntervalUs": 60000000, "vmxGaiePublishers": [ { "agentId": "TCP_ExtGrp_DS_statsAgent" } ] }, { "name": "egress_Ds", "nodePathStatsJsonFilename": "$VMX_HOME/../../server/config/agent/global/preagg/TCP_rtt_Tx_DS.p3.agg.json", "publishIntervalUs": 60000000, "vmxGaiePublishers": [ { "agentId": "TCP_DS_statsAgent" } ] }, { "name": "ingress_ExtGrDs", "nodePathStatsJsonFilename": "$VMX_HOME/../../server/config/agent/global/preagg/TCP_rtt_Rx_ExtGrpDS.p3.agg.json", "publishIntervalUs": 60000000, "vmxGaiePublishers": [ { "agentId": "TCP_ExtGrp_DS_statsAgent" } ] }, { "name": "ingress_Ds", "nodePathStatsJsonFilename": "$VMX_HOME/../../server/config/agent/global/preagg/TCP_rtt_Rx_DS.p3.agg.json", "publishIntervalUs": 60000000, "vmxGaiePublishers": [ { "agentId": "TCP_DS_statsAgent" } ] } ],The vmxGaiePublishers part of the aggregator definition handles the publishing of the aggregated statistics to VMX-Analysis.
These aggregator definitions use the information made available by the TCP_SYN stack probes, and passed through by the P3 process, to group the statistics by ExtGrp and then by DS, or just by DS. These are the two fundamental aggregations available within the Beeks Analytics for Markets templated configuration.
Note there are different aggregator definitions for ingress and egress at this point of the configuration, but there are just two different agentIds: TCP_ExtGrp_DS_statsAgent and TCP_DS_statsAgent. This allows the ingress and egress events for a particular breakdown to be published in the same aggregator view within VMX-Analysis.
P3 Hop configuration
This section should be read in conjunction with the models definition. The configuration shows the egress and ingress correlations with each other. The tags convey the extra information that is fed through to the aggregators listed:
"hops": [ { "from": "egress_syn", "to": "ingress_ack", "tags": [ "session", "beeks.intentity", "beeks.extgroup", "intport", "extport", "extip", "intip", "beeks.vp" ], "aggregators": [ "egress_ExtGrDs", "egress_Ds" ] }, { "from": "ingress_syn", "to": "egress_ack", "tags": [ "session", "beeks.intentity", "beeks.extgroup", "intport", "extport", "extip", "intip", "beeks.vp" ], "aggregators": [ "ingress_ExtGrDs", "ingress_Ds" ] } ]