Beeks Analytics is a transformative analytics solution that performs packet-level monitoring and recording of network traffic, delivering deep performance insights into your trading infrastructure.

Beeks Analytics empowers your business to track and analyse the real-time performance of every single message traversing business-critical processes. Identify outliers and bottlenecks, understand capacity issues, verify new system roll-out performance, and ultimately ensure provision of a consistent and high-quality trading environment.

Beeks Analytics solves

• Do I have performance problems with any of my trading counterparties?

• Is the problem inside my infrastructure or outside my infrastructure?

• I need an overview of the flows going across my network to assist with troubleshooting and capacity.

• I need to maintain Service Level Agreements within my trading system for my customers.

• I need to exactly reproduce a period of sub-par performance in my trading system to try some possible solutions.

In this introduction, we’ll walk through an example problem case to help introduce the key concepts, and show how these can help with specific business problems. Later in the document, we’ll go into the concepts in more detail.

Example Problem Case

Beeks Analytics delivers deep visibility of your traffic and empowers you to extract precision timings for data processing within and between the individual parts of your trading system, even where data formats change or where there is no one-to-one relationship between messages at the different points. 

Let’s take a simplified example of a Broker-Dealer that trades securities on behalf of its customers. 

When the customer places an order and gets a price back from the Broker/Dealer, the order is made up of two messages.  Firstly, a FIX protocol message travels from the Client Trading Engine to the Broker/Dealer FIX Gateway/Execution Management System, and secondly a Binary Exchange protocol message goes from the Broker/Dealer to the Exchange. Similarly, when the Exchange returns a price, it is in Binary Exchange protocol and when the Broker/Dealer sends the price back to the Client Trading Engine, it’s in FIX protocol. 

Conceptually, we represent all the messages that make up the order and price as one business object. 

Example: Entities and messages

To measure how fast these messages take to travel along the network, we create monitoring locations along their route on the network, which we call visibility points. We put probes at visibility points to capture information about these messages. Each probe has a set of agent points whose job is to process the captured data. 

This processing includes identifying all the data that’s related to the messages for a given business object (correlation and association) and then putting it into common format that can be queried in Beeks Analytics (decoding). 

As is the case with all network data, an individual message is broken down into smaller parts called packets and multiple messages are sometimes combined before sending - and each host in the diagram above might do this in a different way. Beeks Analytics applies a range of highly efficient methods that ensure all the data relating to one business object is correctly identified to deliver exact measurement of latencies. 

Beeks Analytics stores these packets in a packet capture repository, which can be searched and queried to help with diagnosis of problems (or for our mdPlay and mdStream solutions to extract market data to be replayed back for test situations). 

As a quick explainer about correlation and association: messages that arrive from different protocol points in the trading system might not have an obvious relationship with other messages for the same business object.  To identify all messages that belong to the business object at different locations on the network, Beeks Analytics uses correlation and association techniques to establish a common identifier, which it then uses to group the relevant messages together into an item (see page 49). Now it can extract the timestamp for each message that relates to the order, and calculate latencies between them. For example, the time between the client sending the order and the exchange receiving the order. 

To explain decoding a little more and why it is needed: each packet contains a hierarchy of one or more different network protocols. The diagram above is a simplified view. In fact, the FIX protocol operates on top of the TCP protocol, which itself operates on top of the IP protocol, which itself normally operates on top of the Ethernet protocol. Different information needed for analytics exists at each different level of the protocol hierarchy. For example, to know the symbol on the exchange that is being traded we would need to look at the FIX protocol, but to understand which computer on the local network it originated from, we would need to look at the addresses in the Ethernet protocol. Decoding puts all of this information into a standard format that Beeks Analytics can understand. 

After the message has been decoded, Beeks Analytics can perform further processing on the message. Typically, this is one of the following:

• Beeks Analytics can a pass a subset of the important fields from the message to persistent storage or to its analysis component. One of these fields will be a timestamp that indicates when the message was first seen by Beeks Analytics. This timestamp will be nanosecond precision. This function is performed by a Message Collector in VMX-Capture, which can pass the message to a third-party system via the Core Data Feed or to VMX-Analysis for enrichment, correlation and/or persistence.  

• If the Client Trading Engine is submitting a very large number of messages, it may not be practical to store all of these or to do computationally intensive analysis on them. In this case, the Analytics system may issue one or more statistic updates for the message but would not pass on any message field themselves. For reasons that will become clear later, these statistics are known as pre-aggregated statistics. This function is performed by a Statistics Collector in VMX-Capture. Again, these statistics can be passed to third-party systems via the Core Data Feed or to VMX-Analysis for aggregation, further calculations, and/or persistence.

Output from Beeks Analytics is available via API and in its Grafana-based tool, VMX Explorer, which supports aggregated views to enable end users to drill down into a more granular level of detail. 

This example is a simple one designed to explain our key concepts at a high level and in the context of a FIX order. In reality, Beeks Analytics is a highly configurable solution that can measure latency between a variety of complex messages. For example, we could measure the time between the client sending the order (the first item) and the client receiving a price back (a second item). See Appendix B: Worked example of complex trading flows section for a richer example of the types of business monitoring which Beeks Analytics can provide.