Introduction to Beeks Analytics

Beeks Analytics is a suite of products that records and analyses latencies at network and application level, delivering exceptional, real-time insight into performance so that our clients can deliver a trading environment that gives their customers the edge.

Beeks Analytics provides unrivalled visibility that enables you to track and analyse the real-time performance of every single price, quote, or trade traversing business critical processes. Identify outliers and bottlenecks, understand capacity issues, verify new system roll-out performance, and ultimately ensure delivery of a consistent and high-quality trading environment.

Some example use cases that are solved by Beeks Analytics:

• Do I have performance problems with any of my trading counterparties?

• Is the problem inside my infrastructure or outside my infrastructure?

• I need an overview of the flows going across my network to assist with troubleshooting and capacity.

• I need to report on performance of my trading system to my customers.

• I need to exactly reproduce a period of sub-par performance in my trading system to try some possible solutions.
  

In this introduction, we’ll walk through an example problem case to help introduce the key concepts, and show how these can help with specific business problems. Later in the document, we’ll go into the concepts in more detail.

Example Problem Case

Let’s take a simplified example of a broker-dealer that trades securities on behalf of its customers.

This broker-dealer receives orders from clients over the FIX protocol, and submits them to the exchange using a proprietary binary exchange protocol.

Both FIX and the binary exchange protocol define certain message types that they support for each type of business object. A business object is an instruction or entity such as the order or a price etc.

We can think of an order submission that passes from the Client Trading Engine to the Exchange as a FIX message sent from the Client Trading Engine to the FIX Gateway/Execution Management System, and as a separate binary exchange protocol message sent from the Broker/Dealer to the Exchange. Conceptually, we represent the order (independent of the messaging protocols used) as a single business object.

We can think of ourselves as looking at this data on the wire at one or more different locations - or visibility points - in the network. This is a useful complement to viewing the data within the application that is sending or receiving the data. Being able to view the data on the wire is essential if there is some network infrastructure between us and a destination that is outside our control, or if we have high performance or visibility demands that demand more granular monitoring.

Note that when we look at this data on the wire, the network stack of the operating system on each host may split up the message into smaller chunks to deliver it. It might also combine multiple messages together before sending them - something which can introduce delays. The chunks of data which pass across the network are called packets. Beeks Analytics stores these packets in a packet capture repository, which can be searched and queried to help with diagnosis of problems (or for our mdPlay solution to extract market data to be replayed back for test situations).

Beeks Analytics receives these packets, and decodes the different protocols in each packet. Each packet contains a hierarchy of one or more different network protocols. The diagram above is a simplified view - the FIX protocol operates on top of the TCP protocol, which itself operates on top of the IP protocol, which itself normally operates on top of the Ethernet protocol. Different information needed for analytics will exist at each different level of the protocol hierarchy. For example, to know the symbol on the exchange that is being traded we would need to look at the FIX protocol, but to understand which computer on the local network it originated from, we would need to look at the addresses in the Ethernet protocol.

After the message has been decoded, Beeks Analytics can perform further processing on the message. Typically, this is one of the following:

• Beeks Analytics can a pass a subset of the important fields - or attributes - from the message to persistent storage or to its analysis component. Each subset of attributes that is passed in this way is known as an agent event. One of these attributes will be a timestamp that indicates when the message was first seen by Beeks Analytics.

• If the Client Trading Engine is submitting a very large number of messages, it may not be practical to store all of these or to do computationally intensive analysis on them. In this case, the Analytics system may issue one or more statistic updates for the message, but would not pass on any message field themselves. For reasons that will become clear later, these statistics are known as pre-aggregated statistics.

We expect the messages related to orders to take a pre-defined route through a network or application, and we instruct Beeks Analytics to listen for these messages at visibility points along this route, and extract the information needed for analysis. This pre-defined route is called the flow, and each flow is specific to a type of business object, which in this case, is any order.

To identify all the agent events that belong to the order at different visibility points on the flow, Beeks Analytics looks for a common identifier in the attributes. This process is called correlation. Beeks Analytics uses a range of mapping strategies to correlate agent events for the same business object. This is because messages that arrive from different protocol points in the trading system may not contain a common format, identifiers, or have an obvious relationship with other messages for the same business object.

Beeks Analytics groups all correlated agent events together. The group is called an item. Now it can extract the timestamp from the attribute in each agent event in the item, and calculate latencies between agent events. For example, the time between the client sending the order and the exchange receiving the order. To configure Beeks Analytics to calculate latencies between agent events, we define intervals.

An interval definition consists simply of the two Agent IDs for the agents that Beeks Analytics will measure the time between. The first agent provides a starting timestamp, and the second provides an ending timestamp. Beeks Analytics will pair agent events from the start and end agents to generate an interval. Intervals contain information from the start and end agents, such as start host, and end host, and the computed timing interval.

Once Beeks Analytics has captured the messages relating to the order, it performs the following tasks for each message: creating an agent event containing the necessary attributes, correlating the agent event with all other agent events, and building the required interval events. This enables Beeks Analytics to produce sophisticated statistics for the order and its timings by applying a calculator to the order data and storing the outcome in an aggregator. Beeks Analytics has a range of built-in calculators that can be used to compute a variety of statistics for all supported business object types.

When Beeks Analytics visualises analytics in its web-based user interface - VMX Explorer - it can use aggregators in its views, which also allow end users to drill down into views that contain a more granular level of detail if needed.

This example is a simple one designed to explain our key concepts in the context of a FIX order. In reality, Beeks Analytics is a highly configurable solution that can build intervals between a variety of complex messages, including intervals between agent events in separate items. For example, we could measure the time between the client sending the order (the first item) and the client receiving a price back (a second item). This process is called association.

Beeks Analytics delivers deep visibility of your traffic and empowers you to extract precision timings for data processing within and between the individual parts of your trading system, even where data formats change or where there is no one-to-one relationship between messages at the different points.