The Message Header contains a single key called "fields", which defines the fields contained in the common message header of the protocol you want to decode.

"MessageHeader" : {
"fields": [
{ "name": "Version", "type": "UINT8" },
{ "name": "Cat", "type": "UINT8" },
{ "name": "Type", "type": "UINT8", "flags":["msg_type"] },
{ "name": "Orig", "type": "ORIG" },
{ "name": "Timestamp", "type": "TIMESTAMP" },
{ "name": "Sequence", "type": "UINT64_BE" },
{ "name": "Token", "type": "UINT64_BE" }
]
  }

Each field follows the format:

{ "name" :"field_name", "type" : "<TypeDefinition name>", "flags":[] }

  • "name" can be any alphanumeric string but must not contain spaces.

  • "type" must be the name of a predefined type from the list of TypeDefinitions.

  • "flags" are optional.

Optional “offset” override:

An "offset" can also be specified for each field as an unsigned numeric value to tell the decoder which offset from the beginning of the packet to decode the field data from.  Care needs to be taken when using this override because the offset is an internal counter maintained by the decoder to iterate through the data based on the current type definitions.  An example use of this override is to skip the parts of a packet which are of no interest.

How the decoder decodes header datafields

At the beginning of the header, the decoder starts with an internal offset value of 0.  It's at this offset that it decodes each datafield based on its type.  So in the above example, at offset 0, it will decode a UINT8 into a datafield called "Version" and so on until all fields are processed.

Field Flags

flag: msg_type

The PacketHeader or MessageHeader is very important because it must inform the decoder which field is the message type.  The message type is used by the decoder to select the fields to decode defined in "Messages" for each message.

We do this by setting the flag "msg_type"  on the field we want to use to select messages on.  In the example below, it's the field "MessageType" and its data type is a uint8.  The decoder will decode this uint8 at the offset of "MessageType" and then use it to determine which "Message" fields to use to decode the payload.

The "type" of a msg_type field can be either a int, uint, char or string.  If the msg_type is char, then the size must be 1.  If the msg_type is a string, its "size" must be from 1-8 bytes.

integer message type, message definitions example

"MessageHeader": {
{ "name":"MessageType", "type": "UINT8", "flags":["msg_type"] },
}...
"Messages": {
"1": {
"name": "FirstMessage",
"fields": []
},
"2": {
"name": "SecondMessage",
"fields": []
},
"3": {
"name": "ThirdMessage",
"fields": []
},
...

string message type, message definitions example

"MessageHeader": {
"MessageType": { "type": "string", "size":2, "flags":["msg_type"] },
}
...
 
"Messages": {
"CA": {
"name": "StartofDay",
"fields": []
},
"CC": {
"name": "FINRAClose",
"fields": []
},
"CL": {
"name": "ResetBlockSequenceNumber",
"fields": []
},
"CO": {
"name": "FINRAOpen",
"fields": []
},...

flag: msg_count

If a protocol contains encodes the number of messages in the packet in its data, this flag can be used to instruct the ACD to loop through the packet "msg_count" number of times.  The internal offset continually updates to iterate through all the messages in the packet.

flag: msg_size

If a protocol contains an encoded length to the next message in its format, this flag can be used to instruct the ACD to use the encoded length to skip to the next message.  The offset is updated with this length for each message looped through by the "msg_count" flag.  If msg_count is not specified, this flag will do nothing.  Using the encoded length to iterate messages like this allows fields to be skipped from the message definition; without this flag, every field in the message has to be defined to allow the decoder to iterate each message correctly.

flag: seq_num

For gap detection to work correctly, this flag must be specified on a uint field which increases monotonically.  Gaps in the numbering of this field updates the msg statistics accordingly.

flag: implied_seq_num

Some protocols which contain multiple messages do not have a sequence number encoded per message, instead they usually have a single sequence number in the header and then the sequence number is "implied" for each additional message.  For example, if a packet contains 3 messages, the header may have a first message sequence number of 23, so the second message sequence number will therefore be 24 and the third will be 25.

flag: seq_map_key

To support protocols which maintain sequence numbers based on an additional identifier, e.g. stream id, we can use this flag to instruct the decoder to create a map of stream id's and maintain sequence numbers for each possible key.  For example, if we have a message with stream id 32, the sequence number for that message would be maintained as the value in the map for stream id 32, and so on.