Calculated Fields

Sometimes, we need to create additional fields that are created from existing datafields. For example, converting Seconds datafields into nanoseconds, or creating a timestamp field from existing fields containing the different resolutions of a timestamp. Calculator Fields are processed after all the datafields have been set by the decoder and just before collection.

Calculator Fields have a "target" and an optional "source", either of which can be a datafield or a previously cached value. If the "target" is not specified for a Calculator field, then the default target is a new datafield. The field is then given a base type (currently only uint and timestamp), and a list of actions.

Actions define a series of operations that are executed and the final result is stored in the "target".

The target starts at 0 and the result is accumulated as each action is processed.

The first action is a special "assign" action; this is the only action where a "source" and "const_int" can be specified. The result of this action is written into the target. The following actions "accumulate" the result.

The first calculation in the example below shows how an existing “SecondsSinceEpoch” datafield is converted into a new nanosecond cached value called "CachedTimestampEpochNS". Cached calculation results persist across message and packet boundaries. The second calculation uses the cached value to set the "FullTimestamp" datafield and then adds the value in the "NanosecondTimestamp" field to yield the final result of the combined nanosecond values. If the "NanosecondTimestamp" datafield does not exist, the new calculated field is not set.

Calculated Fields example

  "CalculatedFields": {
    "fields":[
      { "name":"CachedTimestampEpochNS",
        "target":"Cache",
        "type":"uint",
        "actions":[
          { "action": "multiply", "df_source":"SecondsSinceEpoch", "const_int":1000000000 },
        ]
      },
      { "name":"FullTimestamp",
        "type":"timestamp",
        "actions":[
          { "action": "assign", "cache_source":"CachedTimestampEpochNS" },
          { "action": "add", "df_source":"NanosecondTimestamp" },
        ]
      }
    ]
  }

CalculatedFields object:

JSON key: "name" - required
JSON value: A string representing the name of the new datafield or cache entry.

JSON key: "target" - optional
JSON value: Can either be "Datafield" or "Cache". If this key is not specified the default is "Datafield".

JSON key: "type" - required
JSON value: Currently only "uint" and "timestamp" are supported.

JSON key: "actions" - required
JSON value: A JSON array of objects describing the set of operations to apply to the target.

Actions object:

JSON key: "action" - required
JSON value: A string representing the calculation operation required to apply to the source value.

Valid values are:

"assign" - assigns the starting value to the contents of the source field or const_int.
"add", "subtract", "multiply", "divide" - applies the mathematical operation to the source value and/or the const_int value.

After the action is specified, there must be either one "source" or a "const_int" value for the action to calculate.

Sources can be:

JSON key: "df_source" - optional
JSON value: A string representing the name of the datafield to read the source value from.
JSON key: "cache_source" - optional
JSON value: A string representing the name of the cache entry to read the source value from.
JSON key: "predefined_source" - optional
JSON value: A string representing a keyword with a built-in function:
Valid values are:
"packet_epoch_date_ns" - If this is specified the timezone must also be specified.
JSON key: "timezone" - required if packet_epoch_date_ns is specified.
JSON value: A string representing the timezone required for converstion of timestamps in different timezones.
Examples are: "Europe/London","America/New York".
The timezone strings are defined in the file date_time_zonespec.csv which is part of the VMX-Capture install.

JSON key: "source_time_format" - optional
JSON value: A string representing a boost date/time format specifier as described here:
https://www.boost.org/doc/libs/1_77_0/doc/html/date_time/date_time_io.html#format_strings
E.g. "source_time_format":"%Y%m%d%H%M%S" will decode the string "20220101120000" into a nanosecond epoch time.
JSON value: A string representing the timezone required for converstion of timestamps in different timezones. It can be used to adjust the source_time_format date/time conversion as required.
Examples are: "Europe/London","America/New York".
The timezone strings are defined in the file date_time_zonespec.csv which is part of the VMX-Capture install.

If there is no "source" the following must be specified:

JSON key: "const_int" - optional

JSON value: A signed number which is used in this calculation action, must not be 0.