MQTT Plug-in #

The MQTT Plug-in is designed to publish data to an MQTT broker and receive subscription data from a broker. The payload of the data must currently be in JSON format. The format of the JSON data being published can be configured to some extent, and is discussed in the Publisher Configuration section. The format of the JSON data being received is flexible, but does have limitations. For instance, if the JSON nesting structure is complex, the plug-in will have problems parsing the data properly. Keep the JSON payload as simple as possible. See the Subscriber Configuration section for the limitations of datatype parsing.

Attention

This plug-in assigns CPU affinity and requires elevated permissions to run correctly. Make sure that you have elevated permissions to execute the EC Protocol Bridge.

Note: Any unknown parameters in a configuration file will simply be ignored.

MQTT - Plug-in-Level Configuration Parameters #

Currently, there are three configuration parameters at plug-in level: RT thread core affinity, schedule priority, and the URI of the MQTT broker to connect with. In future, parameters will be added for defining the security of the broker connection.

mqtt: dx-core-affinity #

This optional parameter defines the core to which the publisher, subscriber, or both real-time threads will be assigned.

By default, this value is set to 0.

Example:

dx-core-affinity: 1

mqtt: dx-sched-priority #

This optional parameter defines the core real-time thread priority assigned to the publisher, subscriber, or both.

Example:

dx-sched-priority: 60

mqtt: broker address #

This mandatory parameter specifies the MQTT broker to connect with, to exchange data. It may be the address to an unsecure broker (TCP) or a secure broker (SSL). The unsecure port is usually 1883 and the secure port is usually 8883.

broker-address: tcp://localhost:1883 or ssl://192.168.1.35:8883

mqtt: cafile #

This parameter defines the path and filename to the Certificate Authority (CA) file. The CA file is the certificate that verifies a TLS connection to the broker. This parameter is ignored if the broker-address is unsecure (TCP). However, this parameter is required if the broker-address is SSL or WS.

Example:

cafile: /etc/mosquitto/ca_certificates/ca.crt

MQTT - Publisher Configuration Parameters #

MQTT Publisher - Required Parameters #

The publisher configuration parameters must include a unique publisher ID, a dataset ID that defines the data fields to be published, and the MQTT topic name.

mqtt-publisher: publisher-id #

This parameter defines the publisher identifier. You need to select the value of the publisher-id parameter, and make sure that the value is unique within the configuration file.

Example:

publisher-id: pub1

mqtt-publisher: dataset-id #

This parameter is the dataset containing the data to be published by MQTT.

Example:

dataset-id: mqtt-pub-ds

mqtt-publisher: mqtt-topic #

This parameter defines the topic name for the MQTT publication that is being sent to the broker. Subscribers will use this topic name to select publisher data that they want to receive.

Example:

mqtt-topic: message

Automatic JSON Format #

The format of the JSON payload is generated automatically based on the dataset field IDs defined for the specified dataset. The field names are used as the JSON key name. This is best explained using an example. For instance, consider a dataset with the following field definitions:

dataset:
-
  dataset-fields:
    -
      datafld-id: fld1
      datatype: int32
    -
      datafld-id: fld2
      datatype: double
    -
      datafld-id: fld3
      datatype: string

The following JSON payload format will automatically be generated:

{"fld1":123,"fld2":4.5,"fld3":"hello world"}

Optional Parameters for Custom JSON Format #

There are two ways to modify the JSON payload format.

The first method is used to modify the format of the automatically generated JSON to handle multiple “rows” or groups of repeating fields. The optional json-group-key parameter is used for this purpose.

In the second method, you can provide a complete definition of the JSON format. This allows for increased format flexibility when needed (with certain limitations). The optional json-format parameter is used for this purpose.

Both these JSON formats conform to the RFC 8259 standard.

Method 1: json-group-key #

Example:

json-group-key: data

When specifying this parameter, the JSON payload in the above example will be modified as:

{"data":[{"fld1":123,"fld2":4.5,"fld3":"hello world"}]}

Method 2: json-format #

This is a simple example of the parameter:

Example:

json-format: >
  {"data":[{"Counter":"$fld1","Measurement":"$fld2","Source":"$fld3"}]}

This is an advanced example:

        json-format: >
{
      "static data": {
      "hello": "world",
      "foo": "bar"
      },
      "data": [
              {
              "Counter": "$fld1",
              "Measurement": "$fld2",
              "Source": "$fld3"
              }
      ]
}

All JSON key values must refer to a corresponding dataset field ID that supplies data. This name is prefixed with a dollar sign ($), see $fld1, $fld2, and $fld3 in the above examples. These names are placeholders for the actual data values received, similar to macro substitution. Also, these dataset field references must be contained within the group key. In the above example, Counter, Measurement, and Source are valid keys as they are contained within data, but hello and foo are not.

In either case, the plug-in will no longer automatically generate the JSON payload, but will match the specified format. The following are some important pointers:

This parameter must be used in conjunction with the json-group-key parameter. This json-group-key must name the JSON key that contains the array of the key/value pair objects. See “data” in the above examples.
The supplied JSON must conform to the RFC 8259 standard or the parser will reject it. Use a JSON validator to make sure that the custom JSON is properly formatted and meets this standard.
All YAML formatting rules still apply to this configuration parameter, that is, make sure that the specified JSON format is properly indented.

MQTT - Subscriber Configuration Parameters #

The MQTT subscriber, unlike the publisher, only supports dataset field types of int32, double, bool, and string. This limitation is due to the JSON library, which only handles those four datatypes. Though data is coerced from these supported types into other datatypes, there could be loss of precision or data corruption.

MQTT Subscriber - Required Parameters #

The subscriber configuration parameters must include a unique subscriber ID, a dataset ID that defines the data fields being received by the subscriber, the MQTT topic name, and the format definition of the JSON payload.

mqtt-subscriber: subscriber-id #

This parameter defines the subscriber ID. You need to select the value of the subscriber-id parameter, and make sure that the value is unique within the configuration file.

Example:

subscriber-id: sub1

mqtt-subscriber: dataset-id #

This parameter must reference a dataset defined for this plug-in.

Example:

dataset-id: mqtt-sub-ds

mqtt-subscriber: mqtt-topic #

This parameter defines the topic name of the MQTT subscription that is received from the broker. This topic name must match the topic of the publisher supplying data.

Example:

mqtt-topic: message

mqtt-subscriber: json-format #

This parameter defines the format of the incoming JSON payload. The specified keys must match the name of the incoming JSON payload keys. See Counter, Measurement, and Source in the below example. The specified key values must refer to a corresponding dataset field ID that receives data. These field names are prefixed with a dollar sign ($), see $fld1, $fld2, and $fld3 in the below example. These names are placeholders for the actual data values received, similar to macro substitution.

Note

The key value for the field names must always be within double-quotes, even if the data type is numeric, for example, "$fld1". This is necessary since the field name specifier is always a string and must be in JSON format that will pass the validation check.

Example:

json-format: >
           {"data":[{"Counter":"$fld1","Measurement":"$fld2","Source":"$fld3"}]}

The following are some important pointers:

The supplied JSON must conform to the RFC 8259 standard or the parser will reject it. Use a JSON validator to make sure that the custom JSON is properly formatted and meets this standard.
All YAML formatting rules still apply to this configuration parameter, that is, make sure that the specified JSON format is properly indented.

MQTT - Putting it all together #

This section provides the sample configuration snippets for both the publisher and subscriber. This section also provides the list of example configuration files, which can be used for testing.

MQTT - Publisher Configuration Example #

The following is a snippet of the configuration file for an MQTT publisher:

-
  plugin-id: plg-mqtt-pub
  filename: libplgmqtt.so
  dataset:
  -
        dataset-id: mqtt-pub-ds
        listener-dataset-id: sim-ds
  configuration:
    broker-address: tcp://localhost:1883
    publishers:
      -
        publisher-id: pub1
        dataset-id: mqtt-pub-ds
        mqtt-topic: message
        # Optional parameters for custom JSON format
        json-group-key: data
        json-format: >
              {"data":[{"Counter":"$fld1","Measurement":"$fld2","Source":"$fld3"}]}

MQTT - Subscriber Configuration Example #

The following is a snippet of the configuration file for an MQTT subscriber:

  plugin-id: plg-mqtt-sub
filename: libplgmqtt.so
dataset:
  -
    dataset-id: mqtt-sub-ds
    dataset-fields:
      -
        datafld-id: fld1
        datatype: int32
      -
        datafld-id: fld2
        datatype: double
      -
        datafld-id: fld3
        datatype: string
configuration:
  broker-address: tcp://localhost:1883
  subscribers:
    -
      subscriber-id: sub1
      dataset-id: mqtt-sub-ds
      mqtt-topic: message
      json-format: >
        {"data":[{"Counter":"$fld1","Measurement":"$fld2","Source":"$fld3"}]}

MQTT - Test Configuration Files #

The following configuration files can be found in /opt/ec-protocol-bridge/config:

mqtt-loopback.yaml: Both publisher and subscriber in a single configuration file