How to Configure the MQTT Collector (version 21)

Select the MQTT Collector tile within the Canary Admin application to create and manage MQTT data collection. 


Within the MQTT Collector, tags are organized by connection groups.  Each connection group represents a logging session that is writing tags to a Canary DataSet. To create a logging session, you will need a new Connection Group, the credentials for an MQTT broker, subscription information, and a configured Sender session.

  1. Open the Canary Admin application, select the MQTT Collector tile, and click 'Configuration' from the lower tab menu.
  2. Create a new connection group by selecting the in the upper left hand corner of the window. 
  3. Name your new group. 
  4. Select the refresh icon to the right of the MQTT Client ID field to generate a new MQTT Client ID. A new MQTT Client ID must be generated for every connection group. (If the Broker returns 'Duplicate Client Errors', a new MQTT Client ID must be generated.)
  5. Select the 'Primary Application' checkbox if Canary's MQTT Collector is registered as the Primary Application for the Broker. The Primary Application ensures that the Broker is notified whenever Canary's MQTT Collector is online. Once notified, the Broker publishes all buffered historical data as soon as the MQTT Collector goes online.
    • Ignition's MQTT Transmission module on the Ignition server is capable of buffering.
    • Keep in mind that either all clients are receiving data from the publishing device or none are. When the MQTT Collector is designated as the Primary Application and it goes offline, no other clients are receiving data.
  6. Select Request Device Births only when the publishing device supports Birth and Last Will Testament messaging, otherwise no data will be collected at all.
    • Turning on 'Request Device Births' ensures that the full tag list is sent along with value updates. 


  1. In the 'Connection Group' window, select the 'tcp://localhost:1883' string below 'Servers' to configure the MQTT broker. 
  2. In the 'URI' field enter the machine name and port number of the MQTT Broker, preceded by tcp:// For example, tcp://localhost:1883/ if the broker is local and using unsecure port 1883.
  3. Enter a Username and Password to enable authentication if using secure port 8883. 
  4. Enter a value for the 'Reconnect Interval'. The Collector will try to reconnect to the Broker at the specified frequency (defaulted at 00:01:00).
  5. Enter a value for the 'Keep Alive Interval'. This MQTT session configuration controls how often a Broker is notified that the session is still open. Some Brokers are configured to disconnect after a period of non-interaction (default is 30 and represents seconds).
  6. Select 'Clean Session' to wipe out any session information that the Broker has for one of the MQTT Client Ids used by the MQTT Collector in the past. 
  7. Select 'Enable SSL/TLS' to use SSL/TLS security protocols and to prompt the 'Self Signed Certificate' options to appear.
    • Self Signed Certificate - Enabling this ignores any errors that get reframed by the certificate validation. Otherwise, it will validate that certificate.


  1. Select 'spBv1.0/#' below 'Subscriptions' to configure a new subscription. 
  2. Create a new subscription topic. All sparkplug B topics must start with this namespace identifier: spBv1.0/ and can be broad, subscribing to all available devices and sub nodes or more detailed subscribing to specific sub nodes.
  3. Specify the 'QOS' (Quality of Service) setting which guarantees the reliability of message delivery under different network environments.  MQTT QOS Levels include:
  4. QOS 0 "At most once" Sometimes referred to as "Fire and Forget" this QOS level puts the least strain on system resources, but data may occasionally be lost. This is preferable for client - server connections that are stable.  
    QOS 1 "At Least once" Guarantees that a message is delivered at least one time to the receiver. With this QOS level more strain is put on system resources than QOS 0 but not as much as QOS 2. The downside is that duplicate messages may be delivered if no confirmation response is received from the MQTT Collector.
    QOS 2 "Exactly Once" Guarantees that a message is delivered exactly once, but puts the most strain on system resources.
  5. Next, select 'Enabled'. If at a later time you wish to disable this subscription you may deselect this box.
  6. In the 'DataSet Prefix' specify the name of the DataSet you wish to log to. The collector will automatically create the DataSet if it does not exist.
  7. In the 'Tags Per DataSet', set the tag limit for the above DataSet. If that tag limit is reached, a new DataSet with the same prefix will automatically be created.  New DataSets will be postpended with numeric iterations (e.g., MQTTData1, MQTTData2).
  8. Use the 'Remove Tag Name Prefix' setting to remove unnecessary branches in the tag name. The value entered determines the number of branches to remove from the tag name. Some tags may have generic node prefixes that users may wish to eliminate.  WARNING: often the Unified Name Space provided in the Sparkplug specification may result in tag name duplication if branches are removed.  Only use this feature if you are confident in your tag naming.
  9. Determine whether to 'Automatically Log all Tags'. With this checked any tags that come in are automatically enabled. Users must decide for themselves if its easier to enable all tags and disable those that are unnecessary, or disable all tags and enable those that the users wishes to log. If this feature is disabled, you will need to use the 'Tags' feature to select and enable tags once the session has been enabled.


  1. Select 'localhost' below 'Sender Sessions'. 
  2. In the 'Historian' field designate the historian(s) to send data to. Dual Logging with the MQTT Collector is as simple as typing an additional historian machine name, separating each with a comma (e.g., Historian1, Historian2)
  3. Ensure the Sender Session is 'Enabled' by checking the box. 
  4. Select the 'External Property Storage' checkbox to store the metadata properties of a tag in either an SQLLite or MSSQL database.  For this to function, external property storage must also be configured from the settings menu of the Historian tile.

Filtering provides a method to subscribe to a topic namespace but omit certain tags that are found within the namespace.

  1. Select the icon to create a Filter to quickly include or exclude tags in the tag list. 
  2. Add a Regular Expression in the 'Expression' field to capture tags. 
  3. Select the 'Exclude' checkbox to exclude those tags captured in the RegEx above. 

This feature allows you to manually inject universal properties too all tags within a subscription in additional to any metadata already collected at the device level.  Most use cases will not need to use this feature.

  1. Select the beside 'Tag Properties' to add universal property names and values to all tags within the subscription. 
  2. Set the Property name and add the Property value. 

If 'Automatically Log All Tags' is not selected within the subscription, tags will need to be individually enabled in order to log to the historian.  This can only be done once the Connection Group has been enabled from the 'Status' tab.

  1. Select 'Tags'
  2. Use the paging operator above to shift through a tag list greater than 50,000. A new page is created for every 50,000 tags. 
  3. Use the 'DISABLE ALL', 'DELETE ALL', and 'ENABLE ALL' selectors to configure all highlighted tags at once. 

If you are successfully connected to the broker but no tags are logging, it is likely that you have not enabled the tags.  This can be solved by either selecting 'Automatically Log All Tags' in the subscription settings or by using the 'Tags' setting and enabling all tags.  For either, you will need to first disable the connection group prior.

Users have another way to filter tags through the tag list itself, using the Filter mechanism. 

  1. Select the 'FILTER' button. 
  2. Choose a specific topic or topic(s) to filter through with the 'Published Topic' drop down menu. 
  3. Select either 'Contains' or 'Regex' to determine how the 'Published Name' search feature will operate. 'Contains' switches the search from running off of Regular Expressions to the standard search by character. 
  4. Use the 'Canary DataSet' menu to search by DataSet. 
  5. The 'Initial Birth' and 'Latest Birth' options filter tags based on when the tag values are received. Use the 'Latest Birth' option to quickly find all the recently active tags.
    • The Initial Birth is the date time when the tag value was first received by the collector. It's only set 1 time for a tag and it is the date of the first birth message that the tag was received in.
    • The Latest Birth is set each time a birth message is received by the collector, so it will continually update every time a device sends a birth message.
5replies Oldest first
  • Oldest first
  • Newest first
  • Active threads
  • Popular
  • When using MQTT to log new tags I get this message in the log:

    "Metadata was received on topic '...'.  Logging metadata is unsupported."

    I see metadata can be manually configured for all tags. 

    Is it possible to setup MQTT to set the metadata for each tag when it is auto-created?

    • mike. dash SparkplugB metadata is meant to describe binary data being passed across a Sparkplug payload. An example would be the binary contents of a file and the metadata would be the filename and filesize. We can't think of a valid reason to store this type of information in the historian, so we log that message.

      There is another type of Sparkplug "metadata" on the metrics themselves called a PropertySet which has a key/value type of representation that we do support. These are converted into tag properties on the tag. If these propertyset's are configured in the device publishing to the broker, you would see them being stored in the historian when you drill down to the tag level.

      For more information on PropertySets you can read about them here on page 48.

  • Thanks for the info Steve Mason

    The PropertySet is what I was after.  I had a look at the spec, and it appears the library I'm using to send the Sparkplug B payload is sending the PropertySet correctly.  I have an MQTT monitor setup.  Here is an example of the metric payload in NBIRTH:

    metrics {
      name: "exampletag"
      timestamp: 1633050671111
      datatype: 11
      properties {
        keys: "Eng Units"
        values {
          string_value: "Mikes"
      boolean_value: false

    The tag is created in Canary just fine.  However no properties are set.

    I also noticed that when using the Data Generator tool, it automatically adds properties to the tags if the Target is set to Canary, however, if it is set to MQTT it doesn't add any properties.  From the MQTT monitor it looks like the DataGen tool doesn't send any properties.  I was hoping it would so I could compare what I'm sending to it, and see where the problem is.

    • Hi mike. dash 

      Here is the format we're expecting:

          "timestamp": 1639144921494,
          "metrics": [
                  "name": "RAFTS-HQ/Drilling/Auto_Driller_BIT_Position",
                  "alias": 1,
                  "timestamp": 1639144914000,
                  "dataType": "Float",
                  "isHistorical": true,
                  "properties": {
                      "Description": {
                          "type": "String",
                          "value": "BIT Position"
                      "Eng Units": {
                          "type": "String",
                          "value": "ft"
                  "value": 60.729008

      I'm using a tool called MQTT.fx to capture this, which you can download here http://www.jensd.de/apps/mqttfx/1.7.1/. Within that tool there is a "Sparkplug Decoder" which converts it to JSON. That's how I produced the snippet above. If you run this tool and capture an NBIRTH or DBIRTH message, can you see the properties coming through? Feel free to open a ticket by emailing support@canarylabs.com and we can keep troubleshooting this.


      Regarding the Data Gen tool, it doesn't appear to support properties at this time. I'll make development aware for them to consider adding that feature.

    • mike. dash , as a follow-up, the Data Gen will have support to publish properties in the official v22.0 release.