How to Configure the MQTT Collector (version 23)

Begin Logging MQTT Data

Select the MQTT Collector tile within the Canary Admin application to create and manage MQTT data collection. 

Create a Logging Session

Within the MQTT Collector, tags are organized by connection groups.  Each connection group represents a logging session that is writing tags to a Canary DataSet. To create a logging session, you will need a new Connection Group, the credentials for an MQTT broker, subscription information, and a configured Sender session.

Configure a Connection Group

1. Open the Canary Admin application, select the MQTT Collector tile, and click 'Configuration' from the lower tab menu.
2. Create a new connection group by selecting the + in the upper left hand corner of the window. 
3. Name your new group. 
4. Configure an MQTT Client ID or click the refresh icon to the right of the field to generate a random ID. A client ID should be unique within the broker. (If the Broker returns 'Duplicate Client Errors', a new MQTT Client ID must be generated.)
5. Select the 'Primary Application' checkbox if Canary's MQTT Collector is registered as the Primary Application for the Broker. The Primary Application ensures that the Broker is notified whenever Canary's MQTT Collector is online. Once notified, the Broker publishes all buffered historical data as soon as the MQTT Collector goes online.
   • Ignition's MQTT Transmission module on the Ignition server is capable of buffering.
   • Keep in mind that either all clients are receiving data from the publishing device or none are. When the MQTT Collector is designated as the Primary Application and it goes offline, no other clients are receiving data.
6. Select Request Device Births only when the publishing device supports Birth and Last Will Testament messaging, otherwise no data will be collected at all.
   • Turning on 'Request Device Births' ensures that the full tag list is sent along with value updates. 

Add a Server

1. In the 'Connection Group' window, select the 'tcp://localhost:1883' string below 'Servers' to configure the MQTT broker. 
2. In the 'URI' field enter the machine name and port number of the MQTT Broker, preceded by tcp:// For example, tcp://localhost:1883/ if the broker is local and using unsecure port 1883.
3. Enter a Username and Password to enable authentication if using secure port 8883. 
4. Enter a value for the 'Reconnect Interval'. The Collector will try to reconnect to the Broker at the specified frequency (defaulted at 00:01:00).
5. Enter a value for the 'Keep Alive Interval'. This MQTT session configuration controls how often a Broker is notified that the session is still open. Some Brokers are configured to disconnect after a period of non-interaction (default is 30 and represents seconds).
6. Select 'Clean Session' to wipe out any session information that the Broker has for one of the MQTT Client Ids used by the MQTT Collector in the past. 
7. Select 'Enable SSL/TLS' to use SSL/TLS security protocols and to prompt the 'Self Signed Certificate' options to appear.
   • Self Signed Certificate - Enabling this ignores any errors that get reframed by the certificate validation. Otherwise, it will validate that certificate.
   • Client Certificate - If enabled, uses the configured certificate found in the 'Settings' tab at the bottom.

Choose Broker Subscriptions

1. Select 'spBv1.0/#' below 'Subscriptions' to configure a new subscription. 
2. Create a new subscription topic. All SparkplugB topics must start with this namespace identifier: spBv1.0/ and can be broad, subscribing to all available devices and sub nodes or more detailed subscribing to specific sub nodes. For more information on SparkplugB specifications, click here.
3. Specify the 'QOS' (Quality of Service) setting which guarantees the reliability of message delivery under different network environments.  MQTT QOS Levels include:

4. QOS 0 "At most once"	Sometimes referred to as "Fire and Forget" this QOS level puts the least strain on system resources, but data may occasionally be lost. This is preferable for client - server connections that are stable.
   QOS 1 "At Least once"	Guarantees that a message is delivered at least one time to the receiver. With this QOS level more strain is put on system resources than QOS 0 but not as much as QOS 2. The downside is that duplicate messages may be delivered if no confirmation response is received from the MQTT Collector.
   QOS 2 "Exactly Once"	Guarantees that a message is delivered exactly once, but puts the most strain on system resources.

5. Next, select 'Enabled'. If at a later time you wish to disable this subscription you may deselect this box.
6. In the 'DataSet Prefix' specify the name of the DataSet you wish to log to. The collector will automatically create the DataSet if it does not exist.
7. In the 'Tags Per DataSet', set the tag limit for the above DataSet. If that tag limit is reached, a new DataSet with the same prefix will automatically be created.  New DataSets will be postpended with numeric iterations (e.g., MQTTData1, MQTTData2).
8. Use the 'Remove Tag Name Prefix' setting to remove unnecessary branches in the tag name. The value entered determines the number of branches to remove from the tag name. Some tags may have generic node prefixes that users may wish to eliminate.  WARNING: often the Unified Name Space provided in the Sparkplug specification may result in tag name duplication if branches are removed.  Only use this feature if you are confident in your tag naming.
9. Determine whether to 'Automatically Log all Tags'. With this checked any tags that come in are automatically enabled. Users must decide for themselves if its easier to enable all tags and disable those that are unnecessary, or disable all tags and enable those that the users wishes to log. If this feature is disabled, you will need to use the 'Tags' feature to select and enable tags once the session has been enabled.
10. Sync historian tags with births - If enabled, tags will be added and removed automatically in the historian based on birth messages. If disabled, tags will only be added to the historian and need to be manually removed if necessary.

Create the Sender Sessions 

1. Select 'localhost' below 'Sender Sessions'. 
2. In the 'Historian' field designate the historian(s) to send data to. Dual Logging with the MQTT Collector is as simple as typing an additional historian machine name, separating each with a comma (e.g., Historian1, Historian2)
3. Ensure the Sender Session is 'Enabled' by checking the box. 
4. Select the 'External Property Storage' checkbox to store the metadata properties of a tag in either an SQLLite or MSSQL database.  For this to function, external property storage must also be configured from the settings menu of the Historian tile. (See How to Setup External Property Storage)

Create Optional Filters

Filtering provides a method to subscribe to a topic namespace but omit certain tags that are found within the namespace. Filtering is applied to the entire topic path including the tag name after any number of branches have been removed using the 'Remove tag name prefix (# of branches)' setting.

1. Select the + icon to create a Filter to quickly include or exclude tags in the tag list. 
2. Add a Regular Expression in the 'Expression' field to capture tags. 
3. Select the 'Exclude' checkbox to exclude those tags captured in the RegEx above. 

Add Universal Tag Properties

This feature allows you to manually inject universal properties too all tags within a subscription in additional to any metadata already collected at the device level.  Most use cases will not need to use this feature.

1. Select the + beside 'Tag Properties' to add universal property names and values to all tags within the subscription. 
2. Set the Property name and add the Property value. 

Enable/Disable Tags 

If 'Automatically Log All Tags' is not selected within the subscription, tags will need to be individually enabled in order to log to the historian.  This can only be done once the Connection Group has been enabled from the 'Status' tab.

1. Select 'Tags'
2. Use the paging operator above to shift through a tag list greater than 50,000. A new page is created for every 50,000 tags. 
3. Use the 'DISABLE ALL', 'DELETE ALL', and 'ENABLE ALL' selectors to configure all highlighted tags at once. 

If you are successfully connected to the broker but no tags are logging, it is likely that you have not enabled the tags.  This can be solved by either selecting 'Automatically Log All Tags' in the subscription settings or by using the 'Tags' setting and enabling all tags.  For either, you will need to first disable the connection group prior.

Tag List Filter

Users have another way to filter tags through the tag list itself, using the Filter mechanism. 

1. Select the 'FILTER' button. 
2. Choose a specific topic or topic(s) to filter through with the 'Published Topic' drop down menu. 
3. Select either 'Contains' or 'Regex' to determine how the 'Published Name' search feature will operate. 'Contains' switches the search from running off of Regular Expressions to the standard search by character. 
4. Use the 'Canary DataSet' menu to search by DataSet. 
5. The 'Initial Birth' and 'Latest Birth' options filter tags based on when the tag values are received. Use the 'Latest Birth' option to quickly find all the recently active tags.
   • The Initial Birth is the date time when the tag value was first received by the collector. It's only set 1 time for a tag and it is the date of the first birth message that the tag was received in.
   • The Latest Birth is set each time a birth message is received by the collector, so it will continually update every time a device sends a birth message.