How to Configure the MQTT Collector (version 24)

Create a Logging Session

Within the MQTT Collector, tags are organized by connection groups.  Each connection group represents a logging session that is writing tags to a Canary DataSet. To create a logging session, you will need a new Connection Group, the credentials for an MQTT broker, and subscription information.

Configure a Connection Group

1. Open the Canary Admin client and navigate to the MQTT Collector tile>Configuration screen.
2. By default, a preexisting connection group to a "local mosquitto" broker is created which can be modified or used as needed.
3. If wanting to create a new connection group, select the + in the upper left hand corner of the window. 
4. Name your new group. 
5. Configure an MQTT Client ID or click the refresh icon to the right of the field to generate a random ID. A client ID should be unique within the broker. (If the Broker returns 'Duplicate Client Errors', a new MQTT Client ID must be generated.)
6. Select a Payload Type: SparkplugB 3.00, SparkplugB 2.2, Other (JSON). Proceeding configuration options will vary depending on the payload type. For this example, we will assume SparkplugB 2.2 is selected.
7. Select the Primary Application checkbox if Canary's MQTT Collector is registered as the Primary Application for the Broker. If enabled, data will buffer in the event the Collector disconnects from the broker, provided the publishing devices are capable buffering data. Keep in mind that if the primary application disconnects, all other clients subscribed to the same topic will not receive data until the primary application reconnects.
8. Select Request Device Births to receive a birth message from the publishing devices when the Collector connects to the broker. This should almost always be enabled.
9. Select Log No Data On Deaths if wishing to log a NoData value whenever the Collector receives a NDEATH or DDEATH message.
10. Select Store No Data On Disconnect if wishing to log a NoData value whenever the Collector disconnects from the broker. If the Collector is the primary application and the publishing devices can buffer data, leave this unchecked.

Add a Server

1. Select the tcp://localhost:1883 string below Servers to configure the MQTT broker. 
2. In the URI field enter the machine name and port number of the MQTT Broker, preceded by tcp:// For example, tcp://localhost:1883/ if the broker is local and using unsecure port 1883.
3. Enter a Username and Password to enable authentication if using secure port 8883. 
4. Enter a value for the Reconnect Interval. The Collector will try to reconnect to the Broker at the specified frequency (defaulted at 00:01:00).
5. Enter a value for the Keep Alive Interval. This MQTT session configuration controls how often a Broker is notified that the session is still open. Some Brokers are configured to disconnect after a period of non-interaction (default is 30 and represents seconds).
6. Select Clean Session to wipe out any session information that the Broker has for one of the MQTT Client Ids used by the MQTT Collector in the past. 
7. Select Enable SSL/TLS to use SSL/TLS security protocols and to prompt the Self Signed Certificate options to appear.
   • Self Signed Certificate - Enabling this ignores any errors that get reframed by the certificate validation. Otherwise, it will validate that certificate.
   • Client Certificate - If enabled, uses the configured certificate found in the Settings tab at the bottom.

Choose Broker Subscriptions

1. Select spBv1.0/# below Subscriptions to configure a new subscription. 
2. All SparkplugB topics must start with spBv1.0/ and can be broad, subscribing to all available devices and sub nodes or more detailed subscribing to specific sub nodes. The default topic, spBv1.0/# subscribes to everything in the broker. The # is a wildcard to include 0 or more levels. Similarly, a + can be used to include 1 level. Topics are defined in SparkplugB as follows:

   spBv1.0/<GroupName>/<MessageType>/<NodeName>/<DeviceName>

   The third level of the topic path, <MessageType>, refers to a specific command sent. Please ensure this is set to a + so that the collector can receive all message types (Ex: spBv1.0/Group/+/Node/#). Topics are case-sensitive (Group1 != group1).

   When subscribing to a topic, please ensure that you do NOT subscribe to the device level. The collector needs contextual information from the node level to ensure correct template processing.

   • Good example: spBv1.0/Group1/+/Node3/#

   •  Bad example: spBv1.0/Group1/+/Node3/Device4/

3. Specify the 'QOS' (Quality of Service) setting which guarantees the reliability of message delivery under different network environments.  MQTT QOS Levels include:

4. QOS 0 "At most once"	Sometimes referred to as "Fire and Forget" this QOS level puts the least strain on system resources, but data may occasionally be lost. This is preferable for client - server connections that are stable.
   QOS 1 "At Least once"	Guarantees that a message is delivered at least one time to the receiver. With this QOS level more strain is put on system resources than QOS 0 but not as much as QOS 2. The downside is that duplicate messages may be delivered if no confirmation response is received from the MQTT Collector.
   QOS 2 "Exactly Once"	Guarantees that a message is delivered exactly once, but puts the most strain on system resources.

5. Next, select Enabled. If at a later time you wish to disable this subscription you may uncheck this box.
6. In the DataSet Prefix specify the name of the DataSet you wish to log to. The collector will automatically create the DataSet if it does not exist.
7. In the Tags Per DataSet set the tag limit for the above DataSet. If that tag limit is reached, a new DataSet with the same prefix will automatically be created.  New DataSets will be appended with numeric iterations (e.g., MQTTData1, MQTTData2).
8. Use the Remove Tag Name Prefix setting to remove unnecessary branches in the tag name. The value entered determines the number of branches to remove from the beginning of the tag name. Some tags may have generic node prefixes that users may wish to eliminate.  WARNING: Often the Unified Name Space provided in the Sparkplug specification may result in tag name duplication if branches are removed.  Only use this feature if you are confident in your tag naming.
9. Determine whether to Automatically Log all Tags. With this checked any tags that come in are automatically enabled. Users must decide for themselves if it is easier to enable all tags and disable those that are unnecessary, or disable all tags and enable those that the users wishes to log. If this feature is disabled, you will need to use the Tags table at the bottom of the Connection Group to enable tags once the session has been enabled.
10. Sync historian tags with births - If enabled, tags will be added and removed automatically in the Historian based on the messages the Collector receives. If disabled, tags will only be added to the Historian and need to be manually removed if necessary.

Create the Sender Sessions

1. Select localhost below Sender Sessions. 
2. In the Historian field designate the Historian(s) to send data to. If logging to a local Historian, leave as localhost.
   Dual Logging with the MQTT Collector is as simple as typing an additional Historian machine name, separating each with a comma (e.g., Historian1, Historian2)
3. Ensure the Sender Session is Enabled. 
4. Select the External Property Storage checkbox to store the metadata properties of a tag in a SQLite database (rare).  For this to function, external property storage must also be configured from the Configuration>Settings screen of the Views tile. (See How to Setup External Property Storage)

Create Optional Filters

Filtering provides a method to subscribe to a topic namespace but omit certain tags that are found within the namespace. Filtering is applied to the entire topic path including the tag name after any number of branches have been removed using the Remove tag name prefix (# of branches) setting.

1. Select the + icon to create a Filter to quickly include or exclude tags in the tag list. 
2. Add a Regular Expression in the Expression field to capture tags. 
3. Select the Exclude checkbox to exclude those tags captured in the RegEx above. 

Add Universal Tag Properties

This feature allows you to manually inject universal properties to all tags within a subscription in additional to any metadata already collected at the device level.  Most use cases will not need to use this feature.

1. Select the + beside Tag Properties to add universal property names and values to all tags within the subscription. 
2. Set the Property name and add the Property value. 

Enable/Disable Tags 

If Automatically Log All Tags is not selected within the subscription, tags will need to be individually enabled in order to log to the Historian. This can only be done once the Connection Group has been enabled from the Status screen. Once enabled, the session should populate with the subscribed tags. The session will then need disabled to edit any configuration settings.

1. Select Tags
2. Use the paging operator above to shift through a tag list greater than 50,000. A new page is created for every 50,000 tags. 
3. Use the DISABLE ALL, DELETE ALL, and ENABLE ALL selectors to configure all highlighted tags at once. 

Tag List Filter

Users have another way to filter tags through the tag list itself, using the Filter mechanism. 

1. Select the FILTER button. 
2. Choose a specific topic or topic(s) to filter through with the Published Topic drop down menu. 
3. Select either Contains or Regex to determine how the Published Name search feature will operate. Contains switches the search from running off of Regular Expressions to the standard search by character. 
4. Use the Canary DataSet menu to search by DataSet. 
5. The Initial Birth and Latest Birth options filter tags based on when the tag values are received. Use the Latest Birth option to quickly find all the recently active tags.
   • The Initial Birth is the date time when the tag value was first received by the collector. It's only set 1 time for a tag and it is the date of the first birth message that the tag was received in.
   • The Latest Birth is set each time a birth message is received by the collector, so it will continually update every time a device sends a birth message.