Publisher Tile (version 25)

Knowledge Base / Version 25 / System Admin Duties / Canary Admin

updated 3 hrs ago

Enabled - number of sessions running
Connected - number of sessions connected to their destination
Tags - total number of tags being published
Assets - number of assets being published (only applicable if using the Websocket session type)
Sent - amount of data that has been sent since the service started
Errors - number of errors that have been thrown since the Publisher service started
Version - versioning number of the Publisher service

The Publisher service provides four methods for providing real-time data to third party systems:

SparkplugB (2.2 or 3.0.0) payload over MQTT (3.1.1)
JSON payload over MQTT (3.1.1)
JSON payload over Kafka
JSON payload over a Websocket connection

Like other Canary services and clients, the Publisher requests data through the Views service; therefore, any virtual view is available in addition to the raw Historian archive.

Status Screen

The Status screen will display any publishing sessions that are configured.

NAME - unique name identifying the publishing session
TYPE - type of publishing session, JSONMQTT, SparkplugMQTT, Kafka, or Websocket
ENDPOINT - URI of the server and port number of where the data is being published. Port numbers listed are those commonly used for the particular method, but may vary depending on your environment.
- MQTT (Unsecure) - tcp://hostName:1883/
- MQTT (Secure) - tcp://hostName:8883/
- Kafka (Unsecure) - hostName:9092
- Kafka (Secure) - hostName:9093
- Websocket (Unsecure) - ws://hostName:portNumber/
- Websocket (Secure) - wss://hostName:portNumber/
ENABLED - True or False, if the session is running
STATE - connection status, Connected or Disconnected
CONNECTED SINCE - date/time of when the session last connected to the endpoint
DATA SENT - amount of data that has been sent since the start of the session
TAGS - total number of tags that have been published since the start of the session
ASSETS - total number of assets that have been published since the start of the session. (This option is only used for the Websocket connection.)

Configuration Screen

The Configuration screen is where the user can configure a publishing session to send data via MQTT, Kafka, or Websocket. All sessions will appear in the CONNECTIONS panel with their given name and type.

Settings (JSONMQTT)

Name - unique name identifying the publisher session
Type - JSONMQTT
Client Id - name given to the publisher client that must be unique in the MQTT broker it is connecting to. If two clients with the same ID connect to the broker, the last one in wins. The client ID can be edited so that it is more user-friendly.
MQTT Topic - topic path to publish data to in the broker. This topic is open-ended and does not require a specific structure (unlike SparkplugB).
Max Tags Per Message - a throttling mechanism that controls the amount of tags that are sent in a single message. This setting does not affect the birth message. (1≤ value ≤ 1 million)
Durable Stream - if false, only the most recent value is published. If true, all value changes since the last publish are sent. Not recommended for high throughput streams where updates exceed 10,000 updates/sec.
Publish QoS (Quality of Service) - level of agreement between the Publisher and MQTT broker that a particular message has been received
- AT MOST ONCE (QoS = 0) - also known as "fire and forget", there is no guarantee the broker received the message
- At LEAST ONCE (QoS = 1) - guarantees the message has been sent at least once to the broker. Can be sent multiple times until the broker confirms it has received the message.
- EXACTLY ONCE (QoS = 2) - highest and slowest level of security that guarantees the message is sent exactly once

For more information on QoS, visit https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/

Views Host - server name of the Views service to which the Publisher is reading data from. Default value is localhost.
Views Path - source view and path of the data the Publisher will send. This can be the raw historian view or any virtual view that has been created within the Canary Views service.
Servers - URI of the MQTT server to which the Publisher will send data. The default setting is tcp://localhost:1883/ indicating an unsecure connection to a local broker. Servers can be added to support a high availability MQTT infrastructure. If the Publisher cannot connect to the first configured broker, it will try the next configured one. The Publisher connects to only one broker at a time.
URI - allows the user to edit the server name and port number
Username/Password - credentials used to connect securely to the broker when SSL/TLS is enabled
Reconnect Interval - frequency at which the Publisher attempts to connect to the broker if there is a disconnect
Keep Alive - frequency at which the Publisher will call the broker to ensure the connection is still active. If the connection is very reliable, this can be set to 00:00:00.
Clean Session - when checked, updates the broker with the Publisher's settings such as client ID, QoS, and primary host ID when it connects. This should almost always be checked.
Enable SSL/TLS - used in conjunction with a username and password when connecting securely (8883) to a broker
Client Certificate - only visible when Enable SSL\TLS is checked. Sends the configured client certificate from the settings tab to the MQTT server.
Ignore SSl Errors – only visible when Enable SSL\TLS is checked. Enable only if using self-signed certificates on the server. Ignores errors when validating the server certificate.

Settings (SparkplugMQTT 2.2 and 3.0)

Name - unique name identifying the publisher session
Type - SparkplugMQTT
SparkplugB Version - 2.2 or 3.0 (See SparkplugB 3.0 Compliance for more details.)
Client Id - name given to the publisher client that must be unique in the MQTT broker it is connecting to. If two clients with the same ID connect to the broker, the last one in wins. The client ID can be edited so that it is more user-friendly.
SparkplugB Group Id - group topic name in the SparkplugB structure
SparkplugB Node Id - node topic name in the SparkplugB structure
SparkplugB Device Id - device topic name in the SparkplugB structure
SparkplugB Primary Id - (optional) a unique ID that must be configured on both the publisher and subscriber side so that the Publisher is aware when a subscriber is online or offline. If a subscriber using the same primary host ID disconnects from the broker, the Publisher is notified and will stop sending data. (Note: If using a primary ID, the Publisher will not send data to a broker until a subscriber connects using this primary ID.)
Alias tags - allows the Publisher to alias tag names using an integer index; therein, reducing the amount of data being sent
Compression - compression options when sending data
- NONE
- GZIP
- DEFLATE
Publish Sub ms Time - if enabled, publishes the sub-milliseconds as a property of the metric value. This is only needed if wishing to consume the data back through using the Canary MQTT Collector for testing purposes.
Max Tags Per Message - throttling mechanism that controls the amount of tags that are sent in a single message. This setting does not affect the birth message. (1≤ value ≤ 1 million)
Durable Stream - used in conjunction with the SparkplugB Primary Id, allows the Publisher to "buffer" data for subscribers that disconnect who are using the same primary ID. When the subscriber reconnects, the Publisher will start sending data from the time the subscriber disconnected so there is no data loss. Not recommended for high throughput streams where value updates exceed 10,000 updates/sec.
Stream Flush Mode - visible when Durable Stream is enabled
- Synchronous - will flush historical data first until it gets to live data. Messages are flushed in order.
- Asynchronous - live data will be published in parallel with historical data. Message are flushed out of order.
Publish properties
- DontInclude - properties are never published
- BirthAndChange - properties are only published when a new tag comes online or the property of a tag changes
- EveryTagValueChange - properties are published with each tag value change
Publish QoS (Quality of Service) - level of agreement between the Publisher and MQTT broker that a particular message has been received
- AT MOST ONCE (QoS = 0) - also known as "fire and forget", there is no guarantee the broker received the message
- At LEAST ONCE (QoS = 1) - guarantees the message has been sent at least once to the broker. Can be sent multiple times until the broker confirms it has received the message.
- EXACTLY ONCE (QoS = 2) - highest and slowest level of security that guarantees the message is sent exactly once

For more information on QoS, visit https://www.hivemq.com/blog/mqtt-essentials-part-6-mqtt-quality-of-service-levels/

Views Host - server name of the Views service to which the Publisher is reading data from. Default value is localhost.
Views Path - source view and path of the data the Publisher will send. This can be the raw historian view or any virtual view that has been created within the Canary Views service.
Servers - URI of the MQTT server to which the Publisher will send data. The default setting is tcp://localhost:1883/ indicating an unsecure connection to a local broker. Servers can be added to support a high availability MQTT infrastructure. If the Publisher cannot connect to the first configured broker, it will try the next configured one. The Publisher connects to only one broker at a time.
URI - allows the user to edit the server name and port number
Username/Password - credentials used to connect securely to the broker when SSL/TLS is enabled
Reconnect Interval - frequency at which the Publisher attempts to connect to the broker if there is a disconnect
Keep Alive - frequency at which the Publisher will call the broker to ensure the connection is still active. If the connection is very reliable, this can be set to 00:00:00.
Clean Session - when checked, updates the broker with the Publisher's settings such as client ID, QoS, and primary host ID when it connects. This should almost always be checked.
Enable SSL/TLS - used in conjunction with a username and password when connecting securely (8883) to a broker
Client Certificate - only visible when Enable SSL\TLS is checked. Sends the configured client certificate from the settings tab to the MQTT server.
Ignore SSl Errors – only visible when Enable SSL\TLS is checked. Enable only if using self signed certificates on the server. Ignores errors when validating the server certificate.

Settings (Kafka)

Name - unique name identifying the publisher session
Type - Kafka
Kafka Provider - the Kafka broker to target: Standard, AzureEventHubs, and AmazonMSK. Standard should be used for Confluent or any generic Kafka broker.
Servers - semicolon-delimited (;) list of host names and their port number the Publisher will attempt to connect to
Client Id - client identifier used when connecting to the Kafka broker
Security Protocol (Standard) - protocol used to connect to brokers: Plaintext, Ssl, SaslSsl, and SaslPlaintext
Sasl Mechanism (Standard) - defines the mechanism used for authentication when SaslSsl or SaslPlaintext is enabled
Username/Password (Standard) - credentials to be used when Sasl is configured
Connection String (AzureEventHubs) - connection string used to connect to AzureEventHubs
IAM Access Key (AmazonMSK) - access key for authenticating with AWS IAM
IAM Secret Key (AmazonMSK) - access key secret for authenticating with AWS IAM
Region (AmazonMSK) - the region where the targeted MSK cluster resides
Ignore SSL Errors - enable only if using self-signed certificates on the servers
Acknowledgements - the acknowledgements that need received before the produce operation can complete. Selecting None means the client will not wait for any acknowledgement, resulting in the greatest throughput but the lowest durability. Data may be lost since the client is not notified of failures. All results in the client waiting for the leader and all in sync replicas to commit the message, resulting in the greatest durability but the lowest throughput. Leader means only the leader needs to respond before the operation completes.
Compression Type - compression type used for compressing messages. Options vary depending on the Kafka provider configured.
Batch Size (Bytes) - controls how often the producer sends a batch of messages when processing data received from Views. Increasing can improve throughput with large tag counts.
Max Message Size (Bytes) - maximum size of one message that can be produced. Increasing may provide better performance than decreasing the Max Tags Per Message setting.
Message Timeout - the amount of time to wait for successful delivery before a produce operation fails
Retry Interval - the amount of time to wait after failing to produce the current batch of messages
Topic - the topic to produce messages to
Partition Type
- PartitionByTagName - separate messages will be produced for each tag with the tag name used as the key
- StaticPartitionKey - messages will contain multiple tags and the configured key will be used
  - Partition Key - key to send with each message produced
  - Max Tags Per Message - the max number of tags per published message. The birth message is unaffected by this setting and could be larger. (1≤Max≤1,000,000)
Durable Stream - if false, only the most recent value is published. If true, all value changes since the last publish are sent. Not recommended for high throughput streams where updates exceed 10,000 updates/sec.
Views Host - server name of the Views service to which the Publisher is reading data from. Default value is localhost.
Views Path - source view and path of the data the Publisher will send. This can be the raw historian view or any virtual view that has been created within the Canary Views service.
Include Properties - if true, tag properties will be included in the published records as a separate record type
Schema Registry - schema registry provider to register the Canary schema with

Settings (WebSocket)

Name - unique name identifying the publisher session
Type - WebSocket
URI - server name and port number of where the Publisher is sending data to
JSON Payload
- Canary Defined
- User Defined

A Canary Defined payload will use the following JSON structure:

A User Defined payload gives the user the option to create their own mappings.

Views Host - server name of the Views service to which the Publisher is reading data from. Default value is localhost.
Views Path - source view and path of the data the Publisher will send. This can be the raw historian view or any virtual view that has been created within the Canary Views service.
Asset - (optional) allows the user to publish an asset name and its tags according to the asset type selected. (Note: This option is only useful if assets are defined within a Virtual View that has been selected from the Views Path.)
Asset Filter - used in conjunction with Asset, filters out the asset names and their tags to be published
Flush Interval - frequency at which the Publisher pushes out updates. (Note: If the user desires to have every update published, the flush interval should be set lower than the rate at which a tag is logged. The Publisher only pushes the last value in the interval.)
Keep Alive Interval - frequency at which the Publisher will call the websocket host to ensure the connection is still active. If the connection is very reliable, this can be set to 00:00:00.
Reconnect Interval - frequency at which the Publisher attempts to connect to the websocket host if there is a disconnect
Max Frame Size (bytes) - limits the amount of data that can be sent in each message
Use Client Certificate - used when making a secure connection (wss://machineName:portNumber/). Certificates can be managed on the Settings screen.

Settings Screen

Endpoints

gRPC Admin (55360) - endpoint used specifically by the local Admin service for configuring settings and requesting status information

Settings

SERVICE SETTINGS

Delayed Start Seconds - a general delay from when the service starts until data is actually published
Live Data Check Interval - frequency at which the Publisher service checks for new data values within the Views service
Views Change Check Interval - the frequency at which the Publisher service checks for changes in the Views service as it relates to new tags or assets
Durable stream tag save interval - (MQTT only) frequency at which the Publisher writes the status of the tags sent and their value to disk. If the connection is lost or a machine goes down, the Publisher can know where it left off when it resumes again. With an operation publishing 100k tags or more, the user must ask the question: How frequently do I want to save the status of the tags being published? The user must find the balance between duplicate data being published and system performance.
Filter Views Time Extensions - if enabled, prevents the Publisher from publishing the time-extended values that occur approximately once a minute within the Historian for tags that are being actively logged but have not received a new update within the last minute.

IDENTITY

Identity API Token - token used to authorize tag access when the Publisher is remote from the Views service. This token needs mapped to a Canary user who would need placed on the Views>Bypass Security access control list within the Identity tile>Security>Access Control Lists screen.

Certificate (Client)

The Publisher service will use this certificate when making a secure connection to an MQTT broker or websocket. To enable a secure connection, the user will need to ensure Enable SSL/TLS or Use Client Certificate is checked from the Configuration screen.

Kind - the kind of certificate the Publisher service is configured to use
- SelfSignedCertificate - this is the default certificate that Canary uses. It is stored in the Local Computer\Personal store.
- Certificate - use this option if wishing to upload your own certificate. Doing so will present more options to configure.
  - Store Name - local computer store name where the certificate is installed
  - Find Type - criteria by which the Publisher service searches for the certificate
    - FindBySubjectName
    - FindByThumbprint
    - FindByTemplateName
Subject Name - set to '[machineName]' if using the SelfSignedCertificate option. Otherwise, the value of the Subject, Thumbprint, or Template name if using the Certificate setting.