Enable MQTT Collector to Auto-Log Tags Matching Pattern
Please add a way to set one or more filters for an MQTT Collector Subscription to automatically collect tags which pass at least one of the filters.
The "automatically log all tags" setting for an MQTT Collector Subscription isn't as useful as it could be because logging all tags from an Ignition gateway would result in a large volume of "junk" tags that don't need to be stored with history in Canary.
I'd like to set filters instead for the auto-logger similar to filtering the tag list based on the Published Metric Name (either "Contains" or Regex) and have the collector automatically log only tags which pass at least one of the filters. For example, I'd like to collect all "/PV" and "/Day 1" values, but I don't want to automatically log other tags into Canary like alarms or Day 2 values. Currently, we have to manually update the tag collection when we add new tags that need collecting, but having filters for the auto-logger would automate that process.
-
Hi bmhargis ,
Is this what you're looking for: https://helpcenter.canarylabs.com/t/h7hvlzk/mqtt-collector-overview-version-23#filters ?
-
Steve Mason That looks like it! Can't believe I didn't know about this feature
-
bmhargis It's not used very often, but glad we have it for you! It works in tandem with automatically logging all tags. It uses the filters to enable/disable the tags.
-
Steve Mason two questions:
1) Do I need "Automatically log all tags" to be checked for filters to apply?
2) For inclusive filters, I have a question on the following:"If any inclusive filters are created, then none of the tags from the topic are included by default."
If I already have tags enabled and then add an inclusive filter, will it disable tags which don't pass the filter that were already enabled?
Similarly, if I have an inclusive filter but manually enable a one-off tag, will I be able to do that? Or will the inclusive filter prevent me from enabling the one-off?
I imagine the filters act as, well, a filter over the auto-log tag feature and thus wouldn't effect already enabled tags, but I want to be sure before applying any. -
bmhargis Steve Mason additional question, if I have an existing config already (tags birthed, some enabled, etc), will a new inclusive filter check through all the existing tags in the config? Will it do it during the next birth message?
-
bmhargis The filters is just another mechanism for enabling tags. The "Automatically log all tags" has to be checked for this to work. With that being said, the tags that already exist in the Tags table would be left untouched. The settings are "sticky". So if you make any configuration changes (i.e. adding filters), it would only apply to new tags that are published. If you want it to apply to everything, you would have to delete the Tags table. This could potentially cause issues if the session is writing to multiple datasets. A tag could rerouted to a different dataset than what it was assigned previously.
Once you have filters applied, you can still go in an enable one-off tags manually. -
Steve Mason thanks, so we'd probably have to run one-time updates against the mqttcollector.sqlite file to reflect the filters we'll add. Shouldn't be too bad with some SQL statements.
This feature (and regular auto-log all) does seem to pose a risk when used with master/backup Canary pairs that have a main dataset split into multiple parts (i.e. -Data and -Data2) because tags automatically enabled in both configs may end up being assigned to different dataset numbers if they are not consistently kept synchronized (which is a manual process).
-
Print this page