How to turn off Compression on SQL Collector Tags.
How to turn off Compression on SQL Collector Tags.
I want all values from my SQL Collector.
Example this happed today
Timestamp Value
2025-03-10 11:41:55.978 31.79
2025-03-10 11:25:34.045 31.79
The 2025-03-10 11:41:55.978 is not stored. I had to send it manually after realizing that PostgreSQL and Canary had different summary values.
Note: I am new to Canary, I had sent this on Canary Historian Storage Methodology feedback, I guess this should be a better place.
4 replies
-
Hi ,
Welcome to the Canary Community! Thanks for the question.
By default, the Historian stores by exception whenever there is a value or quality change. This is a way for us to limit the amount of disk space we need to store the data.
If you want to store every timestamp regardless of value or quality change, you'll need to modify a DataSet setting within the C:\ProgramData\Canary\Historian\historian.service.json file.
- Stop the Historian service.
- Find the DataSet within the file above and set the "IsStoreEveryTvqEnabled" parameter to true.
- Start the Historian service.
It is important to note that this functionality does not work with all Collector types as some Collectors log by exception and do not send every timestamp. The SQL Collector is an exception to this in that it does send every timestamp it scrapes from the table.
-
Thanks.
Yes, I understand the store by exception concept, but for lab or other manual inputs, and in this case truck scale data I need all inputs stored to get total weights and trucks to calculate production. What does IsSoftDeleteEnabled do. Also, what is the best way to delete errors in manual input data.
Thanks.
-
The soft delete is the functionality where the Historian doesn't actually delete old hdb files, but moves them to another directory for backup or cold storage.
https://helpcenter.canarylabs.com/t/60yfyf2/how-to-move-historical-data-to-a-temp-directory-for-cold-storage-version-25
As for deleting errors, there is no method for deleting individual records. You would either have to- rename and obsolete the tag, then start over with a new tag which would require you to input ALL of the data with the correct values
- delete the file(s) that contain the errors, but this would also remove all of the data for other tags that reside in that file
- or, overwrite the errors with the correct value using the same method of manual data entry. A timestamp can only be associated with one value, so if you use the exact same timestamp as the erroneous value, it will replace the old value with the new one. This would probably be the easiest method and least "destructive". If you simply want to get rid of the value, you could overwrite it with whatever the previous value was.
Hope this helps
-
@Steve Mason
That would not work for me.
I have worked with another Historian from 1995 which allows the deletion of tag values. For the cost I consider Canary an excellent alternative so far, that is why I recommend it to my current customer. It seems a bit strange that no other customer requires deleting values from Laboratory and inputs which requires event counts and totals. What I have done is to send a bad quality with a value of zero to replace the error, but it shows up as bad if it is current value and total values are highlighted as uncertain, but totals are correct.
Suggestion, why not have a quality that can flag values as deleted. When Canary applications see it, it ignores it (no gaps in trends and uncertain to summaries) apart from when viewing in Canary Admin which will have it flagged as deleted.
With this I would give Canary 100%.
Thanks for answering.
Print this page