Data Seeding via Canary Sender API
Is it possible to use the API to seed historical data from another historian? Following best practice, the API would write a different dataset but I have found when trying to use the Write API it will put the files in the current date, not the historical date. So if I use the create new file call Canary will also create a value with the current date so you are unable to write any previous dates. i.e. if I call storeData for 2017-08-16 in a file with that date, the historian will then create a single tag value on 2024-04-10 preventing me from storing data with the date 2017-08-17.
-
Hi hmatuse ,
Unless the tag already has history and you're doing inserts, the Write API does not have the ability to seed data. It stores it in the current file like you said.
-
Steve Mason Thanks Steve! If I import all the data into a single file, lets say 2 years of data, is there a way to breakout that data into days? I noticed the fileRollOver API call, could I use this to create smaller database files after I have imported the data?
-
-
Hi hmatuse ,
I apologize. I need to retract my words. After discussing this with development, you can create historical files with the Sender API and you would use the /createNewFile function. I think the piece you are missing is the Roll-Over option within the historian. This is what is causing that current file to get created. If you go into the Historian>Configuration screen and select the dataset you are writing to, you want to turn the Roll-Over option to "Never". Essentially, you are going to now control when new files are created and when they are rolled over. You would need to use /createNewFile to start, then you would use /fileRollOver thereafter.
Sorry for the confusion.
-
Steve Mason so looking at the API, tell me if I have this order correct. The dataset I am using to seed the data should have Roll-up set to "Never". I should call to create the new file for the dataset with my first date, '2017-08-16", then I should import that days data, call the fileRollOver with '2017-08-17' and import '2017-08-17' data.
When I call fileRollOver it makes the new date file but does add the current date to the end of that file.
I also tried to import all my data at once, so '2017-08-16' '2017-08-17' '2017-08-18', then call the fileRollOver for 08-17, and it does not move the data after 2017-08-16 12:00:00 AM. It does create the file, for 08-17 but again, it puts the current date in the file so I am unable to use it.
What am I missing?
-
hmatuse , your order of operations is correct. I believe it is extending the last known timestamp to a current timestamp provided the Sender session is active. This is similar to what some of our collectors do if a tag is stagnant but is still being actively logged. You can turn this off when you request a Session token. You will want to specify the "extendData" parameter and set it to False. By default, it is set to True.
-
Steve Mason Thank you, that seemed to be the missing piece that I needed!
-
Print this page