0

Best way for back populating data

Forum / Questions & Answers

I have an existing dataset that is being populated via the SQL Collector. On the 30th Oct I added a number of new tags with "TVQ's" going back to 29th September. All these rows got added to the HDB2 file for the 30th Oct. I now have older data for those tags from earlier than the 29th September that I'd like to get into that dataset.  What is the best way to do so? given that if I did it today I'd have ~7 million TVQs going from April to 28th Sept in a Nov 3rd HDB2 file.

3 replies

null
    • smason
    • 1 yr ago
    • Reported - view

    Hi brian. ingham ,

    Ideally, if you can queue all of the data up again, I'd recommend removing the recent data files within the DataSet then running the data back in chronologically with the 7 million TVQ's. That way the historian is not having to do any inserts. It would be appending all of the data. You would have to do this will all of the data, including the tags that existed in the DataSet beforehand.

      • brian_ingham
      • 1 yr ago
      • Reported - view

      Thanks. Given that I have a smaller tag count licence for test/dev, if I do it on this server to avoid touching the production historian, can the HDB files be copied across if all the naming is identical? 

      • smason
      • 1 yr ago
      • Reported - view

      I think there will be an issue with that because when you try to insert the data file(s) from the dev server, it will only contain the new tags. It will not have the original tags that were currently being logged. So there would be a break in the files of when the original tags drop out of the files, then come back at a later point. This could potentially cause some issues depending on how big that gap is.

      Another option which will be available in 23.2 (maybe 2 weeks out at this time) where you can "link" tags from one dataset to another so the history looks like it coming from one tag. You essentially create a mapping from one tag to the next even though they may reside in different datasets. One of the caveats is that the historical data must fall before the current data, no in between. In other words, you couldn't use this method to fill a gap, it's mainly used for appending historical data which is what you would be doing. It's okay if the two sources of data overlap. I believe you will be able to choose which dataset has precedence in the overlap.

Login to reply

Content aside

print this pagePrint this page
  • 1 yr agoLast active
  • 3Replies
  • 84Views
  • 2 Following