Creating, Deleting, and Renaming DataSets (version 23)

Creating a New DataSet

A DataSet is a user-defined grouping of tags. DataSets play a key role in logging, storing, and reading data. Depending on the collector being used, the DataSet may need created before data can be logged.

The OPC DA Collector requires the user to pre-create the DataSet before logging begins.

The SQL Collector requires the user to pre-create the DataSet if defining it in 'Configuration' U/I. If a DataSet is NOT defined, the DataSet can be dynamically created.

The CSV Collector has the option to automatically create the DataSet based upon the <AutoCreateDataset> parameter configured in the SAF_Import.config file. By default, this is set to True.

All other collectors can automatically create the DataSet when logging begins.

To create a Dataset:

1. Open the Canary Administrator application.
2. Select the 'Historian' tile, then 'Configuration', then 'NEW'.
3. Enter the DataSet name. DataSets must have unique names, not start or end with a 'space' and cannot use the following characters:

   Colon	:
   Equals	=
   Forward Slash	/
   Asterisk	*
   Backwards Slash	\
   Percentage Sign	%
   Question Mark	?
   Greater than or equal to	< >

4. By default, all DataSets are stored in 'C:\Historian Data'. In the 'Path' field, type a new path or select 'Browse' to change the directory location of the data for the DataSet.

5. Finish by selecting 'CREATE'.
6. Adjust parameters within the 'SETTINGS' window at right.  Each setting is explained below.

Keep the total number of DataSets below 300 and maintain less than 40,000 tags per DataSet. While there are no hard limits coded into the historian, increasing beyond these numbers may result in performance issues or errors.

Adjusting DataSet Parameters

Path

The 'Path' parameter displays the directory of where the HDB files are stored for the selected DataSet. If wishing to change this location, an admin would need to:

1. Stop the Historian service
2. Move the HDB files to its new desired location
3. Edit the C:\ProgramData\Canary [Labs]\Historian\Historian.ini file and update the 'path' parameter of the particular DataSet with the new location.
4. Start the Historian service

DataSet Validation Configuration

Background validation checks all DataSets for unvalidated HDB files. An unvalidated HDB file is indicated by a black clock in the top left hand corner of the tile as shown on the Dec-22 HDB file below. 

By default, background validation is turned on. The background validation process may consume additional machine resources but ensures that HDB files do not contain incomplete or corrupt data values. 

A file remains unvalidated while it is still open for logging.  Typically files are validated each night as they 'Roll-Over'. It is not recommended to uncheck background validation.

Configure Email Notifications for DataSets Not Logging Data

The Historian can be configured to send an email to the system admin(s) if a DataSet has not received any data in the specified period of time. 

1. Select the 'Email if no data received in' checkbox and configure the number of minutes. 
2. Designate the user(s) to whom the email will be sent to. 
3. Configure the Canary system to send email alerts.

Enabling this feature is a recommended precaution for monitoring the network connection break between the Sender and Receiver, monitoring the status of a logging machine, or ensuring that logging equipment maintains functionality. 

Configuring the Roll-Over Frequency for a DataSet

A Roll-Over occurs when a new HDB file is created. To ensure data integrity, it is recommended to roll-over files daily.

• When an HDB file is open for writing, it could potentially become corrupted; thus the longer an HDB file is open for writing, the greater the potential for data loss.
• Data corruption can be caused by power outages, machine restarts, and other service disruptions. 

1. Under 'Settings' select the checkbox beside 'Daily' 

Setting the Roll-Over to daily is a recommended best practice to take to ensure your data remains secure despite unexpected network failures.  

Configuring the Roll-Up Frequency for a DataSet

The Roll-Up setting determines how often individual HDB files get condensed into one 'rolled-up' file. A 'rolled-up' file is either daily, weekly, monthly, or quarterly depending on how many HDB files the system administrator wishes to condense.

HDB file roll-ups lessen storage space by deleting redundant tag names and metadata property descriptions from multiple HDB files.

• For example, instead of listing Tag0001 in thirty separate HDB files, a monthly roll-up lists Tag0001 in one monthly file while retaining all of Tag0001's TVQs from the last thirty days.

The Roll-Up feature is off by default. It is not necessary, but an Admin may want to condense many HDB files to save  storage space. The Roll-Up feature can be most effective when using large numbers of DataSets with smaller tag counts.

Delete HDB Files after a Set Period of Time

Defaulted to 'Never' this setting can be customized to automatically delete a DataSet's HDB files after a specific period of time. 

1. Set the 'Delete' section to 'Custom'. Adjust the interval and time parameter accordingly.
2. NEVER make this setting less than the Roll-Up time. Doing so will delete the entire Roll-up file that the latest HDB file is stored within.

Moving Old HDB Files Offline

If wishing to "archive" older HDB files, the user can select the 'Custom' radio button then check the box to 'Move deleted files to [filepath]'. This file path is configured in the Settings tab on the left. For more information about moving files see How to Move Historical Data to a Temp Directory for Cold Storage.

Deleting a DataSet

A DataSet can be deleted from the system by following these steps.

1. From the Canary Admin, select the Historian tile.
2. Click 'Configuration'.
3. Highlight the DataSet you wish to remove.
4. Select 'REMOVE' to permanently delete the DataSet.
5. An option to also 'Delete All Associated Historical Data' is presented.  Only check this box if you wish to permanently delete all data associated with this DataSet.  This cannot be undone.
6. Click 'ok'.

Always delete DataSets from the Historian User Interface. Never delete them from the file directory, doing so will create undesired errors and cause the historian to malfunction. 

A DataSet cannot be removed if it is being actively logged to or accessed. If the user is unable to determine which client is accessing the DataSet, they can disable the Receiver (writing) or Views (reading) service. Doing so will cause temporary interruption for any collector logging data and any user trying to access data in the historian.

Importing DataSets and HDB Files

Entire DataSets or HDB files can be imported as necessary. This can serve the purpose of disaster recovery or sharing HDB files with other parts of an organization. 

Importing HDB Files

1. Open the Windows File Explorer and navigate to your DataSet directory, by default: C:\Historian Data\.
2. Place the HDB files you wish to import within the existing DataSet.
3. Open the Canary Admin. Select the 'Historian' tile. Highlight the DataSet with the new HDB files by clicking on it once, thus turning it yellow. 
4. From the 'DATASET' window (bottom left) click the 'down arrow' to open the DataSet menu.
5. Select 'REFRESH FILE LIST'.  This action scans the selected DataSet and automatically discovers all new HDB files.
6. You may now browse the DataSet to confirm the files were imported.

Importing Entire DataSets

1. Open the Windows File Explorer and navigate to your DataSet directory, by default: C:\Historian Data\.
2. Place the DataSet and corresponding HDB files you wish to import within the folder.
3. Open the Canary Admin. Select the 'Historian' tile. Select the 'Configuration' tab. Select 'SCAN'.
4. Select 'BROWSE'. Find and highlight the newly imported DataSet folder. Select 'OK'.
5. Select 'SCAN'.

You can use the 'Include Subdirectories' checkbox to instruct the scan tool to look to all subdirectories within a folder and automatically detect and create multiple DataSets.

Renaming a DataSet

1. Launch the Canary Admin application. Select the 'Historian' tile. Select the 'Configuration' tab.
2. Select a DataSet. Select 'RENAME'. 
3. Enter the new DataSet name in the 'New name' field. DataSets must have unique names, not start or end with a 'space' and cannot use the following characters:

   Colon	:
   Equals	=
   Forward Slash	/
   Asterisk	*
   Backwards Slash	\
   Percentage Sign	%
   Question Mark	?
   Greater than or equal to	< >

4. Select 'ok'. This has now changed the name of all the HDB files within the DataSet.
5. Although it is not necessary, it may be intuitive to change the folder name of where the HDB files are stored so that it matches the DataSet name. To do this, an admin would need to:
   1. Stop the Historian service
   2. Rename the folder in which the HDB files reside
   3. Edit the C:\ProgramData\Canary [Labs]\Historian\Historian.ini file and update the 'path' parameter of the particular DataSet with the new folder name.
   4. Start the Historian service

The best practice is to rename the DataSet before logging data to it. 

DataSets currently in use cannot be renamed or deleted. A DataSet is in use when data is being written to or read from it.  The Canary Views and subsequent client applications and other modules like Axiom, Events, Calculation Server, Excel Add-in, etc all read data from a DataSet. Follow these steps to ensure data is not being written or read.

1. Launch the Canary Admin. Select the 'Services' tile. 
2. 'STOP' the Receiver service.
3. 'STOP' the Views service.
4. Repeat the steps above to rename the DataSet. 
5. Start the Receiver service and the Views service after the renaming the DataSet.