Historian Tile (version 22)

The 'Historian' tile contains configuration settings for the datasets that reside within the historian where the raw data is stored. Each dataset can have its own unique settings. Data is stored, by default, in daily .hdb2 files that are generated at midnight provided the dataset is being actively written to. The data itself can be viewed by drilling into the dataset and opening the daily file.

Database Screen (DataSet Level)

HISTORIAN

The 'HISTORIAN' window in the upper left displays overall historian properties and disk space.

• DataSet Count - the total number of datasets within the historian
• Licensed Tags - the total number of tags in the historian counted against the license
• Updates/Sec - the number of raw updates the historian is receiving per second
• Readers - the total number of tags that are actively being read from the historian
• Writers - the total number of tags that are actively being written to the historian
• HDB Files - the total number of .hdb2 files within all DataSets
• Offline Files - the total number of .hdb2 files that are offline within all DataSets
• Disk: - the amount of free space on the drive where the historian data is being stored

HISTORIAN OPTIONS

By selecting the white down-arrow, the user has the option of importing properties across multiple DataSets. For information on how to import properties see How to Import Custom Tag Properties.

DATASET

The 'DATASET' window in the lower left will display properties and settings for the selected DataSet.

• Name - the name of the DataSet
• Tag Count - the total number of tags in the most current .hdb2 file
• Updates/Sec - the number of raw updates the DataSet is receiving
• Readers - the total number of tags that are actively being read
• Writers - the total number of tags that are actively being written
• HDB Files - the total number of .hdb2 files within the DataSet
• Last File - the name of the last file in the DataSet. Each file name contains the DataSet name and the date/hour when it was created.
• Offline Files - the total number of .hdb2 files that are offline
• Directory - the path where the .hdb2 files are stored
• Rollover - if logging continues this shows the date/time of the next planned rollover when a new .hdb2 file will be created. If logging is stopped and resumed after the time passes then a new rollover will occur when logging starts.
• Validate - True or False, if the background validation process is enabled for the DataSet. Default is True.

DATASET OPTIONS

By selecting the yellow down-arrow, the user is presented with several options for interacting with the selected DataSet.

• ROLL-OVER - forces the historian to create a new file to write to
• ROLL-UP - forces the historian to combine its daily files into weekly, monthly, or quarterly files if it is configured to do so
• OBSOLETE TAGS - prompts the user for a csv file with a list of tags to obsolete (See Obsoleting Tags from a Dataset for more details.)
• IMPORT PROPERTIES - prompts the user for a csv file with a list of tags and properties they wish to import into the DataSet (See How to Import Custom Tag Properties for more details.)
• DELETE PROPERTY - prompts the user to enter a property they wish to delete from the tags within the DataSet
• REFRESH FILE LIST - refreshes the list of .hdb2 files displayed if a file has been copied into or removed from the directory where the file are stored
• VALIDATE ALL FILES - performs a validation on all files in the DataSet

DataSet Tiles

A tile is displayed for each DataSet configured in the historian. Additional information is displayed when the DataSet is being logged to and being read from.

• Offline Files - displays the number of offline files, only if they exist. Clicking on the text acts as a hyperlink which will open the DataSet and navigate to the first offline file
• Writers - the number of tags that are actively being written to the DataSet
• Updates - the number of TVQ's/sec being written to the DataSet
• Readers - the number of tags that are actively being read from the DataSet

A red exclamation mark (!) indicates the DataSet has not received data in the time configured. (See Configuring an Alert when a DataSet Stops Receiving Data for more details.)

Double-clicking on a tile or selecting the yellow right-arrow when hovering over the tile will allow the user to navigate into the DataSet to view the .hdb2 files.

Database Screen (File Level)

DATASET & DATASET OPTIONS

The metrics displayed in the 'DATASET' window are the same as those displayed at the DataSet level when a particular DataSet is selected. See the DATASET section above for more details.

Likewise, the user is presented with the same 'DATASET OPTIONS' as described previously when selecting the white down-arrow.

FILE

The 'FILE' window displays properties and settings for the selected file.

• Created - the creation date/time of the file
• Validated - if True, the date/time when the file was validated; otherwise, False
• Tag Count - the number of tags in the file
• File Path - the full directory path of the file
• File Size - the size of the file on disk
• Embedded Size - files open for writing will be slightly larger due to allocated space. This will display the size of the file with the extra space released.
• Rolled Over - True or False, depending on whether the file rolled over to a new file at its configured time or if a new file was created upon logging starting
• Offline - True of False, files can be taken offline manually or by background validation if an error is detected. They appear as gray tiles.
• Status - current state of the file

FILE OPTIONS  

By selecting the yellow-down arrow, the user is presented with a few options to perform upon the selected file. These options vary depending if the file is online or not. If the file is online the user can...

• VALIDATE - manually runs the validation process for the selected file(s) checking to see if there are any errors. (The background validation does this automatically with any files that have been modified since it was last validated.)
• TAKE OFFLINE - takes the file offline
• RENAME TAGS - prompts the user for a CSV file containing the current tagname and the new tagname they wish to rename it to. (See How to Rename Tags in a DataSet for more details.)

If the file is offline the user can...

• VALIDATE - manually runs the validation process for the selected file(s) checking to see if there are any errors
• PUT ONLINE - moves the file online
• RECOVER - runs a recovery process in an attempt to recover any unusable data and correct any possible data corruption
• DELETE - permanently deletes the file and its data contents. A new popup menu is presented to the user to confirm deletion.

Diagnostics Screen

The 'Diagnostics' screen displays information about the historian and the server host it is running on. The term TVQ is used extensively in our documentation. It refers to an update containing a Timestamp, Value and Quality.

GENERAL

• CPU System Usage - this is a System counter connected to Processor(Total)\% Processor Time. This % Processor Time (_Total) performance counter under the Process performance object provides overall utilization of all processors.
• CPU Historian Usage - CPU usage is calculated by getting the total amount of time the system has operated since the last measurement (made up of kernel + user) and the total amount of time the process has run (kernel + user).
• The Enterprise Historian is a multi-threaded application and with multi-core PC's, the Historian processing will be distributed across multiple CPUs.
• CPU Historian Peak - indicates the highest CPU usage, since the start of the Historian service.
• Write Client Count - number of clients connected for writing data into the Historian. These clients are identified on the left under the 'CLIENTS' panel.
• Write Handle Count - the number of tags that are being written to by clients at the current time.
• Write TVQ/Sec - the number of TVQs that have been sent to the historian by write clients. It indicates the current writing throughput of the historian.
• Write Request/Sec - a count of write requests made by external clients. One write request will contain multiple TVQs.
• Write Extension/Sec - if an update is not received for a tag within 60 seconds and the Receiver session is valid the Historian will extend the timestamp of the previous value
• Read Client Count - clients making read requests to the Historian. Some clients remain connected while others (i.e. Excel Add-In) will disconnect as soon as request is satisfied.
• Read Handle Count - the number of tags being read by clients at the current time
• Read TVQ/Sec - the total number of TVQs being read for external client applications
• Read Request/Sec (Historical) -  a count of read requests for historical data by external client applications. One read request will contain multiple TVQs.
• Read Request/Sec (Live) - a count of read requests for live data by external client applications. One read request will contain multiple TVQs.
• Historian Admin Request/Sec - requests for historian status information. For example, statistics running at the bottom of the Diagnostics (Up Time, DataSets, Total Tags, etc...).
• Memory Physical Total - total physical memory installed on the machine.
• Memory Physical Usage - the historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of Physical Memory currently being used.
• Memory Physical Peak - indicates the highest usage, since the start of the Historian service
• Memory Page Usage - the historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of the Page File currently being used.
• Memory Virtual Usage - the historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of the Virtual Memory current being used.
• Historian Working Set - this is a system counter connected to \Process(CLIHistorian)\Working Set
• Historian Thread Count - amount of OS threads in the Historian
• Historian Handle Count - amount of OS handles used by the Historian

BACKGROUND

• Indexing - (obsolete)
• Roll-Up - this field will show the current DataSet and file being Rolled-Up as it progresses. The Roll-Up runs a few seconds after the Roll-Over of a DataSet or manually from the Arrow Down on the DataSet panel.
• Validation - checks each DataSet for files to be automatically validated. If a file needs validated this field will show the DataSet and filename as it progresses. The global validation can be enabled/disabled for all DataSets from Historian > Configuration > Settings screen. Individual DataSet validation can be controlled by selecting the DataSet from Historian > Configuration>DataSets screen.

ACTIVITY

• Instant - An instantaneous count of TVQ writes and reads.
• Minute - TVQ write and reads over the past minute.
• Hour - TVQ write and reads over the past hour.
• Day - TVQ write and reads for the past day.
• Month - TVQ write and reads for the past month.
• Total - TVQ write and reads since the Historian was installed on the machine.

CLIENTS

This panel allows the user to monitor any clients that are logging data into the Historian or clients that are reading data from it. The Receiver Service is the primary logging client and viewer clients will vary depending on how the data is needed.

• Writer - displays the number of tags being written and the identity of the client. "HistDiagnostics" will always show as a writer.
• Reader - displays the number of tags requesting data and the identity of the client

STATUS BAR

The gray status bar at the bottom of the screen displays the following parameters:

• Start Time - time when the Historian was last started
• Up Time - length of time the Historian has been running since the last start
• Run Time - cumulative amount of time the Historian has run
• DataSets - the total number of DataSets defined (includes {Diagnostics})
• Licensed Tags - the total number of tags from all DataSets being counted against the Historian license (excludes the {Diagnostics} DataSet). The actual historian license, tag limit is shown in the Licenses tile.
• Total Tags - the total number of tags from all DataSets (includes {Diagnostics})
• Files - total number of files in all defined DataSets (excludes the {Diagnostics} DataSet)
• Version - the version of the Historian where the Canary Admin is connected

Configuration Screen

DataSets

This panel shows all DataSets defined in the Historian and provides creation and removal options.

• NEW - displays fields to define the DataSet name and primary path. DataSet names must be more than 1 character in length. Once a DataSet name and path is provided, click the 'CREATE' button. (Note: With the exception of the OPC DA Collector, a DataSet does NOT need pre-created prior to logging to it.)
• SCAN - allows the user to select a path where the Historian will look for .hdb2 files and automatically create the DataSet accordingly. After entering a Path, click 'SCAN' to search the directory and if 'Include Subdirectories' is checked, any subdirectories.  
• RENAME - allows the user to rename a DataSet name  
• REMOVE- allows the user to remove a DataSet. Clicking the button will display a confirmation dialog with the option to "Delete All Associated Historical Data" if checked. Click the OK button to remove the DataSet.  

Note: DataSet must be closed (no writers or readers) before the Historian will rename or remove it.

SETTINGS

This panel provides file management settings for the selected DataSet.

• DataSet Directory - directory location where the .hdb2 files will be stored. This path can only be set during DataSet creation. Changing it would require removing the DataSet and then doing a Scan if .hdb2 files already exist or New if a new location.
• Validate - when checked, a background thread will validate any files for this DataSet not already validated, except the current file
• Email if no data received in ?? minutes - this requires SMTP Server and email addresses to be setup under Messages > Configuration > Email (See Configuring the Canary System to Send Email Alerts). If no data is received for any tag in this DataSet within the set minutes an email will be sent to the email recipients.
• Notify Email - semicolon-delimited list of recipients to receive an email if no data is received in ?? minutes
• Roll-Over - aside from physical disk corruption, the only time a file is susceptible to corruption is when it is opened for writing. Should something happen during one of these writes such as a power failure or machine reset without a controlled shutdown, the file could become unusable. This information should be considered when deciding when to Roll-Over to a new file. For instance, creating a file every day at midnight would only risk the data from the time of the Roll-Over to when the system failure occurred, limiting the possible loss to only a few hours.
  • Never - (Very rare) all data will be written into one continuous file
  • Daily - (default) a new file will be created every day at hh:mm (hours:minutes) after midnight.
  • Weekly - a new file will be created at hh:mm after midnight on selected day of the week
  • Hourly - a new file will be created starting at 00:00 - 24:00 (24 hour time) every 2, 3, 4, 6, 8 or 12 Hours. This is to accommodate 1 file per shift.
  • Custom - a new file will be created every 0-100, Minutes, Hours or Days.
  • Size - a new file is created when the current file exceeds 10 - 500 megabytes.
• Roll-Up - Roll-Up is a background operation to take files at the Roll-Over rate and combine them into Daily, Weekly, Monthly or Quarterly files. Once rolled up, the files can no longer be converted back to their original roll-over size. This option can be most valuable to customers with large quantities of tags as the tag information is duplicated in each file. If the existing file and the next file to be rolled up exceed 500 MB, the operation will not start and a 2nd Roll-Up file will begin for the time period. Roll-Up does not roll up the last file in the DataSet.
• Delete - Files with a creation time older than 1 - 365 Days, Months or Years will be deleted at the time when a Roll-Over occurs. Never will keep all files.
• Move deleted files - moves the deleted files to another location where they are no longer visible to the Historian. (See How to Move Historical Data to a Temp Directory for Cold Storage)

Settings

• Background Validation - Checks each DataSet for files to be automatically validated. The global validation can be disabled for all DataSets by unchecking this box.
• Deleted Files Directory - the location where deleted files will be stored if a DataSet is configured to "Move deleted files" (See How to Move Historical Data to a Temp Directory for Cold Storage)
• External Property Storage - enables the Views service to look for tag properties in an external SQLite or MSSQL database (See How to Configure External Property Storage)
• Server - the name of the server where the SQL database is located
• Database - the name of the database where the tag properties are stored
• Is Domain User - verifies the specified user is a domain user. If unchecked, the specified user is assumed to be a local SQL account.

Endpoints

The 'Endpoints' screen is used to configure a Canary system in which the historian needs added to a remote Views service (rare). A "Remote Historian" view can be added to the Views service by opening the Canary Admin>Views tile>Configuration>Historians on the server where the Views service resides. The Views service makes a gRPC connection to either the anonymous (55281) or secure (55282) endpoint.

The configured 'CERTIFICATE' will be use for making a secure connection.

• Store Name - the file location of the client certificate within the machine's certificate store
• Find Type - criteria by which the Historian searches for the certificate
  • FindBySubjectName
  • FindByThumbprint
• Subject Name - the value of the Subject Name or Thumbprint

Access

If a remote Views service is making a secure connection to the historian, the server name and unique GUID of the Views client will initially appear on the 'DENY' list and will need manually allowed before the Historian can be added to the Views service.

• DENY - places the selected Views client on the 'DENY' list
• ALLOW - places the selected Views client on the 'ALLOW' list
• REMOVE - removes the selected Views client from the 'DENY' list
• REFRESH - refreshes the 'DENY' list of any new Views clients that may have appeared since the screen has opened