Historian Tile (version 23)

Knowledge Base / Version 23 / System Admin Duties / Canary Admin

updated 1 yr ago

The 'Historian' tile contains configuration settings for the DataSets that reside within the historian where the raw data is stored. Each dataset can have its own unique settings. Data is stored, by default, in daily .hdb2 files that are generated at midnight provided the DataSet is being actively written to. The data itself can be viewed by drilling into the DataSet and opening the daily file.

Database Screen (DataSet Level)

HISTORIAN

The 'HISTORIAN' window in the upper left displays overall historian properties and disk space.

DataSet Count - the total number of DataSets within the historian
Licensed Tags - the total number of tags in the historian counted against the license
Updates/Sec - the number of raw updates the historian is receiving per second
Readers - the total number of tags that are actively being read from the historian
Writers - the total number of tags that are actively being written to the historian
HDB Files - the total number of .hdb2 files within all DataSets
Offline Files - the total number of .hdb2 files that are offline within all DataSets
Disk: - the amount of free space on the drive where the historian data is being stored

HISTORIAN OPTIONS

By selecting the white down-arrow, the user has the option of importing properties across multiple DataSets. For information on how to import properties see How to Import Custom Tag Properties.

DATASET

The 'DATASET' window in the lower left will display properties and settings for the selected DataSet.

Name - the name of the DataSet
Tag Count - the total number of tags in the most current .hdb2 file
Updates/Sec - the number of raw updates the DataSet is receiving
Readers - the total number of tags that are actively being read
Writers - the total number of tags that are actively being written
HDB Files - the total number of .hdb2 files within the DataSet
Last File - the name of the last file in the DataSet. Each file name contains the DataSet name and the date/hour when it was created.
Offline Files - the total number of .hdb2 files that are offline
Directory - the path where the .hdb2 files are stored
Rollover - if logging continues this shows the date/time of the next planned rollover when a new .hdb2 file will be created. If logging is stopped and resumed after the time passes then a new rollover will occur when logging starts.
Validate - True or False, if the background validation process is enabled for the DataSet. Default is True.

DATASET OPTIONS

By selecting the yellow down-arrow, the user is presented with several options for interacting with the selected DataSet.

ROLL-OVER - forces the historian to create a new file to write to
ROLL-UP - forces the historian to combine its daily files into weekly, monthly, or quarterly files if it is configured to do so
OBSOLETE TAGS - prompts the user for a csv file with a list of tags to obsolete (See Obsoleting Tags from a Dataset for more details.)
IMPORT PROPERTIES - prompts the user for a csv file with a list of tags and properties they wish to import into the DataSet (See How to Import Custom Tag Properties for more details.)
DELETE PROPERTY - prompts the user to enter a property they wish to delete from all tags within the DataSet
REFRESH FILE LIST - refreshes the list of .hdb2 files displayed if a file has been copied into or removed from the directory where the file are stored
VALIDATE ALL FILES - performs a validation on all files in the DataSet

DataSet Tiles

A tile is displayed for each DataSet configured in the historian. Additional information is displayed when the DataSet is being logged to and being read from.

Offline Files - displays the number of offline files, only if they exist. Clicking on the text acts as a hyperlink which will open the DataSet and navigate to the first offline file
Writers - the number of tags that are actively being written to the DataSet
Updates - the number of TVQ's/sec being written to the DataSet
Readers - the number of tags that are actively being read from the DataSet

A red exclamation mark (!) indicates the DataSet has not received data in the time configured. (See Configuring an Alert when a DataSet Stops Receiving Data for more details.)

Double-clicking on a tile or selecting the yellow right-arrow when hovering over the tile will allow the user to navigate into the DataSet to view the .hdb2 files.

Database Screen (File Level)

Drilling into a DataSet, the user can see the individual hdb2 files. Blue tiles indicate the file is online. Offline files are gray. By default, the historian creates a daily file to write to. These files are created at midnight when the historian does its "roll-over" for all active DataSets.

A file with a "clock" icon indicates the file has been modified since it was last validated. These files will eventually be validated by the historian's background validation process. The current file will always have this icon if it is being actively written to.

A file with a question mark "?" indicates the file is questionable as it may not have been closed properly when the historian service shutdown. As such, the background validation process will evaluate this file to determine if there is any corruption and needs to be taken offline.

DATASET & DATASET OPTIONS

The metrics displayed in the 'DATASET' window are the same as those displayed at the DataSet level when a particular DataSet is selected. See the DATASET section above for more details.

Likewise, the user is presented with the same 'DATASET OPTIONS' as described previously when selecting the white down-arrow.

FILE

The 'FILE' window displays properties and settings for the selected file.

Created - the creation date/time of the file
Validated - if True, the date/time when the file was validated; otherwise, False
Tag Count - the number of tags in the file
File Path - the full directory path of the file
File Size - the size of the file on disk
Embedded Size - files open for writing will be slightly larger due to allocated space. This will display the size of the file with the extra space released.
Rolled Over - True or False, depending on whether the file rolled over to a new file at its configured time or if a new file was created upon logging starting
Offline - True of False, files can be taken offline manually or by background validation if an error is detected. They appear as gray tiles.
Status - current state of the file

FILE OPTIONS

By selecting the yellow-down arrow, the user is presented with a few options to perform upon the selected file. These options vary depending if the file is online or not. If the file is online the user can...

VALIDATE - manually runs the validation process for the selected file(s) checking to see if there are any errors. (The background validation does this automatically with any files that have been modified since it was last validated.)
TAKE OFFLINE - takes the file offline
RENAME TAGS - prompts the user for a CSV file containing the current tagname and the new tagname they wish to rename it to. (See How to Rename Tags in a DataSet for more details.)

If the file is offline the user can...

VALIDATE - manually runs the validation process for the selected file(s) checking to see if there are any errors
PUT ONLINE - moves the file online
RECOVER - runs a recovery process in an attempt to recover any unusable data and correct any possible data corruption
DELETE - permanently deletes the file and its data contents. A new popup menu is presented to the user to confirm deletion.

Database Screen (Tag Level)

Drilling into an hdb2 file, the user is presented with the list of alphabetized tags contained within the file. Selecting a tag from the list will allow the user to see its individual TVQ's (timestamp value quality) in the grid on the right and its properties (if any) in the PROPERTIES window in the bottom-left.

The tag list can be filtered with the search bar using keywords (case insensitive) separated by a space. Keywords can be excluded from the search by prepending them with an exclamation mark (!). This list can be exported to a CSV file using the down-arrow button to the right of the search bar. The user will be presented with an option to include the parent path which will prepend the DataSet name to the tags if enabled.

When viewing a tag's data, the most recent TVQ is at the bottom. The user can scroll up through the tag's history with the vertical scroll bar on the right or the horizontal scroll bar at the top. The horizontal scroll bar is divided into vectors allowing the user to jump to various points in history throughout the file.

Canary logs data by exception whenever there is a value or quality change; therefore, it is not expected to see a TVQ every second/minute (depending on the collector's sample rate).

If a tag does not change frequently the historian will time-extend the last value approximately once a minute if it is actively logging. This can be seen by looking at the last two rows in the table. The values will be the same but the last one will have a current timestamp. The historian will NOT time-extend bad quality values.

TREND -opens a new screen displaying a basic trend of the tag for the current file
REFRESH - retrieves any new updates that may have arrived since selecting the tag. Only available when viewing the most recent file.

TAG OPTIONS

Selecting the down-arrow in the PROPERTIES window, the user has the option to rename a tag or delete one of its properties.

RENAME - opens a new dialogue menu allowing the user to rename the tag. The user has the option of renaming the tag in the...
- entire DataSet
- current file and all older files
- current file and all newer files
- current file
DELETE PROPERTY - opens a new dialogue menu allowing the user to delete a single property of the tag. The property name must match exactly including case and whitespace. A property cannot be deleted if the tag is being actively logged.

Diagnostics Screen

The 'Diagnostics' screen displays information about the historian and the server host it is running on. The term TVQ is used extensively in our documentation. It refers to an update containing a Timestamp, Value and Quality.

GENERAL

CPU System Usage - this is a System counter connected to Processor(Total)\% Processor Time. This % Processor Time (_Total) performance counter under the Process performance object provides overall utilization of all processors.
CPU Historian Usage - CPU usage is calculated by getting the total amount of time the system has operated since the last measurement (made up of kernel + user) and the total amount of time the process has run (kernel + user).
The Enterprise Historian is a multi-threaded application and with multi-core PC's, the Historian processing will be distributed across multiple CPUs.
CPU Historian Peak - indicates the highest CPU usage, since the start of the Historian service.
Write Client Count - number of clients connected for writing data into the Historian. These clients are identified on the left under the 'CLIENTS' panel.
Write Handle Count - the number of tags that are being written to by clients at the current time.
Write TVQ/Sec - the number of TVQs that have been sent to the historian by write clients. It indicates the current writing throughput of the historian.
Write Request/Sec - a count of write requests made by external clients. One write request will contain multiple TVQs.
Write Extension/Sec - if an update is not received for a tag within 60 seconds and the Receiver session is valid the Historian will extend the timestamp of the previous value
Read Client Count - clients making read requests to the Historian. Some clients remain connected while others (i.e. Excel Add-In) will disconnect as soon as request is satisfied.
Read Handle Count - the number of tags being read by clients at the current time
Read TVQ/Sec - the total number of TVQs being read for external client applications
Read Request/Sec (Historical) - a count of read requests for historical data by external client applications. One read request will contain multiple TVQs.
Read Request/Sec (Live) - a count of read requests for live data by external client applications. One read request will contain multiple TVQs.
Historian Admin Request/Sec - requests for historian status information. For example, statistics running at the bottom of the Diagnostics (Up Time, DataSets, Total Tags, etc...).
Memory Physical Total - total physical memory installed on the machine.
Memory Physical Usage - the historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of Physical Memory currently being used.
Memory Physical Peak - indicates the highest usage, since the start of the Historian service
Memory Page Usage - the historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of the Page File currently being used.
Memory Virtual Usage - the historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of the Virtual Memory current being used.
Historian Working Set - this is a system counter connected to \Process(CLIHistorian)\Working Set
Historian Thread Count - amount of OS threads in the Historian
Historian Handle Count - amount of OS handles used by the Historian

BACKGROUND

Indexing - (obsolete)
Roll-Up - this field will show the current DataSet and file being Rolled-Up as it progresses. The Roll-Up runs a few seconds after the Roll-Over of a DataSet or manually from the Arrow Down on the DataSet panel.
Validation - checks each DataSet for files to be automatically validated. If a file needs validated this field will show the DataSet and filename as it progresses. The global validation can be enabled/disabled for all DataSets from Historian > Configuration > Settings screen. Individual DataSet validation can be controlled by selecting the DataSet from Historian > Configuration>DataSets screen.

ACTIVITY

Instant - An instantaneous count of TVQ writes and reads.
Minute - TVQ write and reads over the past minute.
Hour - TVQ write and reads over the past hour.
Day - TVQ write and reads for the past day.
Month - TVQ write and reads for the past month.
Total - TVQ write and reads since the Historian was installed on the machine.

CLIENTS

This panel allows the user to monitor any clients that are logging data into the Historian or clients that are reading data from it. The Receiver Service is the primary logging client and viewer clients will vary depending on how the data is needed.

Writer - displays the number of tags being written and the identity of the client. "HistDiagnostics" will always show as a writer.
Reader - displays the number of tags requesting data and the identity of the client

STATUS BAR

The gray status bar at the bottom of the screen displays the following parameters:

Start Time - time when the Historian was last started
Up Time - length of time the Historian has been running since the last start
Run Time - cumulative amount of time the Historian has run
DataSets - the total number of DataSets defined (includes {Diagnostics})
Licensed Tags - the total number of tags from all DataSets being counted against the Historian license (excludes the {Diagnostics} DataSet). The actual historian license, tag limit is shown in the Licenses tile.
Total Tags - the total number of tags from all DataSets (includes {Diagnostics})
Files - total number of files in all defined DataSets (excludes the {Diagnostics} DataSet)
Version - the version of the Historian where the Canary Admin is connected

Configuration Screen

DataSets

This panel shows all DataSets defined in the Historian and provides creation and removal options.

NEW - displays fields to define the DataSet name and primary path. DataSet names must be more than 1 character in length. Once a DataSet name and path is provided, click the 'CREATE' button. (Note: With the exception of the OPC DA Collector, a DataSet does NOT need pre-created prior to logging to it.)
SCAN - allows the user to select a path where the Historian will look for .hdb2 files and automatically create the DataSet accordingly. After entering a Path, click 'SCAN' to search the directory and if 'Include Subdirectories' is checked, any subdirectories.
RENAME - allows the user to rename a DataSet name
REMOVE- allows the user to remove a DataSet. Clicking the button will display a confirmation dialog with the option to "Delete All Associated Historical Data" if checked. Click the OK button to remove the DataSet.

Note: DataSet must be closed (no writers or readers) before the Historian will rename or remove it.

SETTINGS

This panel provides file management settings for the selected DataSet.

DataSet Directory - directory location where the .hdb2 files will be stored. This path can only be set during DataSet creation. Changing it would require removing the DataSet and then doing a Scan if .hdb2 files already exist or New if a new location.
Validate - when checked, a background thread will validate any files for this DataSet not already validated, except the current file
Email if no data received in ?? minutes - this requires SMTP Server and email addresses to be setup under Messages > Configuration > Email (See Configuring the Canary System to Send Email Alerts). If no data is received for any tag in this DataSet within the set minutes an email will be sent to the email recipients.
Notify Email - semicolon-delimited list of recipients to receive an email if no data is received in ?? minutes
Roll-Over - aside from physical disk corruption, the only time a file is susceptible to corruption is when it is opened for writing. Should something happen during one of these writes such as a power failure or machine reset without a controlled shutdown, the file could become unusable. This information should be considered when deciding when to Roll-Over to a new file. For instance, creating a file every day at midnight would only risk the data from the time of the Roll-Over to when the system failure occurred, limiting the possible loss to only a few hours.
- Never - (Very rare) all data will be written into one continuous file
- Daily - (default) a new file will be created every day at hh:mm (hours:minutes) after midnight.
- Weekly - a new file will be created at hh:mm after midnight on selected day of the week
- Hourly - a new file will be created starting at 00:00 - 24:00 (24 hour time) every 2, 3, 4, 6, 8 or 12 Hours. This is to accommodate 1 file per shift.
- Custom - a new file will be created every 0-100, Minutes, Hours or Days.
- Size - a new file is created when the current file exceeds 10 - 500 megabytes.
Roll-Up - Roll-Up is a background operation to take files at the Roll-Over rate and combine them into Daily, Weekly, Monthly or Quarterly files. Once rolled up, the files can no longer be converted back to their original roll-over size. This option can be most valuable to customers with large quantities of tags as the tag information is duplicated in each file. If the existing file and the next file to be rolled up exceed 500 MB, the operation will not start and a 2nd Roll-Up file will begin for the time period. Roll-Up does not roll up the last file in the DataSet.
Delete - Files with a creation time older than 1 - 365 Days, Months or Years will be deleted at the time when a Roll-Over occurs. Never will keep all files.
Move deleted files - moves the deleted files to another location where they are no longer visible to the Historian. (See How to Move Historical Data to a Temp Directory for Cold Storage)

Settings

Background Validation - Checks each DataSet for files to be automatically validated. The global validation can be disabled for all DataSets by unchecking this box.
Deleted Files Directory - the location where deleted files will be stored if a DataSet is configured to "Move deleted files" (See How to Move Historical Data to a Temp Directory for Cold Storage)
External Property Storage - enables the Views service to look for tag properties in an external SQLite or MSSQL database (See How to Configure External Property Storage)
Server - the name of the server where the SQL database is located
Database - the name of the database where the tag properties are stored
Is Domain User - verifies the specified user is a domain user. If unchecked, the specified user is assumed to be a local SQL account.

Endpoints

The 'Endpoints' screen is used to configure a Canary system in which the historian needs added to a remote Views service (rare). A "Remote Historian" view can be added to the Views service by opening the Canary Admin>Views tile>Configuration>Historians on the server where the Views service resides. The Views service makes a gRPC connection to either the anonymous (55281) or secure (55282) endpoint.

The configured 'CERTIFICATE' will be use for making a secure connection.

Store Name - the file location of the client certificate within the machine's certificate store
Find Type - criteria by which the Historian searches for the certificate
- FindBySubjectName
- FindByThumbprint
- FindByTemplateName
Subject Name - the value of the Subject Name or Thumbprint

Access

If a remote Views service is making a secure connection to the historian, the server name and unique GUID of the Views client will initially appear on the 'DENY' list and will need manually allowed before the Historian can be added to the Views service.

DENY - places the selected Views client on the 'DENY' list
ALLOW - places the selected Views client on the 'ALLOW' list
REMOVE - removes the selected Views client from the 'DENY' list
REFRESH - refreshes the 'DENY' list of any new Views clients that may have appeared since the screen has opened