Historian Tile (version 24)

The 'Historian' tile contains configuration settings for the DataSets that reside within the historian where the raw data is stored. Each dataset can have its own unique settings. Data is stored, by default, in daily .hdb3 files that are generated at midnight provided the DataSet is being actively written to. The data itself can be viewed by drilling into the DataSet and opening the daily file.

Database Screen (DataSet Level)

HISTORIAN

The 'HISTORIAN' window in the upper left displays overall Historian properties and disk space.

• DataSet Count - total number of DataSets within the Historian
• Licensed Tags - total number of tags in the Historian counted against the license
• Updates/Sec - number of raw updates (TVQs*) the Historian is receiving per second
• Readers - total number of tags that are being actively read from the Historian
• Writers - total number of tags that are being actively written to the Historian
• HDB Files - total number of .hdb3 files within all DataSets
• Offline Files - total number of .hdb3 files that are offline within all DataSets
• Disk: - the amount of free space on the drive where the Historian data is stored

*The term TVQ is used extensively in our documentation. It refers to an update containing a Timestamp, Value and Quality.

HISTORIAN OPTIONS

By selecting the white down-arrow, the user has the option of importing properties across multiple DataSets. For information on how to import properties see How to Import Custom Tag Properties.

DATASET

The 'DATASET' window in the lower left will display properties and settings for the selected DataSet.

• Name - name of the DataSet
• Tag Count - total number of tags in the most current .hdb3 file
• Updates/Sec - number of raw updates (TVQs) the DataSet is receiving
• Readers - total number of tags that are being actively read
• Writers - total number of tags that are being actively written
• HDB Files - total number of .hdb3 files within the DataSet
• Last File - the name of the last file in the DataSet. Each file name contains the DataSet name and the date/hour when it was created.
• Offline Files - total number of .hdb3 files that are offline
• Directory - path where the .hdb3 files are stored
• Rollover - if logging continues, this shows the date/time of the next planned rollover when a new .hdb3 file will be created. If logging is stopped and resumed after the time passes then a new rollover will occur when logging starts.
• Validate - True or False, if the background validation process is enabled for the DataSet. Default is True.

DATASET OPTIONS

By selecting the yellow down-arrow, the user is presented with several options for interacting with the selected DataSet.

• ROLL-OVER - forces the Historian to create a new file to write to
• ROLL-UP - forces the Historian to combine its daily files into weekly, monthly, or quarterly files if it is configured to do so
• OBSOLETE TAGS - prompts the user for a csv file with a list of tags to obsolete (See Obsoleting Tags from a Dataset for more details.)
• IMPORT PROPERTIES - prompts the user for a csv file with a list of tags and properties they wish to import into the DataSet (See How to Import Custom Tag Properties for more details.)
• DELETE PROPERTY - prompts the user to enter a property they wish to delete from all tags within the DataSet
• REFRESH FILE LIST - refreshes the list of .hdb3 files displayed if a file has been copied into or removed from the directory where the files are stored
• VALIDATE ALL FILES - performs a validation on all files in the DataSet

DataSet Tiles

A tile is displayed for each DataSet configured in the Historian. Additional information is displayed when the DataSet is being logged to and being read from.

• Offline Files - displays the number of offline files, only if they exist. Clicking on the text acts as a hyperlink which will open the DataSet and navigate to the first offline file.
• Writers - the number of tags that are being actively written to the DataSet
• Updates - the number of TVQ's/sec being written to the DataSet
• Readers - the number of tags that are being actively read from the DataSet

A red exclamation mark (!) indicates the DataSet has not received data in the time configured. (See Configuring an Alert when a DataSet Stops Receiving Data for more details.)

Double-clicking on a tile or selecting the yellow right-arrow when hovering over the tile will allow the user to navigate into the DataSet to view the .hdb3 files.

Database Screen (File Level)

Drilling into a DataSet, the user can see the individual .hdb3 files. Blue tiles indicate the file is online. Offline files are gray. By default, the Historian creates a daily file to write to. These files are created at midnight when the Historian does its "roll-over" for all active DataSets.

A file with a "clock" icon indicates the file has been modified since it was last validated. These files will eventually be validated by the Historian's background validation process. The current file will always have this icon if it is being actively written to.

A file with a question mark "?" indicates the file is questionable as it may not have been closed properly while it was being written to when the Historian service shutdown. As such, the background validation process will evaluate this file to determine if there is any corruption and/or needs to be taken offline.

DATASET & DATASET OPTIONS

The metrics displayed in the 'DATASET' window are the same as those displayed at the DataSet level when a particular DataSet is selected. See the DATASET section above for more details.

Likewise, the user is presented with the same 'DATASET OPTIONS' as described previously when selecting the white down-arrow.

FILE

The 'FILE' window displays properties and settings for the selected file.

• Created - date/time when the file was created
• Validated - if True, the date/time when the file was validated; otherwise, False
• Tag Count - number of tags in the file
• File Path - full directory path of the file
• File Size - size of the file on disk
• Embedded Size - files open for writing will be slightly larger due to allocated space. This will display the size of the file with the extra space released.
• Rolled Over - True or False, depending on whether the file rolled over to a new file at its configured time or if a new file was created upon logging starting
• Offline - True of False, files can be taken offline manually or by background validation if an error is detected. They appear as gray tiles.
• Status - current state of the file

FILE OPTIONS  

By selecting the yellow-down arrow, the user is presented with a few options to perform upon the selected file. These options vary depending if the file is online or not. If the file is online the user can...

• VALIDATE - manually runs the validation process for the selected file(s) checking to see if there are any errors. (The background validation does this automatically with any files that have been modified since it was last validated.)
• TAKE OFFLINE - takes the file offline
• RENAME TAGS - prompts the user for a CSV file containing the current tagname and the new tagname they wish to rename it to. (See How to Rename Tags in a DataSet for more details.)

If the file is offline the user can...

• VALIDATE - manually runs the validation process for the selected file(s) checking to see if there are any errors
• PUT ONLINE - moves the file online
• RECOVER - runs a recovery process in an attempt to recover any unusable data and correct any possible data corruption
• DELETE - permanently deletes the file and its data contents. A new popup menu is presented to the user to confirm deletion.

Database Screen (Tag Level)

Drilling into an .hdb3 file, the user is presented with tags listed alphanumerically. Selecting a tag from the list will allow the user to see its individual TVQs in the grid on the right and its properties (if any) in the PROPERTIES window in the bottom-left.

The tag list can be filtered with the search bar using keywords (case insensitive) separated by a space. Keywords can be excluded from the search by prepending them with an exclamation mark (!). This list can be exported to a CSV file using the down-arrow button to the right of the search bar. The user will be presented with an option to include the parent path which will prepend the DataSet name to the tags if enabled.

When viewing a tag's data, the most recent TVQ is at the bottom. The user can scroll up through the tag's history with the vertical scroll bar on the right or the horizontal scroll bar at the top. The horizontal scroll bar is divided into vectors allowing the user to jump to various points in history throughout the file.

Canary logs data by exception whenever there is a value or quality change; therefore, it is not expected to see a TVQ every second/minute (depending on the Collector's sample rate).

If a tag does not change frequently the Historian will time-extend the last Good value approximately once a minute if it is actively logging. This can be seen by looking at the last two rows in the table. The values will be the same but the last one will have a current timestamp. The historian will NOT time-extend Bad quality values.

• TREND -opens a new screen displaying a basic trend of the tag for the current file

  

• REFRESH - retrieves any new updates that may have arrived since selecting the tag. Only available when viewing the most recent file.

TAG OPTIONS

Selecting the down-arrow in the PROPERTIES window, the user has the option to rename a tag or delete one of its properties.

• RENAME - opens a new dialogue menu allowing the user to rename the tag. The user has the option of renaming the tag in the...
  • entire DataSet
  • current file and all older files
  • current file and all newer files
  • current file

    

• DELETE PROPERTY - opens a new dialogue menu allowing the user to delete a single property of the tag. The property name must match exactly including case and whitespace. A property cannot be deleted if the tag is being actively logged.

  

Diagnostics Screen

The 'Diagnostics' screen displays information about the Historian and the server host it is running on.

GENERAL

• CPU System Usage - this is a System counter connected to Processor(Total)\% Processor Time. This % Processor Time (_Total) performance counter under the Process performance object provides overall utilization of all processors.
• CPU Historian Usage - CPU usage is calculated by getting the total amount of time the system has operated since the last measurement (made up of kernel + user) and the total amount of time the process has run (kernel + user). The Historian is a multi-threaded application and with multi-core PC's, the Historian processing will be distributed across multiple CPUs.
• CPU Historian Peak - indicates the highest CPU usage since the start of the Historian service.
• Write Client Count - number of clients connected for writing data into the Historian. These clients are identified on the left under the 'CLIENTS' panel.
• Write Handle Count - number of tags that are being written to by clients at the current time.
• Write TVQ/Sec - the number of TVQs that have been sent to the Historian by write clients. It indicates the current writing throughput of the Historian.
• Write Request/Sec - a count of write requests made by external clients. One write request will contain multiple TVQs.
• Write Extension/Sec - if an update is not received for a tag within 60 seconds and the Receiver session is valid, the Historian will extend the timestamp of the previous value
• Read Client Count - clients making read requests to the Historian. Some clients remain connected while others (i.e. Excel Add-In) will disconnect as soon as request is satisfied.
• Read Handle Count - number of tags being read by clients at the current time
• Read TVQ/Sec - total number of TVQs being read for external client applications
• Read Request/Sec (Historical) -  count of read requests for historical data by external client applications. One read request will contain multiple TVQs.
• Read Request/Sec (Live) - count of read requests for live data by external client applications. One read request will contain multiple TVQs.
• Historian Admin Request/Sec - requests for Historian status information. For example, statistics running at the bottom of the Diagnostics (Up Time, DataSets, Total Tags, etc...).
• Memory Physical Total - total physical memory installed on the machine.
• Memory Physical Usage - the Historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of Physical Memory currently being used.
• Memory Physical Peak - indicates the highest usage since the start of the Historian service
• Memory Page Usage - the Historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of the Page File currently being used.
• Memory Virtual Usage - the Historian calls the OS function GlobalMemoryStatusEx and from that information sets the 3 Memory counters. This counter shows the percentage of the Virtual Memory current being used.
• Historian Working Set - this is a system counter connected to \Process(CLIHistorian)\Working Set
• Historian Thread Count - amount of OS threads in the Historian
• Historian Handle Count - amount of OS handles used by the Historian

BACKGROUND

• Indexing - (obsoleted)
• Roll-Up - this field will show the current DataSet and file being Rolled-Up as it progresses. The Roll-Up runs a few seconds after the Roll-Over of a DataSet or manually from the Arrow Down on the DataSet panel.
• Validation - checks each DataSet for files to be automatically validated. If a file needs validated this field will show the DataSet and filename as it progresses. The global validation can be enabled/disabled for all DataSets from Historian > Configuration > Settings screen. Individual DataSet validation can be controlled by selecting the DataSet from Historian > Configuration>DataSets screen.

ACTIVITY

• Instant - instantaneous count of TVQ writes and reads.
• Minute - TVQ write and reads over the past minute.
• Hour - TVQ write and reads over the past hour.
• Day - TVQ write and reads for the past day.
• Month - TVQ write and reads for the past month.
• Total - TVQ write and reads since the Historian was installed on the machine.

CLIENTS

This panel allows the user to monitor any clients that are logging data into the Historian or clients that are reading data from it. The Store and Forward (SaF) Service is the primary logging client and viewer clients will vary depending on how the data is needed.

• Writer - displays the number of tags being written and the identity of the client. "Historian Diagnostics" will always show as a writer.
• Reader - displays the number of tags requesting data and the identity of the client

STATUS BAR

The gray status bar at the bottom of the screen displays the following parameters:

• Start Time - time when the Historian was last started
• Up Time - length of time the Historian has been running since the last start
• Run Time - cumulative amount of time the Historian has run
• DataSets - the total number of DataSets defined (includes {Diagnostics})
• Licensed Tags - total number of tags from all DataSets being counted against the Historian license (excludes the {Diagnostics} DataSet). The actual Historian license, tag limit is shown in the Licenses tile.
• Total Tags - total number of tags from all DataSets (includes {Diagnostics})
• Files - total number of files in all defined DataSets (excludes the {Diagnostics} DataSet)
• Version - the version of the Historian where the Canary Admin is connected

Configuration Screen

DataSets

This panel shows all DataSets defined in the Historian and provides creation and removal options.

• NEW - creates a new DataSet. DataSet names must be more than 1 character in length. Once a DataSet name and path is provided, click the 'CREATE' button. (Note: With the exception of the OPC DA Collector, a DataSet does NOT need pre-created prior to logging to it.)
• SCAN - allows the user to select a path where the Historian will look for .hdb3 files and automatically create the DataSet accordingly. After entering a Path, click 'SCAN' to search the directory and if 'Include Subdirectories' is checked, any subdirectories.  
• RENAME - allows the user to rename a DataSet name  
• REMOVE- allows the user to remove a DataSet. Clicking the button will display a confirmation dialog with the option to "Delete All Associated Historical Data" if checked. Click the OK button to remove the DataSet.  

Note: DataSet must be closed (no writers or readers) before the Historian will rename or remove it.

SETTINGS

This panel provides file management settings for the selected DataSet.

• DataSet Directory - directory location where the .hdb3 files are stored
• Validate - when checked, a background thread will validate any files for the DataSet not already validated, except the current file
• Email if no data received in ?? minutes - this requires SMTP Server and email addresses to be setup under Messages > Configuration > Email (See Configuring the Canary System to Send Email Alerts). An email notification will be sent to the recipients if no data is received for all tags in the DataSet within the set time.
• Notify Email - semicolon-delimited list of recipients to receive an email if no data is received in ?? minutes
• Roll-Over - the frequency at which the Historian creates a new file to write to. Aside from physical disk corruption, the only time a file is susceptible to corruption is when it is opened for writing. Should something happen during one of these writes such as a power failure or machine reset without a controlled shutdown, the file could become unusable. This information should be considered when deciding when to Roll-Over to a new file. For instance, creating a file every day at midnight would only risk the data from the time of the Roll-Over to when the system failure occurred, limiting the possible loss to only a few hours.
  • Never - (Very rare) all data will be written into one continuous file
  • Daily - (default) a new file will be created every day at hh:mm (hours:minutes) after midnight.
  • Weekly - a new file will be created at hh:mm after midnight on selected day of the week
  • Hourly - a new file will be created starting at 00:00 - 24:00 (24 hour time) every 2, 3, 4, 6, 8 or 12 Hours. This is to accommodate 1 file per shift.
  • Custom - a new file will be created every 0-100, Minutes, Hours or Days.
  • Size - a new file is created when the current file exceeds 10 - 500 megabytes.
• Roll-Up - a background operation to take files at the Roll-Over rate and combine them into Daily, Weekly, Monthly or Quarterly files. Once rolled up, the files can no longer be converted back to their original Roll-Over size. This option can be most valuable to customers with large quantities of tags as the tag information is duplicated in each file. If the existing file and the next file to be rolled up exceed 500 MB, the operation will not start and a 2nd Roll-Up file will begin for the time period. Roll-Up does not roll up the last file in the DataSet.
• Delete - Files with a creation time older than 1 - 365 days, months or years will be deleted at the time when a Roll-Over occurs. Never will keep all files.
• Move deleted files - moves the deleted files to another location where they are no longer visible to the Historian. (See How to Move Historical Data to a Temp Directory for Cold Storage)

Settings

• Background Validation - Checks each DataSet for files to be automatically validated. Global validation can be disabled for all DataSets by unchecking this box.
• Deleted Files Directory - the location where deleted files will be stored if a DataSet is configured to "Move deleted files" (See How to Move Historical Data to a Temp Directory for Cold Storage)

Endpoints

• IPC - endpoint used by local Canary services to store/retrieve data and perform other DataSet operations
• gRPC Admin (55283) - endpoint used specifically by the local Admin service for configuring settings and requesting status information
• gRPC API (55282) - endpoint used by remote Canary services to store/retrieve data and perform other DataSet operations. Client identity is validated using a Canary JWT or an API token in the message headers.

CERTIFICATE (Secure)

By default, Canary generates a self-signed certificate for its services that require a secure connection. These can be reconfigured if wishing to use a different certificate that has been installed on the server. Clicking the INFO button will open a new window displaying the certificate's details.

• Kind - the kind of certificate the Historian service is configured to use
  • SelfSignedCertificate - this is the default certificate that Canary uses. It is stored in the Local Computer\Personal store.
  • Certificate - use this option if wishing to upload your own certificate. Doing so will present more options to configure.

    

    • Store Name - the local computer store name where the certificate is installed
    • Find Type - criteria by which the Historian service searches for the certificate
      • FindBySubjectName
      • FindByThumbprint
      • FindByTemplateName
• Subject Name - set to 'default' if using the SelfSignedCertificate option. Otherwise, the value of the Subject, Thumbprint, or Template name if using the Certificate setting.