Creating a Virtual View or Asset Model (version 23)

Since the Views service is the single access point for all historian data queries, it is possible to create virtualized views, or Virtual Views, of the underlying historian archive.  This process allows system administrators to alias tag names, add structure, leverage metadata properties, and ultimately group tags into asset models.

Initial Considerations

To be successful with Virtual Views, it is important to first consider the following:

1. Firmly understand the methodology behind how to structure tag names to most simply group and identify assets.  See Virtual Views and Asset Models Primer for more information.
2. Multiple Virtual Views can be built.  One Virtual View does not have to suit all use cases.
3. Virtual Views can be built from multiple underlying DataSets.
4. A Virtual View can be built on top of another Virtual View.  For example, one view could establish a unified name space, while a  second view built upon the first establishes an asset model.
5. Consider how a user wants to report or browse data.  This proves a good indicator on how to organize a Virtual View.  For example you may want to implement views that address some or all of these common model types:
   • Flat Models - If the only desire is to create a table of all known 'Boilers' across the entire organization, spending time on large detailed asset models with multiple levels of asset hierarchy may not be necessary.  The model could be very flat and only contain one level.
   • Production Focused - Business units that may care only about production values do not need to, or care to, see process value tags.  The model they access should have very limited tags but clearly link production figures to sections of the operation that can then be rolled up into production tags that represent several levels of the operation... perhaps line production statistics roll to section stats, section to plant, plant to district, district to region, and ultimately region to company production numbers.  
   • Deep Models - Some models need to be very complex and deep in their hierarchy, representing multiple levels of parent-child nesting.  This allows visibility of how one set of assets impacts others.
   • Maintenance and Preventative Models - If the desire is to incorporate maintenance ticketing systems or enterprise asset management systems, the Virtual View should be constructed to match the existing structure that has already been defined.  This can often be most efficiently accomplished by supplementing the model with metadata properties that are actively pulled from these existing systems. 
   • SCADA/DCS Models - Duplicating the existing data views of a SCADA or DCS system can be useful when providing data queries to staff that are accustomed to interacting with data in that format.
   • No Defined Assets - Some Virtual Views don't use asset modeling at all; they are built to filter out unwanted tags from the browse tree and sometimes also to alias tag names.  These views can be very helpful from a data access point of view.  Security permissions can be applied to an entire Virtual View and allow for designated users to only access data from the provided Virtual View.

How to Structure Tag Names

If a Virtual View is being built to construct an asset model, it is important to understand how Canary reads tag structure.  The goal is to add periods, or dots, within the tag path that indicate multiple levels of hierarchy and establish asset structure. There are no set limits in place on the number of branches a tag name may contain, thus allowing for very deep models.

Notice in the below example how the tag names in the first column use the period to section the tag into three parts.  

In this example, the first section of the tag name represents to which line instance the tag belongs.  The second section of the tag name affiliates it with either a boiler, filler, or water main.  As demonstrated, multiple asset types can occupy the same branch or sub node.  Finally, the third section of the tag name describes the sensor or I/O that is being measured.

There is no need to separate the asset type and the corresponding asset instances in the tag structure.  The following structure would be problematic,  "Line.1.Boiler.001.Pressure".  Instead structure the tag as "Line 1.Boiler 001.Pressure" , combining the asset type and asset instance inside the same section. 

There is flexibility in tag structure as well.  For instance, not all tags need to have the same number of branches.  If two tags were added to the above model, each named 'Line1.Production_YTD' and 'Line1.Alarm' these tags would be associated with a 'Line' asset type but would not be associated with an additional piece of equipment.  Instead, because the tag name terminates in the second branch, the tags 'Production_YTD' and 'Alarm' would be linked to the asset type 'Line'.

Understanding Asset Components

Asset models are built from asset types.  Asset types are user defined.  They can represent physical assets, geographic areas, or even business units.  Once an asset type is defined, the number of occurrences of that asset are referred to as asset instances. 

Above, we demonstrated four asset types, 'Line', 'Boiler', 'Filler', and 'WaterMain'.  There is only a single asset instance of 'Line' currently and it is called 'Line1'.  There are two asset instances of the 'Boiler' asset type.  They are known as 'Boiler001' and 'Boiler002'.

Since the asset instances of the 'Boiler' asset type are nested under a 'Line' asset type, we know that 'Boiler001' and 'Boiler002' are children of 'Line1'.  We also know that 'Boiler001' and 'Boiler002' are related to 'Filler001' and 'Filler002' in that all four are children to 'Line1'.

Note, we have not established that 'Boiler001' is related to 'Filler001' and not to 'Filler002'.  In order to do this, the tag names would need to be adjusted.  How would the model change using the following tag names?

Based on the above tags, we can derive the following:

1. This model has three levels of asset types. 
2. The first level is 'Line', which has two tags associated with it and three linked asset types, 'Boiler', 'WaterMain', and 'Filler'. 
3. 'WaterMain' is represented by two tags and contains no child asset types.
4. 'Boiler' is defined by three tags as well as the 'Filler' asset type.
5. Some 'Boiler' instances have a single 'Filler' (Boiler001) while another 'Boiler' instance has two 'Filler' instances (Boiler002).
6. 'Filler' is defined by five tags and has no child asset types.

An asset instance can also be associated with multiple parent asset types.  For example, a single 'WaterMain' instance may need associated with multiple 'Line' asset type instances.

Transforming Tag Names

Often tag names have the existing logic required to be able to transform them into asset models.  If tag names are lacking enough logic, tag metadata can also be used.  Sometimes tag metadata is provided at the point of logging;  however, for most organizations, this data lives in other databases. This data can be actively read and added to the Views service if it is held in Microsoft SQL or SQLite.

To utilize metadata properties from an external database, open the Canary Admin application and select the Historian tile. Navigate to the 'Configuration' menu at the bottom and select the 'Settings' tab on the left.  

The linked database can be configured from this application screen but will require additional table configuration.  Specifically, the creation of stored procedures within the SQL database as well as the creation of a DataSet table within the SQL database.  Canary does not require a fixed table schema.

In nearly all cases, assistance from the Canary Support Team is necessary for metadata property integration.  

Using Regular Expressions to Alias Tags and Build Models

Since the building of Virtual Views often requires tag names to be transformed based on existing logic found within the names or the metadata linked to the tag, system admins need a fast and efficient way to find, capture, and replace tag names in mass.

Regular Expressions (also called RegEx) allow for rules to be created and applied that achieve the following outcome:

1. The exclusion of tags not useful to the view.
2. Capturing useful tag names while excluding or removing useless content.
3. The addition of or reordering of tag structure.
4. Identifying asset types and their corresponding instances.
5. Determining which part of a tag string provides the unique name of an asset instance.

Types of RegEx Rules

The above is accomplished by creating rules within the Virtual View.  Currently, three categories of rules exist.

• Model Rules - These rules focus primarily on restructuring and aliasing tag names based on existing tag name logic and any other metadata properties needed.
  • Example - Transforming the tag name '20-BL001-PT-001-PV' to be aliased and structured as 'Line1.Boiler001.SteamOut'.
• Asset Rules - Identifies asset types within tag name branches and determines how asset instances of those types are named.
  • Example - Identifying the following tag 'Line1.Boiler001.SteamOut' as containing an asset type of 'Line' within the section of the tag name and defining the asset instance of a 'Line' as well as the numeric value that immediately follows 'Line' but not the rest of the tag string, resulting in the asset instance being named 'Line1'.
• Parent/Child Rules - Less commonly used but highly effective for cloning or duplicating an asset that is a child of multiple parent assets.
  • Example - The tag 'Line1.WaterMain.Volume' may also need cloned to represent 'Line2.WaterMain.Volume' if both 'Line' asset types share a single physical 'WaterMain' asset instance.
• Copy Rules - Used to copy a branch of tags from one section of the hierarchy to another.
  • Example - The 'Boiler' assets may need moved to be seen as a top-level asset in addition to a child-asset of the 'Line'.

How to Build a Virtual View

Virtual Views can take many forms.  Following these steps will give you, as the system administrator, the necessary tools to create views for your clients.

Create a New View

1. Open the application Canary Admin and click on the 'Views' tile.
2. From the 'VIEWS' window, select the down arrow icon and choose 'CREATE NEW VIEW'.
3. Name your view in the 'View Name' text field.
4. Use the drop down box to select the existing view you wish to include.  If the 'Local Historian' view is selected, you may further filter the results by selecting a DataSet.  Additional DataSets may also be included by clicking the 'ADD SOURCE' button.
5. Choose to include metadata associated with the tag names by listing the corresponding property titles within the 'Properties' field.  Multiple properties may be listed when separated with a semicolon (;).  If metadata is not necessary, leave this field empty.
6. Click 'OK'.

Navigating the View Creation Screen 

Once the new Virtual View has been created, four windows are displayed which provide the necessary information to add RegEx rules and further refine the view.

In the top left 'VIEWS' window, you will find diagnostic information relating to the Views service.

The bottom left 'VIEW' window displays data that relates to the current view you are building.  This includes the last known update to the view, the sources the view is built upon, tag count, and a list of all known asset types once created.

The main window at top right 'View Name Defined Rules' will display all existing rules and data around those rules.  In addition, a series of options present as buttons at the bottom of this window.  These buttons allow for the creation, modification, removal, and diagnostics of view rules.

Below this window, separated with a drag bar, is a list of all included tags. This tag list is useful for monitoring the progress of the view rules.

Adding a Rule

To create a Model, Parent/Child, or Asset rule, click the 'ADD' button located within the main window.  A new 'Rule Definition' dialogue box will present.  Select which type of rule you wish to create.

Model Rule

Enter the appropriate RegEx in the 'Match' field.  As your RegEx matches tags, the first matching tag will be displayed directly below the text box.

Next enter the appropriate text or RegEx in the 'Replace' field.  Below this text box you will see how the above referenced tag is being transformed.

Select the 'Exclude' box if you wish to exclude all matching tags from the model.  If this is the case, the 'Replace' field would not be necessary.

Click 'OK' to apply your rule.

Asset Rule

Once all tags are formatted using model rules, you can identify asset types and corresponding asset instances using the Asset Rule.  

Enter the appropriate RegEx in the 'Match' field.  As your RegEx matches tags, the first matching tag will be displayed directly below the text box. Ensure you properly identify the entire section of the tag name that indicates the asset type you desire.  Your RegEx should stop at the dot ('.') that immediately follows your asset.

Name the asset type by entering the text in the 'Asset Type' field.

Click 'OK' to apply your rule.

Parent/Child & Copy Rules

For this feature to function properly it is recommended you consult with Canary staff prior to attempting to execute the rule.  Parent/Child rules are extremely complicated and require external properties be logged to tags.  Most operations will not benefit from this rule type.  

Proofing Views Rules

Once rules have been applied, they will appear in the '...Defined Rules' window.  Depending on the rule type, some or all of the columns will be used.  Close attention should be paid to the 'DUPLICATES' and 'TIME (MS)' columns. 

Duplicates should be avoided.  Having a rule that creates duplicates means that the uniqueness of the tag name has been removed and now matches one or more other tag names in the view.  The result will be for all duplicates to be ignored and essentially excluded by the view.  If duplicates do exist, you may diagnose the issue by selecting the rule, a single click will highlight the rule in question, and then clicking on the 'RULE DETAILS' button.  The 'Rule Details' window will show all tags that have been matched in a list, along with what each tag was replaced with.  In addition, when duplicates are present, a 'Duplicates' radio button will be available.  Selecting this option will show all known duplicates.

The 'TAG TRACING' button will provide a tag search dialogue that will allow you to find a tag that is included in the view, and see how it has been effected by each rule that has been applied.  When using the search, enter the tag name as it appears in the historian archive.  Once the tag has been found, click on it to see how it has been transformed by the view rules.

Editing or Removing Rules

You may edit or remove rules by selecting the rule with a single-click, and then choosing either to remove or edit the rule.  You may also re-order rules by simply clicking and dragging the rule above or below existing rules.  Rule order is important as the rules are applied chronologically.

Applying the Virtual View

As rules are applied, you may also browse the tags below the '...Defined Rules' window.  This can be done by expanding the tag lists by clicking on the down arrows.  When all tags are grouped as desired, you are ready to apply the rules and create the Virtual View.

In the top right hand corner of the view window, a red bar with the button 'APPLY' should be visible.  Click this button.  You may now close the view from edit mode by clicking the 'CLOSE' button.  Browse the view to ensure the results are what you expected.