Industrial Automation Automation Industrielle Industrielle Automation

4 Access to devices
4.3.4 OPC Historical Data Access

Prof. Dr. H. Kirrmann ABB Research Centre, Baden, Switzerland

2007 May, HK

HDA: Overview
OPC Common Overview: usage and specifications OPC as an integration tool Clients and Servers: configuration OPC Technology, client and custom interface OPC Data Access Overview: browsing the server Objects, types and properties Communication model Simple Programming Example Standard and components OPC Alarms and Events Specification Overview: definitions and objects Events Alarm Conditions Automation Interface OPC Historical Data Specification Overview

Industrial Automation

4.3.4 OPC HDA - 2

Historians: An introduction

The role of HDA will become clearer after a first glance at historians

The process database decouples the plant from the analysis tools. It contains recent and historical data.

Invensys (Wonderware) Archestra Industrial Automation 4.3.4 OPC HDA - 3

Historians Example

GE Fanuc/Intellution iHistorian (iFix...)

Questions to the historian: What was the value of FIC101 last week ? What was the flow average during October ? Which were the daily averages in October ? What was the total flow in each month ? How much fuel did we use for the batch ? Give the answer in form of tables, pie diagrams, spreadsheet, reports

Features: Unlimited Point Collection Sub-Second Data Collection Rates Enhanced Data Compression True Thin Client Administration Fault Tolerant Architecture Industrial Automation

data collection, archiving and retrieval report generation computations (e.g. VBScript) secure access (FDA 21 CFR 11) 20'000 actions/ s, up to 100'000 data points 4.3.4 OPC HDA - 4

Historians: Market

2003 Global Information Management Market by Industry

2% 1% 6% 11% 21%

13% 13% 12% 21%

Chemical Food and Beverage Petroleum Pulp and Paper Pharmaceuticals Power Textiles Waste and Water Other

Industrial Automation

4.3.4 OPC HDA - 6

Historians: Players
ABB

Information Manager (formerly Enterprise Historian, bases on Oracle)

InfoPlus.21

AIP/PPA (embedded) Historian is another product (small player) AspenTech

Complete offering with wide range of accompanying applications and services Very good user interface

OSISoft PI 3.4

Largest installed base (perceived as de facto standard) Easy-to-use Perceived low cost (PI 3.4: 1 Mio data points / 80'000 /s)

GE-Fanuc Intellution iHistorian

Relatively new to market. Product created with Mountain Systems Causing a stir for OSIsoft, AspenTech on pricing

Some small players

Honeywell (POMS) Uniformance Invensys (Wonderware) IndustrialSQL 8.0 Rockwell Automation (RSBizWare) Historian Siemens (WinCC) Historian (bases on MS SQL Server)

Industrial Automation

4.3.4 OPC HDA - 7

Historian implementation

application OPC HDA

ODBC/SQL and OPC HDA are complementary

log organisation "SQL"

history database
collector operator actions "OPC" field data Industrial Automation 4.3.4 OPC HDA - 8 corrections

HDA: Historical Data Access

e.g. Trend Analysis e.g. Event Logger

HDA Clients independent processes

OPC HDA Server

hidden calculations

history database
collector

raw and ordered data

OPC DA Server

proprietary data acquisition

Field device

Field device

Industrial Automation

4.3.4 OPC HDA - 9

HDA: Purpose An OPC HDA server gives access to a historical data base (logs) in which data from the process have been collected and time-stamped, possibly through an OPC DA interface. The OPC HDA interface clients, such as trend analysis, product tracking or data mining, that require ordered access these data logs. The OPC HDA interface allows to: - browse the historical data base - retrieve data through proper filtering, e.g. by date range, by identity, by property - build aggregates over the retrieved data, such as average, minimum, maximum.

- enter new entries, correct entries or remove entries

- enter / delete annotations in the history data base

Industrial Automation

4.3.4 OPC HDA - 10

HDA: Raw log

12.3.02 13:40 12.3.02 13:40 12.3.02 13:40 12.3.02 13:40 12.3.02 13:41 12.3.02 13:41 12.3.02 13:41 12.3.02 13:41 12.3.02 13:41 12.3.02 13:41 12.3.02 13:41 12.3.02 13:42 12.3.02 13:42 12.3.02 13:42 12.3.02 13:43 12.3.02 13:43 12.3.02 13:43 12.3.02 13:43 12.3.02 13:43 12.3.02 13:43 12.3.02 13:43 12.3.02 13:43 12.3.02 13:44 12.3.02 13:44 12.3.02 13:44 12.3.02 13:44 12.3.02 13:44 12.3.02 13:44 12.3.02 13:44 Gpcpt2ofpbonne Cpt2bac Gpcpt2bac Gpcptbe2 Gpcpt1bac Gpcpt1ofpbonne Gpcptae2 Cpt1bac Gpdefr2 Gpvoydef Gpr3tempscycleprd Gpstn1e1 Gpalarme1 Gpalarme2 Gptpscycle Gpetatmodemarche Gpdefgene1 Gpetatmodemarche Gptpscycle Gpdefr2 Gpvoydef Gpdefgene1 Gpetatmodemarche Gpr2tempscycleprd Gptpscycle Gpalarme3 Gpalarme4 Gpalarme3 Gpcpt2ofpbonne 4824 50 70 45 151 4826 45 49 64 2 318 16 0 0 2 1346 1 16 0 317 0 0 0 1 1992 435 1 1 0 4823

By definition, values are registered when they change. (even if data are acquired by periodic polling) Data in the historical database are identified by their itemID (here, represented by their name), value, (of the respective type) quality (good, stale, bad), and timestamp (UTC).

12.3.02 13:43 Gpetatmodemarche

Industrial Automation

4.3.4 OPC HDA - 11

HDA: How to reduce raw data (even before HDA comes into play) Tag values are sent to the OPC DA server when they change (on events). The collector records these data in a log that is organized cyclically. Since a log has a limited storage capacity, data are reduced by several means: 1) if a variable is received more often than the log's minimal storage interval (e.g. 1s), the log keeps the latest of all values of the interval 2) if (and only if) a received variable changed by more than the "exception deviation" (e.g. 5%, analog values only), it is entered into the log. 3) if a variable changed by less than the exception deviation, it may nevertheless be forced into the log after the log's maximum storage interval (e.g. 4 s) elapsed.
min time max exception deviation min time max time max time max time

= process value (event)

Industrial Automation

= log entry
4.3.4 OPC HDA - 12

HDA Application: Trend Display

Parameters: log: how were data sampled time scale (with possible offset, zoom, pan) amplitude scale (low range, high range, scale units) style: smoothed, stepped, filled (several ways to display the same data) extrapolate: how to display values not received (e.g. because they did not yet change)
Industrial Automation 4.3.4 OPC HDA - 13

HDA: More logs

A log registers only one data point !

log 2.1 log 2.2

(log templates can be used for several datapoints)

log 1

log 2

Data in the log may be:

raw log Collector (OPC DA Client) other servers OPC DA Server XML / CSV

Measured values (original, from the OPC DA) Calculated values Corrected values (replaced or inserted) Annotations (explanation strings)

Field device

Field device

Industrial Automation

4.3.4 OPC HDA - 14

HDA: Hierarchical logs

long-term log 1 hour_forever 5.2 MB / Year

3 27 28 29 30 september octobre

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 october

1 mn_7 days log 6 MB

d-7 d-6 d-5 d-4 d-3 d-2 yesterday today

1s_24 hours log: 50 traces @ 12 B 52 MB

23

24

10

11

12

13

14

15

16

17

18

19

20

21

22

A hierarchical log is built on the data contained in the parent log. To reduce the log size, several aggregations can be applied: - record only maximum, minimum, average over a period, etc...
Industrial Automation

Actual data

4.3.4 OPC HDA - 15

HDA role An HDA server allows to browse the logs and filter the logs to be read. It can filter them according to their attributes, such as name of the datapoints, type of the datapoint, engineering units,.... An HDA server then selects the logs to be read. An HDA can then read the logs, defining a certain period (Start time, End Time, and the maximum number of entries to be retrieved. When reading the log, the HDA server can process the log by applying aggregations The "min time", "deviation", "max time" rules used to collect raw data is also used as an aggregation to build new logs. Therefore, hierarchical logs can be built out of a raw log by an OPC HDA service.

Do not confound aggregates and attributes.....

Industrial Automation

4.3.4 OPC HDA - 16

HDA: Server The client can query the HDA server for its capabilities, and in particular: - which LocaleID (national languages) the server supports (as in DA and AE servers) - which attributes the server supports (the attributes of its items is a subset thereof) - which aggregates the server support (and the client can request) - if the server supports correction / deletion of data (synchronous and/or asynchronous) - if the server supports insertion of annotations (synchronous and/or asynchronous) Many capabilities of the HDA server are optional.

Industrial Automation

4.3.4 OPC HDA - 17

HDA: Browsing: structure Like the DA and AE interfaces, OPC HDA server offers a browser with methods allowing to navigate up and down, and to retrieve branches and leaves at each position. If filters are defined at its creation time, an HDA browser only browse for items with specific attributes (e.g. Engineering Units = m/s). The filtering attributes are additive.

HDA (contrarily to DA and AE) allows several browsers, each with its different filter.

root Leaves Object Object Area 5 Log 2 Log 1

Log 3 Area 51 Area 52

Industrial Automation

4.3.4 OPC HDA - 18

HDA Browsing and filtering The filter criteria are based on attributes (see the following slides). Attributes cannot be modified afterwards. Set MyOPCHDABrowser = MyOPCHDAServer.Createbrowser (NumCriteria, AttributeIDs, operatorCode, Filters, Errors

MyOPCHDABrowser.MoveDown (MyOPCHDABrowser.OPCHDABranches(1))

Items = MyOPCHDABrowser.OPCHDAItems
.... list these items

Industrial Automation

4.3.4 OPC HDA - 19

HDA: General attributes of the items

Code Attributes

Description
data type for the item. See the definition of a VARIANT for valid values (VT_R4, etc.) text that describes the item. label to use in displays to define the units for the item (e.g., kg/s). whether data from the history repository should be displayed as interpolated (sloped lines between points) or stepped (vertically-connected horizontal lines between points) data. whether historian is recording data for this item (0 means no). equation to be used by a derived item to calculate its value. This is a free-form string. machine which is the source for the item. This is intended to be the broadest category for defining sources. For an OPC Data Access Server source, this is the node name or IP address of the server. For non-OPC sources, the meaning of this field is server-specific. process which is the source for the item. This is intended to the second-broadest category for defining sources. For an OPC DA server, this would be the registered server name. For non-OPC sources, the meaning of this field is server-specific. name of the item on the source. For an OPC DA server, this is the ItemID. For non-OPC sources, the meaning of this field is server-specific. sort of source produces the data for the item. For an OPC DA server, this would be "OPC". For non-OPC sources, the meaning of this field is server-specific. upper limit for the normal value range for the item, is used for trend display default scaling and exception deviation limit calculations, should be the normal high value for the item. lower limit for the normal value range for the item, is used for trend display default scaling and exception deviation limit calculations, should be the normal low value for the item. item identifier. This is used to allow filtering in the CreateBrowse method.

1 2 3 4 5 6 7

DATA_TYPE DESCRIPTION ENG_UNITS STEPPED ARCHIVING DERIVE_EQUATION NODE_NAME

PROCESS_NAME

9 10

SOURCE_NAME SOURCE_TYPE

11
12 13

NORMAL_MAXIMUM
NORMAL_MINIMUM ITEMID

Industrial Automation

4.3.4 OPC HDA - 20

HDA: Attributes that affect how data are archived

Code Attributes 14 Max Time Interval Data Type VT_FILETIME Description Specifies the maximum interval between data points in the history repository regardless of their value change.A new value shall be stored in history whenever OPCHDA_MAX_TIME_INT seconds have passed since the last value stored for the item. Specifies the minimum interval between data points in the history repository regardless of their value change.A new value shall be not be stored in history unless OPCHDA_MIN_TIME_INT seconds have passed since the last value stored for the item. Specifies the minimum amount that the data for the item must change in order for the change to be reported to the history database. See OPCHDA_EXCEPTION_DEV_TYPE for the specific meaning of this field. Specifies whether the OPCHDA_EXCEPTION_DEV is given as an absolute value, percent of span, or percent of value. The span is defined as OPCHDA_HIGH_ENTRY_LIMIT OPCHDA_LOW_ENTRY_LIMIT. Specifies the highest valid value for the item. A value for the item that is above OPCHDA_HIGH_ENTRY_LIMIT cannot be entered into history. This is the top of the span. Specifies the lowest valid value for the item. A value for the item that is below OPCHDA_LOW_ENTRY_LIMIT cannot be entered into history. This is the zero for the span.

15

Min Time Interval

VT_FILETIME

16

Exception Deviation

VT_R8

17

Exception Deviation Type

VT_I2

18

High Entry Limit

VT_R8

19

Low Entry Limit

VT_R8

Industrial Automation

4.3.4 OPC HDA - 21

HDA: Selecting the items Each server has only one group of items, the collection OPCHDAItems. By adding an OPCHDAItemID to that collection, the client defines its client handle and receives the server's server handle. The OPCHDAItemID (a string) should be retrieved by using the browser (like in DA or AE), since entering a string directly is unsafe. The server handle is used to access the log. Read operations are applied to the whole collection OPCHDAItems.

.... Set colOPCHDAItems = MyOPCHDAServer.OPCHDAItems

colOPCHDAItems.AddItem OPCHDAItemID ClientHandle

colOPCHDAItems.SyncReadRaw .....

Industrial Automation

4.3.4 OPC HDA - 22

HDA: Reading the data When reading the data, the client has the choice to read synchronously or asynchronously (if the server permits the latter). The client specifies the time interval (Start time and end time) and whether the end values of that interval shall be included. The client has the choice to read the raw data or to apply an aggregation to the read data. The aggregations (see next slides) allow to reduce data.

Industrial Automation

4.3.4 OPC HDA - 23

HDA: Aggregates
Symbol 0 NoAggregate 1 Interpolative 2 Total 3 Average 4 TimeAverage 5 Count 6 StDev 7 MinimumActualTime 8 Minimum 9 MaximumActualTime 10 Maximum 11 Start 12 End 13 Delta 14 RegSlope 15 RegConst 16 RegDev 17 Variance 18 Range 19 DurationGood 20 DurationBad 21 PercentGood 22 PercentBad 23 WorstQuality 24 Annotations Description Do not retrieve an aggregate (use only start time - stop time and filtering). Get interpolated values. Get the totalized value (time integral) of the data over the resample interval. Get the average data over the resample interval. Get the time weighted average data over the resample interval. Get the number of raw values over the resample interval. Get the standard deviation over the resample interval. Get the minimum value in the resample interval and the timestamp of the minimum value. Get the minimum value in the resample interval. Get the maximum value in the resample interval and the timestamp of the maximum value. Get the maximum value in the resample interval. Get the value at the beginning of the resample interval (time stamp). Get the value at the end of the resample interval (time stamp of end of the interval). Get the difference between the first and last value in the resample interval. Get the slope of the regression line over the resample interval. Get the intercept of the regression line over the resample interval, (time start of interval). Get the standard deviation of the regression line over the resample interval. Get the variance over the sample interval . Get the difference between the minimum and maximum value over the sample interval. Get the duration (in seconds) of time in the interval during which the data is good. Get the duration (in seconds) of time in the interval during which the data is bad. Get the percent of data (1 = 100 %) in the interval which has good quality. Get the percent of data (1 = 100 %) in the interval which has bad quality. Get the worst quality of data in the interval. Get the number of annotations in the interval.

Industrial Automation

4.3.4 OPC HDA - 24

HDA interpolation interpolation is an aggregate used when the time scale of the derived log is smaller than that of the parent log.
= parent log = filtered log resample interval of the new log

time

This is useful to display smoothly trend diagrams.

Industrial Automation

4.3.4 OPC HDA - 25

HDA corrections and annotations

These features are seldom implemented. Basically, the values are accessed using the server handles that OPCHDAItems.AddItem returned.

Industrial Automation

4.3.4 OPC HDA - 26

HDA Automation Model

OPC Common Overview: usage and specifications OPC as an integration tool Clients and Servers: configuration OPC Technology, client and custom interface OPC Data Access Overview: browsing the server Objects, types and properties Communication model Simple Programming Example Standard and components OPC Alarms and Events Specification Overview: definitions and objects Events Alarm Conditions Automation Model OPC Historical Data Specification Overview Automation Model Industrial Automation 4.3.4 OPC HDA - 27

HDA: Automation Interface summary OPCHDAServer OPCHDAItems (col) OPCHDAItem OPCHDAItem OPCHDABrowsers
An instance of an OPC HDA Server. An OPCHDAServer object must be created to can get references to other objects. A collection containing all the OPCHDAItem objects this client has created within the scope of the OPCHDAServer object

An object that maintains the items definition. Note the Custom Interface does not provide a separate Item Object.
An object that browses item names in the servers configuration.

OPCHDAHistory

An collection containing a series of either OPCHDAValue or OPCHDAEntry objects, representing historical values of an item.

OPCHDAValue OPCHDAValue OPCHDAEntry OPCHDAEntry

Industrial Automation

An object that represents a discrete historical value for an item or an attribute.

An object that extends the OPCHDAValue object to include additional information

4.3.4 OPC HDA - 28

HDA: Object Interfaces (1)

Industrial Automation

4.3.4 OPC HDA - 29

HDA: Object Interfaces (2)

Industrial Automation

4.3.4 OPC HDA - 30

To probe further. OPC Foundation: Specifications http://www.opcfoundation.org SoftwareToolbox Examples in Visual Basic
http://www.softwaretoolbox.com/Tech_Support/TechExpertiseCenter/OPC/opc.html

The Code Project OPC and .NET

http://www.codeproject.com/useritems/opcdotnet.asp

Matrikon Free client and server:

http://www.matrikon.com

WinTech Toolkit for an OPC server

http://www.win-tech.com/html/opcstk.htm

NewAge Automation Toolkit for an OPC server

http://www.newageautomation.com

Industrial Automation

4.3.4 OPC HDA - 31

To probe further. OPC Foundation: Specifications http://www.opcfoundation.org SoftwareToolbox Examples in Visual Basic
http://www.softwaretoolbox.com/Tech_Support/TechExpertiseCenter/OPC/opc.html

The Code Project OPC and .NET

http://www.codeproject.com/useritems/opcdotnet.asp

Matrikon Free client and server:

http://www.matrikon.com

WinTech Toolkit for an OPC server

http://www.win-tech.com/html/opcstk.htm

NewAge Automation Toolkit for an OPC server

http://www.newageautomation.com

Industrial Automation

4.3.4 OPC HDA - 32