Documente Academic
Documente Profesional
Documente Cultură
CIMPLICITY HMI
Server Redundancy
Operation Manual
GFK-1353D
March 1999
GFL-002
Warning
Warning notices are used in this publication to emphasize that hazardous voltages, currents, temperatures, or other
conditions that could cause personal injury exist in the equipment or may be associated with its use.
In situations where inattention could cause either personal injury or damage to equipment, a Warning notice is used.
Caution
Caution notices are used where equipment might be damaged if care is not taken.
Note
Notes merely call attention to information that is especially significant to understanding and operating the equipment.
This document is based on information available at the time of publication. While efforts have been made to be accurate,
the information contained herein does not purport to cover all details or variations in hardware or software, not to provide
for every possible contingency in connection with installation, operation, or maintenance. Features may be described
herein which are not present in all hardware and software systems. GE Fanuc Automation assumes no obligation of
notice to holders of this document with respect to changes subsequently made.
GE Fanuc Automation makes no representation of warranty, expressed, implied, or statutory with respect to, and assumes
no responsibility for the accuracy, completeness, sufficiency, or usefulness of the information contained herein. No
warranties of merchantability or fitness for purpose shall apply.
Preface
Content of this Manual
Chapter 1. Introduction. Describes CIMPLICITY functionality and discusses the
various type of redundancy.
Chapter 2. CIMPLICITY Server Redundancy. Discusses the types of nodes used
for CIMPLICITY Server Redundancy and summarizes redundant system behavior.
Chapter 3. Hardware Requirements. Documents the hardware requirements for
CIMPLICITY Server Redundancy.
Chapter 4. Redundancy Configuration Procedures. Shows the configuration
procedures that support CIMPLICITY Server Redundancy.
Chapter 5. Recovery Procedures. Documents the recovery procedures for
CIMPLICITY Server Redundancy.
Chapter 6. Computer Cabling Redundancy. Shows you how the Computer
Cabling Redundancy option works.
Chapter 7. Monitoring Network and Socket Status. Documents the API
procedures you can implement to monitor the network and socket status.
Appendix A. Supported Communication Interfaces. Discusses the
communication interfaces supported by CIMPLICITY Server Redundancy.
Appendix B. Configuration Parameters. Documents the configuration parameters
needed for CIMPLICITY Server Redundancy.
Appendix C. Computer Cabling Redundancy Status Log Messages. Lists the
Status Log messages generated by the Computer Cabling Redundancy option.
Appendix D. Troubleshooting Database Merging. Lists issues and solutions for
database merging.
GFK-1353D
iii
Related Publications
For more information, refer to these publications:
CIMPLICITY HMI for Windows NT and Windows 95 Base System Users Manual
(GFK-1180). This book describes all the basic features of the CIMPLICITY HMI
for Windows NT and Windows 95 product.
CIMPLICITY HMI for Windows NT and Windows 95 Device Communications
Manual (GFK-1181). This book documents all the device communication enablers
for the CIMPLICITY HMI for Windows NT and Windows 95 product.
iv
GFK-1353D
Contents
Introduction
1-1
2-1
Hardware Requirements
3-1
4-1
GFK-1353D
Recovery Procedures
5-1
6-1
7-1
A-1
vi
GFK-1353D
B-1
C-1
Error Messages.......................................................................................................................C-1
Binding Failures.......................................................................................................C-1
Connection Failures .................................................................................................C-1
Socket Failures.........................................................................................................C-2
Missed Communications ..........................................................................................C-2
D-1
Index
GFK-1353D
Contents
vii
Introduction
Levels of Redundancy
The principle of redundancy in automated systems provides for switchover of functionality to
a backup component in case of failure of a primary component. The switchover is considered
automatic if no operator intervention is required. Redundancy applies to both hardware and
software, and implies minimal loss of continuity during the transfer of control between
primary (active) and redundant (backup) components. Redundant systems reduce single
points of failure, preventing loss of functionality.
For cell control systems, the major levels of redundancy include:
PLC
Computer networks
Computer
Each level of redundancy provides a failover system that allows continuous system activity
with minimal loss of data. The following sections briefly describe each level.
GFK-1353D
1-1
PLC Redundancy
PLC redundancy lets control transfer from a primary programmable controller to a redundant
one in case of failure.
When the primary PLC comes back on line, control can be transferred from the redundant
PLC back to the primary with minimal loss of data.
The redundancy can be synchronous or independent. Synchronous systems coordinate control
and handling of data between CPUs of the active and backup units, while in independent
systems each PLC acts like an active unit and is not constrained by the others.
Some CIMPLICITY HMI communication options support PLC redundancy. See the
CIMPLICITY HMI for Windows NT and Windows 95 Device Communications Manual (GFK1181) for more information.
Cabling Redundancy
Cabling redundancy involves separate physical connections to the same device.
The devices can be on a LAN (GENIUS, MAP, etc.) or may require serial connections (SNP,
CCM, etc.). Redundant cabling provides an alternate communication path to the device in
case of primary path failure. The implementation of cable redundancy with respect to host
monitoring/control systems differs with the device protocol involved.
Some CIMPLICITY HMI communication options support cabling redundancy. See the
CIMPLICITY HMI for Windows NT and Windows 95 Device Communications Manual (GFK1181) for more information.
1-2
GFK-1353D
Server Redundancy
Server redundancy involves a primary factory monitoring server and a secondary "Hot
Standby" server.
The secondary server is essentially a mirror image of the primary server, running alternate
monitoring/control processes and applications. Data collection is performed via independent
or shared network paths to the same devices, depending on the protocol. The characteristics
of the selected communications protocol(s) determine the details of the configuration.
Upon detection of failure of the primary server, the secondary server can assume control of
data collection, alarm functions, applications, and allow user access with minimal loss of
continuity. When the primary server comes back on line, control can be transferred back, and
the secondary server will resume its backup role.
GFK-1353D
Introduction
1-3
Server Redundancy
Server Redundancy
Server Redundancy is fully integrated with CIMPLICITY HMI softwares base system
functionality, enhancing its already powerful monitoring capability in a full range of
computer-integrated manufacturing environments.
1-4
GFK-1353D
Hardware Requirements
Server redundancy has hardware requirements for the computer and network.
1.
2.
3.
Using the Windows NT Performance Monitor observe, the Memory / Pages/Sec Counter. This value should be zero.
2-1
GFK-1353D
Server Redundancy requires that the primary and secondary computers run the Windows NT
operating system. Because Microsoft positions Windows NT for mission critical application
we recommend that Windows NT be used on your viewers as well. If you are using a
development viewer license with your redundant system, that viewer must be running
Windows NT and must have the redundancy option installed on it.
5.
6.
The servers should not use DHCP unless the leases never
expire.
2.
3.
Server redundancy requires a reliable network, if network reliability is an issue you should
consider implementing cabling redundancy between the servers and viewers.
Application Requirements
Server redundancy provides automatic synchronization of Point and Alarm Databases. Server
redundancy provides automatic switchover of CimView application using Points and Alarms.
Before you start building your application you should review the section in this manual
entitled Limitation of Server Redundancy, to verify that the features of CIMPLICITY that
you intend on using are supported in server redundancy.:.
2
A large volume of network traffic occurs between the primary and secondary computers. These two computers should
be plugged into a network switch that will isolate the inter-server communications from the rest of the network.
3
Using the Windows NT Performance Monitor observe, the Memory / Pages/Sec Counter. This value should be zero.
4
A large volume of network traffic occurs between the primary and secondary computers. These two computers should
be plugged into a network switch that will isolate the inter-server communications from the rest of the network.
2-2
GFK-1353D
CIMPLICITY will run your application as you design it. CIMPLICITY cannot automatically
fix your project if you design it incorrectly. Therefore, it is important that you design your
project to be mission critical from the ground up. Also, it is imperative that you test your
application in a server redundant environment with viewers during the development stage.
Only with a properly configured project can you switch on server redundancy and have it
work flawlessly.
We at GE Fanuc have designed many redundant systems using CIMPLICITY. We
understand the methodology and design techniques needed to build a robust system.
Therefore, we do recommend contacting your salesperson to obtain several days of design
consultation before you start your first project, and several days of on-site support during
deployment.
Scripting Requirements
The single biggest issue in building a server redundancy system is your user defined scripts.
During fail over, point values may be unavailable for a short time. Scripts must be written to
properly handle these intermittent periods and to exit cleanly. Scripts that depend on cleanly
exiting must be coded to trap the errors that can occur when a point goes unavailable. You
must test your scripts during fail over to verify they operate correctly.
2-3
Overview
CIMPLICITY HMI softwares Base System Functionality fully integrates Automatic Server
Redundancy. This functionality transfers control from a primary to a secondary server when
the primary goes down and, as a result, the connection between the primary and secondary is
severed.
Redundant features are integrated into Point Management, Device Communications, User
Registration and Alarm Management. The focus of redundancy in CIMPLICITY HMI
software centers on:
Data collection
Alarms
CIMPLICITY HMI also offers the capability for manual redundancy. Manual Server
Redundancy lets control be transferred from a primary to a secondary server, even if the
primary is active and the two servers are connected. Transfer capability includes:
For CIMPLICITY Server Redundancy, there are two configured computersthe primary
server and the secondary server.
A Primary Server is the Server that normally takes the primary role in a redundant
configuration. Each Primary Server has one Secondary Server.
A Secondary Server is essentially a mirror image of the Primary Server. It runs the same
version of the software as the Primary Server and communicates to the same devices. When
the Primary Server fails, the Secondary Server assumes control of the appropriate functions
that normally run on the Primary Server. A Secondary Server cannot be a primary
configuration node, and does not support any configuration functions.
2-4
GFK-1353D
Principles of Operation
This section will provide a general overview of how server redundancy operates so you can
accurately design your mission critical application,
Server Redundancy is configured from within the Workbench on the primary computer. The
Workbench will automatically distribute the configuration data to the secondary computer and
can control startup / shutdown of the pair.
In a normal state:
See Resetting the Primary Server after Recovery in the Recovery Procedures chapter of this
manual for more information about restarting the primary server.
GFK-1353D
2-5
2.
3.
Recipes
SPC
Tracker
BCEUI displays
4.
5.
Trending
SPC
6.
7.
8.
9.
During fail over, device values are not read and setpoints are
not written.
2-6
GFK-1353D
Data Collection
A runtime Point Management database that holds current data values is maintained on the
primary server and duplicated on the secondary server. The primary Point Manager
1.
2.
If device communications processes are running on the primary server, the corresponding
processes also run on the secondary server.
While the primary server is the master, the device communication modules on the secondary
server operate in standby mode to minimize the impact of redundant data collection on the
communications LAN or the programmable controller.
When the primary server terminates, the:
1.
2.
GFK-1353D
2-7
CB_PLC
(A) transmits
point,
alarm and user
registration data to
secondary server
(B).
( B) is connected
on standby.
Database Logging
Alarm Viewer
Base Control Engine
Point Manager
User Registration
Standby
Primary Server
(A) is down.
Secondary server
(B) automatically
controls all
functions.
CB_PLC
Offline
Database Logging
Alarm Viewer
Base Control Engine
Point Manager
User Registration
2.
3.
Normally CIMPLICITY reports point values as they change. After a failover, CIMPLCITY sends all the current point
values to all interested applications regardless of whether the value has changed. For example, events that trigger off
the point value being equal to some value may trigger again. It is your responsibility to design the project to function
properly under these circumstances.
2-8
GFK-1353D
4.
Alarm, User
Registration data
CB_PLC
System Manager
issues manual
command to restore
control to (A).
Database Logging
Base Control Engine
Point Manager
Point Data
CB_PLC
A
B
Alarm Viewer
User Registration
Database Logging
Alarm Viewer
Base Control Engine
User Registration
Point Manager
Collected Data
(A) sends
alarm, user
registration and point
data to secondary
server (B).
CB_PLC
A
B
(B) is on standby.
GFK-1353D
Database Logging
Alarm Viewer
Base Control Engine
Point Manager
User Registration
Standby
2-9
Setpoints
Users can make setpoint requests on either the primary or secondary server via:
CimView
While the primary server is running, all setpoints from the secondary server except those from
the Automatic Control Function will be routed to the primary computer. All setpoint
originating from Automatic Control Functions on the secondary will be discarded when the
primary is in control.
Lets consider the case of the Event Manager. The Event Manager runs on both the primary
and secondary computers. Events are triggered on both the primary and secondary computers.
All setpoint requests invoked from the action or script tied to the event will be ignored on the
slave computer. In other words, your scripts execute in tandem on both computers, but the
output to the points is processed only on the master computer.
A Custom Program would be a PTMAP API program written by you that executes as a
resident process within CIMPLICITY. Setpoints originating from this program will work the
same as the Event Manager.
Database Logging
When the primary server is in control, both the primary and the secondary server log alarm
and point data into their separate databases. As a result, if the primary fails the secondary
computer can continue to log data without loss of information.
When you bring a server back on line after a failure, a datamerge.exe utility:
1.
2.
2-10
GFK-1353D
2.
3.
4.
Primary Server
Secondary Server
Important: Viewer applications, such as Trending, that use logged data from a server will
not fail over to the database on the redundant server.
GFK-1353D
2-11
Alarm Management
The Alarm Manager on the primary server receives its updates from CIMPLICITY services
on both the primary and secondary servers. CIMPLICITY applications that generate alarms
(Point Management, Event Manager6 etc.) will not generate alarms when the corresponding
application is running on the primary server.
Exceptions to this rule are:
User Registration
A runtime database for users is maintained by User Registration on the primary server. This
information is passed to User Registration on the secondary server.
CimView
CimView applications running on the primary server or viewers receive point updates from
the primary point manager. CimView applications running on secondary server receives
updates from the point manager running on the secondary server. All setpoints are routed to
the primary point manager. When the primary server is lost, CimView applications on
viewers automatically begin receiving updates from the secondary point manager.
Important: Trend Controls on CimView screens that use logged data will not fail over to the
database on the redundant server.
Failover
CIMPLICITY Interprocess Communications has a built-in probing mechanism, independent
of TCP/IPs network probing mechanism. This was introduced so that you can configure a
smaller failover period than the TCP/IP default timeout period of 2 hours. This expedites
detection of a failed node for Server Redundancy. The default time out is 15 seconds.
From a Basic Script running in the Event Manager there is a way to force an alarm to be generated when executed on
the Slave Computer. Consult the documentation for AlarmGenerate in the Basic Control Engine Language Reference
Manual.
2-12
GFK-1353D
2.
2.
The functions reside in the Redundancy.dll and can be called by any programming language,
like the Basic Control Engine, that is capable of calling a DLL entry point.
GFK-1353D
2-13
If on the
Primary
Secondary.
int redundant_remote_index()
Returns the index of the global point element that has the status of the remote device
Returns
If on the
Primary
1
Secondary.
You, the system manager, will configure a specific global point and provide the logic to
determine when a changeover will occur. Basically the logic can be whatever you want, as
long as it is running as part of the project.
2-14
GFK-1353D
Note: Aids that are in your CIMPLICITY HMI directory, if you installed the server
redundancy option include:
Remote PTM
Local PTM
A global BOOLEAN array point of two (2) elements indicates the status of the device
connection.
The value of:
1
0
GFK-1353D
1.
2.
Be a Boolean point
3.
Have two (2) elements (for example, the status of the device
on the primary server and the status of the device on the
secondary server)
4.
Be a global point
2-15
Data collection
If the primary server stops collecting data from one device, but is still running and
communicating with the secondary computer, there is no automatic fail over.
Under these circumstances or for whatever reasons you specify, you can manually transfer
point management from the primary to the secondary server.
After the transfer the:
Database logging
Alarm viewer
Devcom
2-16
1.
2.
3.
failover_data_collection()
Specify what actions should occur if the point changes from 1
to 0 through a Basic script.
GFK-1353D
Example
Your primary server is connected to a PLC for a conveyor belt called CB_PLC and a
CimView screen.
You have configured a global Boolean point called CB_PLC that:
The primary server stops collecting data from the CB_PLC device.
The system manager is alerted and switches data collection to the secondary server.
The secondary server takes over point management function.
Example of Manual Data Collection Transfer Function
(Manual Server Redundancy)
CB_PLC
Boolean point
CB_PLC changes
from 1 to 0.
Database Logging
Alarm Viewer
Base Control Engine
Point Manager
User Registration
Standby
CB_PLC
GFK-1353D
Point Manager
Database Logging
Alarm Viewer
Base Control Engine
User Registration
2-17
2.
3.
failover_project ()
Specify what actions should occur if the point changes from 1
to 0 through a Basic script.
Example
Your primary server is connected to a PLC for a conveyor belt called CB_PLC and a
CimView screen.
You have configured a global Boolean point called CB_PLC that:
2-18
GFK-1353D
Primary server
(A) is off line.
CB_PLC
Standby
Database Logging
Alarm Viewer
Base Control Engine
Point Manager
User Registration
Secondary server
(B) becomes
active.
CB_PLC
Off line
GFK-1353D
Database Logging
Alarm Viewer
Base Control Engine
Point Manager
User Registration
2-19
Hardware Requirements
GFK-1353D
3-1
Redundancy Configuration
Procedures
GFK-1353D
1.
A project
2.
3.
Device communications
4.
Global points
4-1
Configuring a Project
The first step in using server redundancy is to configure your project to be redundant.
You use the Project Properties window to tell the primary server where to send files and
screens and collect data (after a failure) from the secondary server.
To configure the path from the primary to secondary server:
1.
2.
3.
4.
5.
4-2
GFK-1353D
6.
Computer name
Project path
Configuration files and screens are copied from the primary server to the Project path
whenever a Configuration Update is performed.
Important: Make sure you configure the logging setup on both the primary and secondary
server through the Database Logger in the CIMPLICITY HMI Workbench. See "Managing
Database Logging" in the CIMPLICITY HMI Base System Users Manual.
Configuring Networks
The second step when configuring a base system for server redundancy is to configure and
verify the network.
Host Names
Important: SR requires that all computers (primary, secondary and viewer) must have their
names and IP addresses configured and these names must match the actual computer names.
You may configure the host names in DNS, WINS or in the local host file on each computer,
depending on the networking resources available at your site. SR will not function correctly if
this information is not configured. If you do not understand network configuration you should
obtain the services of someone that does.
Once the configuration is complete, the following tests should be run.
GFK-1353D
1.
From primary computer ping primary, secondary and all viewers by name and by address.
2.
From secondary computer ping primary, secondary and all viewers by name and by
address.
3.
From each viewer, ping primary and secondary by name and address.
4.
4-3
The following example demonstrates how to ping by name, by address and determine the
computer name.
C:\WINNT\system32>ping albsagp2
C:\WINNT\system32>ping -a 3.26.4.215
C:\WINNT\system32>
4-4
Ping by Name In the above example ping albsagp2 first translates albsagp2 to an IP
Address and then verifies communication to the computer. In the above example, we
discover the albsagp2 has an IP address of 3.26.4.215. We also determine the time
required to Ping. This number must be less than 10ms between primary and secondary
and less than 30ms between viewers. The successful completion of this step verified that
the network software can convert a hostname to an IP Address.
Ping by Address The next step is to ping by address. Here we type ping a
3.26.4.215. We obtained the IP Address by looking at the output of ping albsagp2.
This step verifies that the network software can convert the IP Address back to the same
node name as entered in the first step. If you obtain a different IP Address back this may
indicate that you have duplicate entries for the IP Address in you network lookup tables.
This must be corrected before continuing.
Continue Pinging the remaining computers in your list in this example we just ping one
computer, you would continue and ping the other computers (secondary, viewers, etc)
Determine Computer Name The final step on each computer is to determine if the
systems computer name is the same as the computer name configured in the network.
Typing set computername will return the current setting. This setting must match the
name returned in the above two tests. If not this must be corrected by either changing
your computername or changing the network software configuration before continuing.
GFK-1353D
Keep Alives
Important: In order for SR to work all computers in the system (servers and viewer) must
have the TCP/IP Keep Alive Configured.
Windows NT
To activate the Keep Alive mechanism on Windows NT:
1.
2.
3.
4.
5.
2.
3.
this value to 1.
GFK-1353D
4-5
Unsolicited Data
The Device Communications module receives and processes unsolicited data reported from
factory devices.
Unsolicited data must be directed to the to the secondary server in addition to the primary
server, so it can be processed by the Device Communications/Point Manager on the secondary
server when the primary server fails.
Type is virtual
Class is Digital
The current Primary Point Manager will only change the values. This implies that point
updates to the global points will occur when:
4-6
GFK-1353D
Step 1: Display the System DSN tab of the ODBC Data Source Administrator:
1.
2.
The first time you select the System DSN tab, it will have the same CIMPLICITY HMI SQL
Server entry on both the primary and secondary server. It is called CIMPLICITY SQL Server
Logging.
GFK-1353D
4-7
to open the
The Create New Data Source dialog box displays a list of drivers installed in your
network, from which you select the appropriate driver.
4-8
GFK-1353D
2.
3.
Press Finish
3.
4.
5.
6.
Press OK.
GFK-1353D
4-9
Step 4: Configure the second server network, address and database name:
On the Second server
1. Press Configure
on the System DSN tab of the
ODBC Data Source Administrator dialog box, if you are
configuring a driver that already exists.
An ODBC SQL Server dialog box appears if you pressed Configure or if you
selected a new driver in the Create New Data Source dialog box
4-10
GFK-1353D
2.
3.
4.
5.
6.
Press OK.
GFK-1353D
4-11
2.
Select Properties.
3.
4.
5.
Press Settings.
4-12
GFK-1353D
6.
7.
8.
9.
GFK-1353D
4-13
12. Repeat these steps for the other three tabs so you will have
configured all four tabs:
Point Logging Primary (master)
Point Logging Second (slave)
Alarm Logging - Primary
Alarm Logging - Second
Important: On each tab, make sure that you select the correct data source for the
computer (master / slave) that the tab represents.
13. Press OK or select the Parameters tab. CIMPLICITY HMI
will validate your entries.
If the Data Logger is unable to connect to the selected database, validation fails.
Guidelines and notes about specific data sources:
1.
2.
If you are connecting to a SQL Server, you may be prompted for a database
name during validation.
Oracle Database
You may see the ODBC data source that you created for Oracle.
You may be prompted for a Server ID during validation. Enter the Alias Name
for the Oracle database in this field.
4-14
GFK-1353D
Second Server
Primary Server
1
2
Select the System
DSN tab of the ODBC
Data Source
Administrator (2).
The same CIMPLICITY
entries display on the
primary and second
server.
4
4
Click Options to expand the dialog
box..
Enter
Server,network
address and
database name in
the expanded
ODBC Setup (5).
Second
3.26.5.28
CIMPPROJECT
Local
3.26.5.25
CIMPPROJECT
5
6
GFK-1353D
4-15
Recovery Procedures
Normal Operating Procedures
.Normal operating procedures in server redundancy involve:
This section describes how to insure that the operating procedures perform smoothly.
2.
Select Run.
The Redundant project start dialog box opens.
3.
4.
GFK-1353D
5-1
2.
Select Stop.
The Redundant project start dialog box opens.
3.
3.
Important: The important thing to realize here is that you should NOT start on the Primary
using the CIMPLICITY Options under most circumstances. This is because the Workbench:
A. Allows you to start up both systems simultaneously, and coordinates that startup.
If you use startup options on the primary, you need to wait for the primary to
finish its startup and then start the secondary.
B. Will update the slave node as required before starting the project. If you dont
do this, you run the risk of having the master and slave nodes out-of-sync.
5-2
GFK-1353D
GFK-1353D
Recovery Procedures
5-3
Note: Between the time that the primary server is lost and before the secondary server takes
over, users may notice an interruption in system performance. During this time, point values
will not be updated, setpoints and alarm acknowledgments will not complete, and users will
not be able to log on or off.
Alarm Management
The Alarm Manager on the secondary server becomes the primary Alarm Manager. No alarm
data is lost because the Alarm Manager on the primary server continually updated the alarm
list for the Alarm Manager on the secondary server. The new primary Alarm Manager will
now process all alarm updates and provide alarm information to all interested processes.
User Logons
The User Registration process on the secondary server becomes the primary User Registration
process. No user registration data is lost because User Registration on the primary server
continually updated the user list for User Registration on the secondary server. The new
primary User Registration will now process all user logins and logouts and provide
information on user views.
5-4
GFK-1353D
Runtime Interfaces
When the primary server fails, all CimView, Alarm Viewer and Point Control Panel sessions
running on that server are lost.
Server failure is detected by the IPC Router, which is the communications process that runs on
each server.
1.
The Router sets up links to each server in the system and sends
messages to each node at a set interval.
The probe interval is defined in REDUND_PROBE_DELAY
which is set to 1000 milliseconds by default.
2.
3.
GFK-1353D
Recovery Procedures
5-5
2.
3.
4.
5.
Note: The Alarm Manager and User Registration on the primary and secondary servers will
automatically resynchronize themselves to their primary and secondary roles when the primary
server initially comes on line.
The following will occur as part of the reset:
The Point Manager on the primary server will resume its primary role, and will initiate device
communications modules on the primary server to start collecting data.
With <timestamp> as the date and time the file was created the
ptnr_<timestamp>.log records on the:
5-6
GFK-1353D
2.
3.
4.
dest
time1
time2
Important: You can only run datamerge.exe on the primary server - that is, the server
that has all of the ODBC data sources defined. Therefore, the primary server has to be running
before you begin.
As it executes, the datamerge.exe utility:
1.
2.
3.
4.
5.
Note: When you run the datamerge.exe utility with specific start and end times, the
ptnr_timestamp.log files on the secondary and primary servers are not used.
GFK-1353D
Recovery Procedures
5-7
Process Failures
A process failure occurs when a single process on a server fails. If this occurs on a primary
server, the recovery method depends on which process failed.
2.
3.
4.
5-8
GFK-1353D
Network Failures
Network partition occurs when the computer network fails. Each CIMPLICITY server
continues to perform its own functions, but there is no communications between the servers.
Secondary servers will perform their automatic takeover procedures. In other words, both the
primary and secondary servers act as master.
To make one server master when two are performing automatic takeover:
1.
2.
3.
Note: The above procedure assumes that the CIMPLICITY HMI project is still running on
the primary server. If the project has been shut down, then use the normal startup procedures
to restart the project on both the primary and secondary server.
GFK-1353D
Recovery Procedures
5-9
A computer with dual IP addresses will be able to continue to communicate with other
computers with dual IP addresses even if one of the Ethernet network connections is lost.
A computer with a single IP address will be able to communicate with a computer with dual
IP addresses. If the Ethernet connection that the computer with the single IP address is using
is lost, then all communication to the dual IP address computer will be lost.
When a CIMPLICITY HMI project detects that it is sending a message to a computer that has
Computer Cabling Redundancy, the project sends duplicate messages to each IP address.
CIMPLICITY HMI software on the receiving computer processes the first message and
deletes the second one.
GFK-1353D
6-1
Functionality Limits
The following functionality limitations apply for Computer Cabling Redundancy:
6-2
GFK-1353D
Hardware Requirements
CIMPLICITY Computer Cabling Redundancy supports two network interface cards in each
computer.
Each card must be configured for a different sub-network.
Each card must be connected to a different physical network. This is needed to provide for a
backup network in the case that one of the networks is lost.
GFK-1353D
6-3
Entering IP Addresses
For each computer that will be supporting CIMPLICITY HMI Computer Cabling
Redundancy you will need to enter all the IP addresses used by that computer.
The format for the cimhosts.txt file is:
<IP address> <hostname>
where <IP address> is an IP address for a computer and <hostname> is the name of
the computer.
Example
A sample entry would be:
3.26.5.5 alnt37
6-4
GFK-1353D
GFK-1353D
Value
Output
Print errors
6-5
6-6
GFK-1353D
The Socket Status API provides more detail about each of the
connections in use by CIMPLICITY HMI.
The APIs are callable from any programming language that can call exported C functions
from a DLL. This includes the CIMPLICITY BASIC Control Engine.
The functions reside in the redwinsock.dll DLL.
In the api\redundant_api directory under the CIMPLICITY HMI root directory are two
samples programs that demonstrate the APIs.
IP Status API
Use the Computer Cabling Redundancy IP Status API to monitor the state of connections to
IP addresses.
The following BASIC script generates or clears an alarm depending upon the state of the
connection to an IP address.
GFK-1353D
7-1
Sub Main()
Dim data As Long
Dim node As String
Dim ipAddress As Long
Dim state As Long
Dim n256 As Long
n256 = 256
data = InitSocketChange()
If data <> 0 Then
While 1
i = WaitSocketChange(data, 5000)
If i = 1 Then
node = Space(256)
While GetNextSocketChange(data, node, ipAddress, state)
ipStr$ = CvtIPAddress(ipAddress)
message$ = "IP address " & ipStr$
refId$ = ipAddress
If state = 2 Then
AlarmUpdate "", "IPALARM", "$SYSTEM", AM_RESET_M, _
message$, "", refId$
Else
AlarmGenerate "", "IPALARM", "$SYSTEM", message$, _
"", refId$, True
End If
7-2
GFK-1353D
node = Space(256)
Wend
End If
Wend
i = CloseSocketChange(data)
End If
End Sub
CvtIPAddress = mod1 & "." & mod2 & "." & mod3 & "." & mod4
End Function
GFK-1353D
7-3
InitSocketChange
Syntax
void *InitSocketChange();
Description
Comments
Example
Dim data As Long
data = InitSocketChange()
WaitSocketChange
Syntax
Description
Comments
Example
Dim data As Long
i = WaitSocketChange(data, 5000)
CloseSocketChange
Syntax
Description
Comments
Example
Dim data As Long
i = CloseSocketChange(data)
7-4
GFK-1353D
GetNextSocketChange
Syntax
Description
Comments
Example
Dim data As Long
Dim node As String
Dim ipAddress As Long
Dim state As Long
Node = Space(256)
While GetNextSocketChange (data,node,ipAddress,state)
Wend
GFK-1353D
7-5
#include <string.h>
#include <inc_path/cor.h>
#include <inc_path/redwinsock.h>
TCHAR *socketUse[] =
{
_T("None"),
_T("Listen"),
_T("Connect"),
_T("Accept"),
};
TCHAR *socketState[] =
{
_T("None"),
_T("Connecting"),
_T("Connected"),
};
int main()
{
HANDLE dwChangeHandle;
dwChangeHandle = FindFirstSocketChangeNotification();
if(dwChangeHandle == INVALID_HANDLE_VALUE)
return 0;
HANDLE objectArray[1];
DWORD objectCount = 1;
objectArray[0] = dwChangeHandle;
switch(dwWaitStatus)
{
case WAIT_OBJECT_0:
7-6
GFK-1353D
unsigned int i;
for(i = 0; i < findSocketData.connectionCount; i++)
{
if(findSocketData.sockets[i].isOpen)
{
printf("\t%2d: %-10s ", i,
socketState[findSocketData.sockets[i].socketState]);
printf("%-9s ", findSocketData.sockets[i].hasException
? "Exception" : "");
if(findSocketData.socketUse == SOCKET_ACCEPT
|| findSocketData.socketUse == SOCKET_CONNECT)
{
print_sockaddr_in(
&findSocketData.sockets[i].partnerAddr);
}
else if(findSocketData.socketUse == SOCKET_LISTEN)
{
print_sockaddr_in(
&findSocketData.sockets[i].localAddr);
}
printf("\n");
}
}
}
GFK-1353D
7-7
} while(FindNextSocket(dwChangeHandle, &findSocketData));
printf("\n");
FindCloseSocket(dwChangeHandle);
}
break;
default:
FindCloseSocketChangeNotification(dwChangeHandle);
return 0;
}
dwWaitStatus = WaitForMultipleObjects(objectCount,
objectArray,
FALSE,
INFINITE);
if(FindNextSocketChangeNotification(dwChangeHandle) == FALSE)
{
FindCloseSocketChangeNotification(dwChangeHandle);
return 0;
}
}
return 0;
}
7-8
GFK-1353D
FindFirstSocketChangeNotification
Syntax
HANDLE FindFirstSocketChangeNotification();
Description
Comments
Example
HANDLE dwChangeHandle;
dwChangeHandle = FindFirstSocketChangeNotification();
if(dwChangeHandle == INVALID_HANDLE_VALUE)
return 0;
FindNextSocketChangeNotification
Syntax
BOOL FindNextSocketChangeNotification (
HANDLE changeHandle);
Description
Comments
Example
HANDLE dwChangeHandle;
if(FindNextSocketChangeNotification(dwChangeHandle)==FALSE)
{
FindCloseSocketChangeNotification(dwChangeHandle);
return 0;
}
GFK-1353D
7-9
FindCloseSocketChangeNotification
Syntax
BOOL FindCloseSocketChangeNotification (
HANDLE changeHandle);
Description
Comments
Example
HANDLE dwChangeHandle;
FindCloseSocketChangeNotification(dwChangeHandle);
FindFirstSocket
Syntax
BOOL FindFirstSocket (
HANDLE changeHandle,
Struct SocketFindData *findSocketData);
Description
Comments
Example
HANDLE dwChangeHandle;
struct SocketFindData findSocketData;
if(FindFirstSocket(dwChangeHandle, &findSocketData))
;
7-10
GFK-1353D
FindNextSocket
Syntax
BOOL FindNextSocket (
HANDLE changeHandle,
Struct SocketFindData *findSocketData);
Description
Comments
Example
HANDLE dwChangeHandle;
struct SocketFindData findSocketData;
do
{
} while(FindNextSocket(dwChangeHandle, &findSocketData));
FindCloseSocket
Syntax
BOOL FindCloseSocket (
HANDLE changeHandle);,
Description
Comments
Example
HANDLE dwChangeHandle;
FindCloseSocket(dwChangeHandle);
GFK-1353D
7-11
Appendix A - Supported
Communication Interfaces
About Supported Communication Interfaces
This appendix documents the redundant communication interfaces supported by Server
Redundancy.
The supported redundant communication interfaces are:
Series 90 TCP/IP
CCM2
Genius
SNPX
Allen-Bradley Communications
APPLICOM
DDE Client
Modbus Plus
Modbus RTU
Modbus TCP/IP
Point Bridge
In addition, CIMPLICITY HMI for Windows NT and Windows 95 supports the development
of Server Redundant Device Communication Toolkit drivers. Use the heartbeat function in
the enabler to determine if the secondary server can communicate with its configured devices.
Further details can be found in the CIMPLICITY HMI for Windows NT and Windows 95
Device Communications Toolkit Application Developer Guide (GFK-1202).
GFK-1353D
A-1
The primary and secondary servers are on the same Ethernet LAN connected to a single
Ethernet card in the programmable controller.
PLC redundancy
Cabling redundancy
The primary and secondary servers are on the same Ethernet LAN connected to Ethernet cards
in the primary and backup programmable controllers.
For detailed information on the configurations supported by the Series 90 TCP/IP Redundancy
Communication options, see the CIMPLICITY HMI for Windows NT and Windows 95 Device
Communications Manual (GFK-1181)
A-2
GFK-1353D
CCM2 Communications
Server redundant computer configurations that use the CCM2 Communications option must be
configured as in the following diagram:
The primary and secondary servers must have independent cable paths to the PLC. The PLC
must have two serial ports available for CCM2 communications. Both of the PLCs serial
ports must be configured with the same CPU ID.
The recommended configuration is to use the CMM module in CCM2 mode. Both ports on
the CMM module can be configured for CCM2 communications.
Genius Communications
Server redundant computer configurations that use the Genius Communications option must
be configured as in the following diagram:
The primary and secondary servers must have different PCIM addresses. Unsolicited
datagrams sent from the PLC must be sent to both the primary and secondary servers.
GFK-1353D
A-3
SNPX Communications
Server redundant computer configurations that use the SNPX Communications option must be
configured as in the following diagram:
The primary and secondary servers must have independent cable paths to the PLC. The PLC must have
two serial ports available for SNPX communications. Both of the PLCs serial ports must be
configured with the same CPU ID.
Allen-Bradley Communications
Server redundant computer configurations that use the Allen-Bradley Communications option
must be configured as in the following diagram:
Communications can be done over an Ethernet network or over a Data Highway Plus network
using Allen-Bradley 1784 KTX cards. Unsolicited data sent from the PLC must be sent to
both the primary and secondary servers.
A-4
GFK-1353D
1
7
8
4
Secondary
Server
1
7
8
4
KT
KT
PLC
DH+
Both 1784-KT/B cards are on the same network and must be at different Station Addresses.
Unsolicited data sent from the PLC must be sent to both the primary and secondary servers.
APPLICOM Communications
If the protocol you are implementing via the APPLICOM Communications option allows
multiple servers on the bus, then Server redundancy is supported for that protocol.
GFK-1353D
A-5
S
A
8
5
Secondary
Server
S
A
8
5
Programmable
MB+
Controller
Both SA85 cards are on the same network and must be at different Node Addresses.
Unsolicited data sent from the PLC must be sent to both the primary and secondary servers.
Unsolicited data sent from the PLC must be sent to both the primary and secondary servers.
A-6
GFK-1353D
The primary and secondary servers are on the same Ethernet LAN connected to a single
Ethernet card in the programmable controller.
Point Bridge
Point Bridge communications is fully supported in server redundant computer configurations.
GFK-1353D
A-7
Appendix B - Configuration
Parameters
Failover Rate Configuration
The default value for the failover period is 15 seconds. It can be increased or decreased,
depending on the needs of the application. The failover rate should never be modified to less
than 3 seconds.
The failover period is defined as:
REDUND_PROBE_DELAY * (REDUND_PROBE_COUNT + 1)
where <millisec> is the number of milliseconds for the probe delay and <count> is the number
of probes to make.
Sample entries for these parameters would be:
REDUND_PROBE_DELAY|3|1000
REDUND_PROBE_COUNT|3|3
GFK-1353D
1.
Type Cd master
2.
3.
4.
5.
6.
B-1
Slave Startup
If the project on both the primary (master) and secondary (slave) servers is configured to start
at boot, you can use the SLAVE_STARTUP_TIMEOUT global parameter to delay starting
the project on the secondary server until after the project on the primary server starts. This
helps avoid race conditions between the two servers when they are trying to determine which
server is the master.Timeout.
If you do not define this global parameter, the default time is 0 (zero) minutes.
The format for this parameter is:
SLAVE_STARTUP_TIMEOUT|1|<time>
where <time> is the time in minutes to wait before starting the project on the secondary
server.
For example, to delay the project startup on the secondary server by 2 minutes, enter the
following in the global parameters file:
SLAVE_STARTUP_TIMEOUT|1|2
B-2
GFK-1353D
Error Messages
The following is a list of errors you may encounter in the CIMPLICITY Status Log relating to
Computer Cabling Redundancy.
Binding Failures
This error should only occur if a non-CIMPLICITY HMI communication program is using IP
ports in the same range as the Computer Cabling Redundancy option.
Failed to bind to port
CIMPLICITY HMI attempts to use the same IP port on both network interface cards (NIC).
If this error occurs try changing the range of IP ports that Computer Cabling Redundancy is
using. See the section on TCP/IP Port Use in chapter 6.
Connection Failures
These errors occur when the Computer Cabling Redundancy option starts a connection with
another computer:
Failed to receive connection ID: invalid ID
Failed to complete connection: <IP Address>
Failed to receive connection ID: invalid ID
Failed to receive connection ID: timeout
Failed to receive connection ID: exception
Failed to receive connection ID: recv failed
When the option starts a connection with another computer it exchanges some identifying
information. If this information is not received the connection will fail.
This is typically caused by an incorrect configuration. If the local computer believes that the
remote computer is supporting Computer Cabling Redundancy but the remote computer is not
GFK-1353D
C-1
configured for Computer Cabling Redundancy, then this error will occur. This error can also
occur if the network is broken during the connection setup time.
Socket Failures
This error occurs when the local socket is waiting for data:
Error performing select on socket
This error should never occur.
Missed Communications
This error occurs when network traffic is not being received from the remote computer:
Missed hearing from partner: < IP Address>
If this error occurs and the network is functioning you should check the PING_INTERVAL and
PING_COUNT parameters in the cimhosts.txt file. You may need to increase them
depending on the load on your computers or network.
C-2
GFK-1353D
Appendix D - Troubleshooting
Database Merging
Symptoms
WARNING:Unable to translate
SITE_ROOT.dlrp_init_merge
Fri Apr 10 13:25:47 1998 failure 189 <No
Prname> dlrp_init_table 93
Error 2 from fio file data_field: No such
file or directory
Error of type COR_SC_ERR:source 130 code
2 COR_logstatus FATAL error resulting in
exit
Solution
GFK-1353D
1.
2.
3.
4.
D-1
Problem
Symptoms
CIpcPortShm::Register
m_ConnOut.Attach Err:2
Solution
D-2
1.
2.
3.
4.
5.
Problem
Symptoms
Solution
Problem
Using data source with the same names on the primary and
secondary servers that are pointing to different databases.
Symptoms
Solution
Problem
Symptoms
Solution
Problem
Symptoms
Solution
GFK-1353D
Index
A
About
Computer cabling redundancy 6-1
Supported communication interfaces, Server
redundancy A-1
Alarm management
Operation during failover 5-4
Redundant system behavior 2-14
Allen-Bradley communications
Redundant configuration A-4
Allen-Bradley Data Highway Plus
Redundant configuration A-5
Application
Requirements for server redundancy 2-3
Scripting requirements for server redundancy 2-3
Applications
Database logging requirements for server redundancy
2-3
Network configuration requirements for server
redundancy 2-3
Primary and secondary computer requirements for
server redundancy 2-3
Time synchronization requirements for server
redundancy 2-4
APPLICOM
Redundant configuration A-5
Automatic Redundancy
Introduction 2-5
Recover from process failure 5-8
B
Base System
Order of configuration 4-1
Boot
Configuring the project to start at 5-3
GFK-1353D
D
Data collection
Redundant system behavior 2-8
Database Logging
Configuration 4-7
FindNextSocket 7-11
FindNextSocketChangeNotification 7-9
Functionality limitations
Computer cabling redundancy 6-2
Server redundancy 2-6
Functions
IP status API 7-4
Socket status API 7-9
Error messages
Computer cabling redundancy C-1
Computer cabling redundancy, Binding failures C-1
Computer cabling redundancy, Connection failures C1
Computer cabling redundancy, Missed
communications C-2
Computer cabling redundancy, Socket failures C-2
Hardware
Computer requirements for server redundancy 2-1
Network requirements for server redundancy 2-2
Hardware requirements
Computer cabling redundancy 6-3
Server redundancy 3-1
F
Failover
Alarm management operation during 5-4
Automatic-overview 2-6
Device communications operation during 5-4
Point management operation during 5-4
Runtime interfaces during 5-5
Server redundancy 2-14
System operation during 5-4
User logons operation during 5-4
Failover rate configuration
Computer cabling redundancy 6-5
Server redundancy B-1
Failure
Detecting cause on primary server 5-5
Network failure exceptions 5-9
Process failure exceptions 5-8
Failure exceptions
Server redundancy 5-8
Failure recovery limitations
Computer cabling redundancy 6-2
Server redundancy 2-8
FindCloseSocket 7-11
FindCloseSocketChangeNotification 7-10
FindFirstSocket 7-10
FindFirstSocketChangeNotification 7-9
ii
G
Genius
Redundant configuration A-3
GetNextSocketChange 7-5
Global Point
Server redundancy configuration 4-6
I
InitSocketChange 7-4
Integration
Redundant features 2-5
Introduction
Server redundancy 2-5
IP addresses
cimhosts.txt 6-4
IP status API 7-1
CloseSocketChange 7-4
Functions 7-4
GetNextSocketChange 7-5
InitSocketChange 7-4
WaitSocketChange 7-4
L
Limitations
Circumvent in automatic redundancy 5-8
Limitations, Failure recovery
Computer cabling redundancy 6-2
Limitations, Functionality
Computer cabling redundancy 6-2
M
Manual Redundancy
Introduction 2-5
GFK-1353D
Missed communications
Error messages, Computer cabling redundancy C-2
Modbus RTU
Redundant configuration A-6
Modbus TCP/IP
Redundant configuration A-7
Modubs Plus
Redundant configuration A-6
Monitoring API
Computer cabling redundancy 7-1
Project
Configuration in server redundancy 4-2
Configure project to start at boot 5-3
Start a redundant project 5-1
Start from a secondary server 5-2
Stop a redundant project 5-2
Project Failover
Automatic-overview 2-6
REDUND_LINK_SLEEP B-2
REDUND_PROBE_DELAY B-1
REDUND_PROBE_INTERVAL B-1
Redundancy
Cabling 1-2
CIMPLICITY HMI, definition of 1-4
General definition 1-1
Host Computer network 1-3
PLC 1-2
Server 1-3
Redundant configuration
Allen-Bradley communications A-4
Allen-Bradley Data Highway Plus A-5
APPLICOM A-5
CCM2 A-3
DDE client A-5
Genius A-3
Modbus Plus A-6
Modbus RTU A-6
Modbus TCP/IP A-7
Point bridge A-7
Series 90 TCP/IP A-2
Series 90 TCP/IP redundancy A-2
SNPX A-4
Redundant system behavior
Alarm management 2-14
CimView 2-14
Data collection 2-8
Setpoints 2-12
User registration 2-14
Requirements for Server Redundancy
Application 2-3
Computer 2-1
Database logging 2-3
Network 2-2
Network configuration 2-3
Scripting 2-3
Time synchronization 2-4
Use of primary/secondary computers 2-3
Reset
After automatic failover 2-6
Primary server after recovery 5-6
Re-synchronize
Database logging files 5-6
Network
Configuration requirements for server redundancy 2-3
Requirements for server redundancy 2-2
Server redundancy failures 5-9
Network configuration
Server redundancy 4-3
Normal operating procedures
Server redundancy 5-1
Normal operating procedures, Startup and shutdown
Server redundancy 5-1
number_of_ports 6-6
O
Overview
Before you start 2-1
P
ping_count 6-5
ping_interval 6-5
PLC redundancy 1-2
Point bridge
Redundant configuration A-7
Point management
Operation during failover 5-4
Primary
sysnchronize databases on the primary and secondary
nodes 5-7
Primary and Secondary Computers
Requirements for server redundancy 2-3
Primary Server
Automatic failover overview 2-6
Definition 2-5
Failover return overview 2-6
Reset after recovery, Server redundancy 5-6
Problems
Database merging D-1
Procedures
Server redundancy 4-1
Process Failures
Server redundancy 5-8
GFK-1353D
Index
iii
Rules of operation
Computer cabling redundancy 6-2
Runtime interfaces
Operation during failover 5-5
S
Scripting
Requirements for server redundancy 2-3
Secondary
Synchronzie databases on the primary and secondary
nodes 5-7
Secondary Server
Automatic failover overview 2-6
Definition 2-5
Failover return overview 2-6
Starting a project from 5-2
Series 90 TCP/IP
Redundant configuration A-2
Series 90 TCP/IP redundancy
Redundant configuration A-2
Server redundancy 1-3
Allen-Bradley communications A-4
Allen-Bradley Data Highway Plus A-5
APPLICOM A-5
CCM2 A-3
DDE client A-5
Definition for CIMPLICITY HMI 1-4
Devcom configuration 4-6
Failover 2-14
Failover rate configuration B-1
Failure recover limitiations 2-8
Functionality limitations 2-6
Genius A-3
Hardware requirements 3-1
Modbus Plus A-6
Modbus RTU A-6
Modbus TCP/IP A-7
Network configuration 4-3
Point bridge A-7
Series 90 TCP/IP A-2
Series 90 TCP/IP redundancy A-2
Slave startup timeout B-2
SNPX A-4
Supported communication interfaces, About A-1
TCP/IP keep alives, Windows 95 4-5
User registration synchronization B-2
Server Redundancy
Configuration procedures 4-1
Database logging files, Resynchronizing 5-6
Global point configuration 4-6
Process failures 5-8
Resetting primary server after recovery 5-6
Server RedundancyFailure exceptions 5-8
Server RedundancyNetwork failures 5-9
iv
T
TCP/IP keep alives
Windows 95 4-5
TCP/IP port use
Computer cabling redundancy 6-6
Time Synchronization
Requirements for server redundancy 2-4
U
User logons
Operation during failover 5-4
User registration
Redundant system behavior 2-14
User registration synchronization
Server redundancy B-2
W
WaitSocketChange 7-4
GFK-1353D