Communication Solutions for the Future

Converged Networks for Mission Critical Applications

White Paper

Converged Networks for Mission Critical Applications

Commtel White Paper

Abstract

Communication Network is a key business enabler for utility markets such as Oil & Gas, Power and Railway companies although it is not their core business. The communication systems are used to gather and transmit real time mission critical applications such as critical control system information as well as voice and video communication between sites, which are geographically spaced apart. Control systems are of utmost importance to enable the operator to monitor and control the operations and to immediately react in emergency situations. For example in Oil & Gas fields the control systems information from RTUs (interfaced to control the pumps and valves) will be connected to central monitoring facilities by means of the transmission networks via low speed data or Ethernet interfaces. Also mission critical applications like signalling in railways & airports and teleprotection for tripping the distance relays in power sector requires communication between various sites. Apart from the data communication, voice communication systems (PABX, UHF/VHF radios, etc.) are technologies that enable personnel to communicate and access information from anyone to anywhere. Including data and voice communications, the security systems are also becoming increasingly important in utilities. Security systems include video surveillance cameras and monitoring consoles, intrusion detection, and access security devices. These security devices are connected to a central monitoring center via transmission network helping to increase security. With more applications such as SCADA & DCS, voice and video becoming Ethernet based, IP convergence allows the simplification of multi network infrastructure into a single converged network that not only reduces capital and operational costs but also improves productivity. Ethernet is emerging as an economical solution for enterprise, carriers and industry communication platforms to transport lot of applications. Industrial Ethernet is very different to Enterprise Ethernet, which should be reliable and requires redundancy to operate reliably for supporting real time mission critical applications. With various Ethernet based devices interfacing to transport backbones, it is a challenge to develop a future proof system robust enough to keep data flowing even with multiple connection failures. This paper outlines what is required to establish a converged network (triple play services), which supports features such as fault recovery, error control, efficient bandwidth utilization, link optimization, etc. to provide the same type of trouble free communications that circuit switched networks provided for years.

-1-

Converged Networks for Mission Critical Applications

Commtel White Paper

Converged Networks

The end user requirements and critical communication needs have changed drastically from legacy voice and low speed data interfaces to bandwidth hungry applications, which are mostly Ethernet based. The boundaries between telecommunication systems and data networking applications are overlapping to some extent.

VOICE

DATA

VIDEO

Convergence on to IP Network

VOICE

DATA

VIDEO

Telephone Network

IP Network

Broadcast Network

Telephone Network

IP Network

Broadcast Network

Figure-1: Triple Play Services on Converged Networks

The integration of Ethernet based triple play services such as data, voice and video on to a converged network requires a switching infrastructure that can carry real time mission critical applications via dedicated paths or channels and handle each traffic type according to its unique requirements. So it has become necessary to select the correct system to interconnect the converged networks (IP based) across offices, campuses, stations, etc which are geographically spaced apart. One of the most future proof and unlimited bandwidth supporting media is optical fiber. Keeping in view the inherent advantages of using an optical fiber cable as transmission medium for utility applications, it is the most preferred and economical choice for building a telecommunication network. However, for supporting mission critical applications it is not going to help just by selecting fiber as media but it is necessary to select correct end device and technology which should be reliable and requires redundancy to operate. As the telecommunication network carries mission critical applications of the utility operators, it is imperative that the proposed system offers the following: Robust & Reliable Equipment For supporting mission critical applications the equipment should be highly robust and reliable for operating in the harsh environment (industrial applications) and should have a proven track record in similar applications. Resilience & High Network Availability The system and network design should offer self-healing protection and equipment should have high MTBF Real Time Transmission of Signals The system should be able to carry real time applications via dedicated paths or channels with minimum latency in signal transmission. Future Proof & Long Term Support The system design should cater to easy and in-service upgradation capable of supporting future data, voice and video applications at reasonable capital expenditure.

-2-

Converged Networks for Mission Critical Applications

Commtel White Paper

Ethernet over TDM is the Way Forward

Demand for the bandwidth has grown dramatically as a result of the way business environments are using telecommunication networks to improve operations. The development of enhanced applications along with new applications that converge data, voice and video have also helped in boosting the demand for bandwidth. Greater demands are being placed on the networks for supporting complexity and mission critical needs of the business environments while providing hassle free communications. With the explosion in Ethernet based services and applications ranging from purely data to voice and video, Ethernet is emerging as an economical solution for establishing converged networks. Ethernet by default is a connectionless protocol i.e., the connection is established only when required and is designed to carry packet-based data. Ethernet on its own provides no protection mechanisms, no end-to-end service monitoring, and no default Quality of Service (QoS). In Ethernet/IP transport networks one bandwidth pipe is used jointly by several applications as shown in Figure-2, one application can affect the other applications hence the arrival time of the packet is also not guaranteed, which is disastrous for real time mission critical applications.

Ethernet / IP Backbone
VLAN 1 VLAN 2 VLAN 3 VLAN 4 VLAN 1 VLAN 2 VLAN 3 VLAN 4

Ethernet Switch

Ethernet Switch

Figure-2: Data Flow in Ethernet/IP Transport Networks

Keeping in view the above-mentioned limitations of Ethernet/IP transport networks, for supporting the mission critical applications of industrial communication platforms, IP transport networks should be as reliable and dedicated as TDM based infrastructure. Time Division Multiplexing (TDM) is connection-oriented protocol i.e., connection is always available. The end-to-end dedicated traffic channel guarantees the flow of data from source to destination (Guaranteed QoS) as shown in Figure-3. The advances in technology have resulted in Next Generation SDH transmission equipment with Multi Service Provisioning Platform (MSPP), which not only provides the TDM services, but also Ethernet based services for fulfilling the end user requirements such as interconnectivity and extension of SCADA LAN, VoIP LAN, Enterprise Data LAN, CCTV/Security LAN, Video Conferencing LAN, Etc.

TDM Backbone
Appl. 1 VLAN 1 Appl. 1

Appl. 2

SDH Node

VLAN 2

SDH Node

Appl. 2

Appl. 3 Appl. 4

VLAN 3 VLAN 4

Appl. 3 Appl. 4

Figure-3: Data Flow in TDM Transport Networks

-3-

Converged Networks for Mission Critical Applications

Commtel White Paper

With the availability of Next Generation SDH equipment, provisioning of various Ethernet based services (end-to-end dedicated communication paths) is possible directly through Layer-1 and Layer-2 Ethernet interfaces of SDH for realizing reliable transmission network design. Ethernet over SDH (EoS) technology deploys GFP VCAT & LCAS schemes for realizing efficient, effective, scalable , and fault tolerant LAN extension among the stations. The process flow of Ethernet over SDH is shown in Figure-4.

User-to-Network Interface (UNI) Ethernet Ethernet

Ethernet Domain

SDH Domain

Optical Network

N x VC - 12

Ethernet

Ethernet Layer 2 Switch

Ethernet

Ethernet Over SDH Mapper (WAN Mapper)

N x VC - 12 TDM Multiplexer N x VC - 12 STM -n

Ethernet

Ethernet

Ethernet

Ethernet

N x VC - 12

Figure-4: Ethernet over SDH Process Flow

With advances in Ethernet over SDH technologies such as Layer-2 switching (between LAN & WAN), VLAN tagging and Rate Limitation, it is possible to interconnect and extend Ethernet in different configuration such as point-to-point, point-to-multipoint and multipoint-to-multipoint known as EPL, EVPL, EPLAN or EVPLAN. With availability of the above standards of EoS and inherent features of SDH, not only the WAN bandwidth of Ethernet data streams increased in multifold but also provided the protection of traffic by routing through different paths. The traffic can be protected in various configurations through the protection schemes of SDH such as MSP SNCP and MSP-Ring etc. or a combination of these , protection schemes with tiered/ring network architecture resulting in resilience and high network availability.

Comparative Analysis

As mentioned above, advantages of provisioning of Ethernet over TDM transport networks compared with Ethernet/IP transport networks are tabulated below:

Sr. No. 1

Ethernet over SDH (EoS) The EoS technology is based on concept of mapping Ethernet data packets on concatenated payloads of SDH signal, which are Virtual Containers (VC) with different capacities. The numbers of payloads decide the amount of WAN bandwidth allotted. This mapping is performed using ITU-T G.7041 GFP framing procedure with Virtual Concatenation (VCAT) of payloads as per ITU-T G.707.

Ethernet Switches with Optical Uplink The Ethernet switches with optical uplink ports are based on the concept of aggregation and largely depend upon switch fabric for effective aggregation and transport of ethernet data over uplink port. Typically uplink ports are 100/1000 Base-FX. Unlike in EoS, the WAN bandwidth (aggregating bandwidth) per port cannot be defined unless switch is managed and supports port configuration and link aggregation.

-4-

Converged Networks for Mission Critical Applications

Commtel White Paper

Sr. No. 2

Ethernet over SDH (EoS) In EoS the WAN bandwidth allotted is fixed, dedicated and assured on end-to-end basis. This ensures and guarantees the stable network on end-to-end basis. The EoS has robust error control on end-to-end basis at various stages of data mapping and multiplexing. This ensures that Ethernet connectivity is available even in case of deterioration of optical link albeit optical signal should be within the receiver sensitivity, which is much higher than native optical interfaces of Ethernet switches. The EoS uses SDH STM-N optical signal as aggregate transport carrier, which allows interconnecting stations with longer link lengths. The EoS services can be protected at two levels viz. Ethernet level and SDH (aggregate) level. At Ethernet level protections such as STP or RSTP are available while at SDH (aggregate) level protections such as MSP SNCP and LCAS is available. ,

Ethernet Switches with Optical Uplink The total uplink bandwidth is shared among different ports of switch.

The error control mechanism is not as robust as EoS.

The fiber interfaces of Ethernet switches support typically SH or LH/LX type of interfaces which limits the inter station distance. Ethernet protection schemes such as STP or RSTP is available which has high fault recovery time.

The EoS is tributary interface of SDH STM-N equipment which can be equipped with protected 1+1 configuration for all critical units such as cross connect unit, synchronisation unit, power supply as well as optical units with 1+1 path protection. This ensures that EoS delivery with highest availability on end-to-end basis and there can be no single point of failure, when SDH configured with equipment protection as well as path protection.

The Ethernet switch with optical uplink interfaces can also be equipped with protected configuration of switch fabric (control unit) & power supply units; however this type of configuration is available generally in high-end core/carrier switches.

As the EoS services are delivered by SDH nodes between different locations using various Ethernet provisioning schemes such as EPL, EVPL, EPLAN and EVPLAN, allows effective utilization of WAN bandwidth with QoS on individual Ethernet streams. The EoS supports aggregation at LAN (Access) & WAN (Trunk) level, which gives flexibility in extending Ethernet services. The SDH equipment supports both TDM as well as Ethernet services simultaneously. This allows interconnection of circuit switched devices such as 64k channels, PABX trunk lines (PRI), Video Conferencing trunk lines (PRI/BRI) as well as packet switched devices such as routers with nx64k interfaces, Intranet, MIS, SAP , etc.

There is no such Ethernet provisioning scheme options available.

The aggregation is possible only at uplink port level from different ports locally.

The Ethernet switch supports only packet switch devices while some high-end routers will have provision for nx64k and 2 Mbit/s interfaces with limited QoS.

-5-

Converged Networks for Mission Critical Applications

Commtel White Paper

Converged Network Architecture for Mission Critical Applications

The availability of different configurations of LAN services through next generation SDH equipment has resulted in endless possibilities of transporting different Ethernet based applications such as SCADA, VoIP CCTV and many more. Most of the recent , transmission networks deployed in utility sectors demand these applications from effective operations and maintenance point of view. With all these end user applications provisioned from next generation SDH the role Network Management Systems (NMS) also became very critical. Keeping in view the above facts, design of the transmission network architecture should be based on following:

High Capacity Optical TDM Transport Equipment (Next Generation SDH) Tiered Network or Ring Network (Flat) Architecture Redundant Configuration for Critical Control Units (Equipment Protection) Capable of Delivering Triple Play Services (Data, Voice & Video) Dedicated Ethernet and TDM Access Support of Standard Interfaces as per ITU-T Redundant Network Management System

The Figure-5 shows the typical architecture of the transmission network using the Ethernet over TDM technology (NG-SDH) along with legacy primary multiplexer equipment for delivering various LAN services and low speed data & voice interfaces for critical communications like signaling and teleprotection applications in utility sectors.

Station-1

Station-5

SCADA Server

CCTV Server

Station-2, 3 & 4
CCTV Cameras

SDH
STM-64 OR STM-16

SDH
STM-64 OR STM-16

SCADA LAN A SCADA LAN B

SCADA LAN

CCTV LAN Switch

SDH
STM-4 OR STM-1
Primary Multiplexer Voice I/F Data I/F

SCADA LAN A SCADA LAN B

NMS LAN Network Management System(Main)

Primary Multiplexer Voice I/F Data I/F

CCTV Cameras

Primary Multiplexer Voice I/F Data I/F

CCTV Cameras

Hotline Phone

Conference Subscriber Phone Phone

Data Line

Hotline Phone

Conference Subscriber Phone Phone

Data Line

Hotline Phone

Conference Subscriber Phone Phone

Data Line

Station-9

SCADA Server

CCTV Server

Station-6, 7 & 8
CCTV Cameras

SDH
STM-64 OR STM-16

SCADA LAN

CCTV LAN Switch

SDH
STM-4 OR STM-1
Primary Multiplexer Voice I/F Data I/F

SCADA LAN A SCADA LAN B

NMS LAN Network Management System(Backup)

Primary Multiplexer Voice I/F Data I/F

CCTV Cameras

Hotline Phone

Conference Subscriber Phone Phone

Data Line

Hotline Phone

Conference Subscriber Phone Phone

Data Line

Figure-5: Tiered Network Architecture

-6-

Converged Networks for Mission Critical Applications

Commtel White Paper

Conclusion

The benefits and challenges of the converged networks are highlighted in this paper. The convergence of technology has provided many opportunities for end user applications. The next generation networks based on multi service provisioning capability along with legacy dedicated networks based on TDM technology is well proven, matured and has competence to operate and manage the converged networks. The TDM technology with the features of delivering dedicated and assured bandwidth has the capabilities required for supporting the legacy networks along with Ethernet support. SDH along with primary multiplexer equipment based on TDM technology provides reliable network architecture for supporting mission critical applications. Ethernet is becoming the prevalent protocol for many applications as mentioned above. The utility operators are quickly adopting these newer technologies to remain competitive in the fast pace industry. As a migration strategy, communication infrastructure and the network solution that assures the coexistence of TDM and packet provides a strong choice and enables the investment protection. As the complexity of the networks increases the technological platforms like ASON/ASTN, which comprise of next generation SDH equipment with intelligent control plane will find its place in future.

Abbreviations

ASON ASTN BRI CCTV DCS EoS EPL EPLAN EVPL EVPLAN GFP IP ITU-T

Automatic Switched Optical Network Automatic Switched Transmission Network Basic Rate Interface Closed Circuit Television Distributed Control System Ethernet over SDH Ethernet Private Line Ethernet Private Local Area Network Ethernet Virtual Private Line Ethernet Virtual Private Local Area Network Generic Framing Procedure Internet Protocol International Telecommunications Union Telecommunication Sector Local Area Network Link Capacity Adjustment Scheme Management Information System Multiplex Section Protection Multi Service Provisioning Platform Mean Time Between Failures Network Management System -7-

PABX PRI QoS RSTP RTU SAP

Private Automatic Branch Exchange Primary Rate Interface Quality of Service Rapid Spanning Tree Protocol Remote Terminal Unit Systems Application and Products in Data Processing Supervisory Control And Data Acquisition Synchronisation Digital Hierarchy Sub Network Connection Protection Synchronous Transfer Module Spanning Tree Protocol Time Division Multiplexing Ultra High Frequency User to Network Interface Virtual Container Virtual Concatenation Very High Frequency Virtual Local Area Network Voice over Internet Protocol Wide Area Network

SCADA SDH SNCP STM STP TDM UHF UNI VC VCAT VHF VLAN VoIP WAN

LAN LCAS MIS MSP MSPP MTBF NMS

Converged Networks for Mission Critical Applications

Commtel White Paper

About Commtel

An engineering and technology-based company Commtels key attribute is to engineer and deploy converged networks providing voice, data and video services in vital environments. Being acutely aware of the crucial nature of every application, Commtel goes all out to ensure extreme reliability, value for money and support for many years beyond commissioning. Commtels advanced telecommunications network products and services are specifically engineered for mission critical applications to meet a variety of environmental operating conditions and also local and international standards. Commtel promotes, fosters and maintains a culture of innovation in keeping with its three principles: engineering excellence, value for money and innovative communication solutions. Commtels design engineering and implementation capability ranges from simple point-to-point network to complex communications network designed to meet specific customer needs. Commtels Triple C service capability - Conceptualise, Construct and Care - ensures customer support on end-to-end basis.

Disclaimer: This Document is Copyright 2008 by Commtel Networks Pvt. Ltd. Commtel retains full copyright ownership, rights and protection in all material contained in this document. You may use this document for your own purposes. You may distribute this document to other persons provided that you attribute the document as having been generated by Commtel Networks and that the document is available free of charge on Commtel Networks Web site.

-8-

Communication Solutions for the Future

Commtel Networks Pvt. Ltd.

Corporate Office: 23, White Castle 34-35 Union Park, Chembur Mumbai- 400 071. India Tel: +91 22 2520 1210 Fax: +91 22 2520 1220 Email: sales@commtelnetworks.com Navi Mumbai: Tel: +91 22 6780 2700 Fax : +91 22 6780 2888 New Delhi: Tel: +91 11 2952 1272 Fax: +91 11 2952 1274 United Arab Emirates: Tel: +971 6 557 3038 Fax: +971 6 557 3039

Web: www.commtelnetworks.com