2014 China International Conference on Electricity Distribution (CICED 2014)
2014
Fast Peer-to-Peer Real-Time Data Transmission for Distributed Control of
Distribution Network
'
Kaijun Fan , Bingyin XU2•3• Guofang Zhu', Jie Gao2
1.School of Electrical Engineer ing, Shandong University. Jinan, 250061 ,China; 2. College of Electrical and Electronic Engineering , Shandong University of
Technology, Zibo, 255049,China; 3. Shandong Kehui Power Automation Co. Ltd., Zibo.255087,China.
Abstract
The real-time data transmission delay among Smart
Terminal Units(STUs) of related sites from a control
application is required to be no more than 10ms
for distributed control. D ata from distributed control
applications and other performance monitoring/
management applications are mixed transmitted
in distribution network, t ran smission delay is
increased if the network is busy, how to ensure fast
and reliable transmission of real-time data from
distributed control applications when network traffic
is large in distribution network is a key te chnical
issue.The composition of communication delay and
the main factors that affecting end-to-end real-time
communication are analysed, after the traditional
and IEC61850 GOOSE communication mechanisms
are given, a GOOSE mapped to UDP ( GOOSE over
UDP) transmission mechanism is presented and the
implementation method is given, the high priority of
GOOSE over UDP packets is obtained through the
MAC and IP layers setting, communication between
STUs has higher priority than comm unication
betwe en STU and master station, data with higher
priority can prior through switches and routers, thus
fast real-time transmission of control information is
ensured. Latency test result of GOOSE over UDP
communication is 4-8ms which could fully satisfying
the needs of distributed control applications in
distribution network.
Keywords:d i s t r i b u t e d co n t r o l; p e e r to p e e r
communication; GOOSE over UDP; T OS
1 Introduction
Distributed control of distribution network, which
not only could use information from multiple sites to
improve the performance of protection and control
applications, but also could avoid the problem of
long time latency in the centralized control mode,
has gained widespread attention now[1]. In order to
ensure the response speed of distributed control, the
time latency for real-time data transmission between
the smart terminal units (STUs) of related sites from
a control application are required to be no longer
than 10ms. In intelligent substation systems, the
time latency requirements which is less than 4ms
for real-time data transmission could be meet by
Generic Object Oriented Substation Event (G OOSE)
and Sampled Analogue Value(SAV) method in the
(CICED 2014 Session3) Paper No.
Shenzhen, 23-26 Sep.
process communication layer, or realized by using
directly connected fiber optic network[2]. While in
the distribution automation systems, the number
of distribution equipments is very large and the
equipments are spread over a broad area, in order
to improve the utilization rate of the communication
s ys t e m a n d r e d u c e i n v e s t m e n t , d a t a f r o m
distributed control and other operation monitoring
and management applications (such as the" three
remotes" data) are transmitted on the same network
togeth er, the t i m e delay of data transmission
increases along with the information flow increases
on th e network, how to ensure fast and reliable
transmission of real-time data for distributed control
applications in the distribution network under the
condition of heavy network flow is a key technical
issue[3].
Th e data which need to be transmit ted in the
distribution network is classified according to its
different time latency requirements in this paper. The
composition of communication d e lay is analysed,
the significant impact of higher level protocols
on the communication delay is determined. After
the traditional and GOOSE mapped to MAC data
communication mechanisms are analysed, t he
GOOSE mapped to UDP ( GOOSE o v e r UDP)
transmission mechanism is proposed, the realization
method of GOOSE over UDP transmission mode is
given. The communication data between STUs has
higher priority than communication data between
STU and the main station by the MAC layer and IP
layer priority settings ( GOOSE messages are set
to high priority), the communication data between
STUs is prior to through switches and routers, thus
the fast real-time transmission of control information
is ensured. Test result shows that the time latency
in this system is within 4ms-8ms which could fully
satisfy the requirements for distributed control
applications in distribution network.
2 The transmission delay requirements of
distributed control a p p licat ion data
D ue to the inherent characteristics of electricity
industry, general conclusions of performance
analysis from computer LAN cannot be fully applied
to the communication network of distribution network.
The related communication packets of distributed
CP0913 1 /5 Page /1041
 2014 China International Conference on Electricity Distribution (CICED 2014)
control applications in distribution network are
classified into four kinds according to their different
delivery time requirements as shown in Table 1[2].
Tab 1 Message transfer time
sequence messag e classification time require-
number tvoe number ment(ms)
1 fast speed 1 £20
messag e
2 medium 2 £100
speed
messag e
3 command 3 £20
messane
4 time 5 meet the
synchroni-
zation
Currently, tw o solutions can be used to solve
the problem of fast peer-to-peer real-time data
transmission. One way is to use GOOSE, whose
time latency is very low, GOOSE is a relatively
mature technology applied in substation automation
systems, y et its configuration is based on Media
Ac c e s s Control( MA C) ad dresses which leads
to higher configuration requirements for project
implementation; and its coverage of communication
interoperability is limited to the s ame LAN wh ich
affects the working scope of distributed control
applications. Another way is to use User D atagram
Protocol (UD P), which is widely used for fast data
transmission in the ethernet network, theoretical
analysis and test results show that the transmission
delay of UDP mode is no more than Sms if the
network traffic is less than 5Mbps, which could fully
meet th e time latency requirements of distributed
control applications, but the priority of UDP packets
is the same with the priority of communication data
(mainly in TCP transmission mode ) from Master
station, the d ata transmission delay is h eavily
affected by the network load, UDP packets may loss
under th e condition of large network traffic, therefore,
fast peer-to- peer real-time data transm ission
between STUs cannot be guaranteed.
Ethernet information transfer delay is random and
unpredictable, in order to ensure the performance
requirements of distributed control applications in
distribution automation, th e use of effective way s
to make information sent and received between
STUs in a timely manner is required, the end-to-end
transmission de lay must be limited within a certain
period of time in the worst case.
3 The composition of communication latency
The defin i t i o n of m ess a g e tr a n s f e r delay i s
(CICED 2014 Session3)
Shenzhen, 23-26 Sep. 2014
give n before analyzing the packet transfer delay.
Transmission time of the message includ es th e
proce ssing time which re quired on the send and
receive ends and the transmission time on the
japplication notE internet. Communication latency is composed of
fast tripping transmission delay, propagation delay, processing
command,and
delay and queuing delay[4].
t> tatus changin�
message
between
Transmission delay is the time which host or router
terminals
state value, needed to send data frames. That is to say, the time
measurement
needed from sending the first bit to the last bit of
and parameter
transfer the data frame. Transmission delay equals to data
between frame length divided by channel bandwidth. For a
terminals
between given network, the transmission delay is not fixed,
terminals but proportional to the transmission frame length,
precision
and inversely proportional to the bandwidth of the
requirements channel.
of the whole
svstem.
Propagation delay is the time cost by electromagnetic
wave propagating a certain distance in the channel.
Propagation delay equals to the channel length
divided by propagation speed of electromagnetic
wave in the channel. The velocity of electromagnetic
wave propagation in free space is the speed of light,
propagation rate in the copper cable is 2.3' 10Skml
s, propagation rate in th e optical fiber is about
2.0'10Skm/s. The transmission delay occurs inside
the machine (such as network adapter), propagation
delay occurs on the transmissio n media which is
outside of the machine.
Processing delay refers to the time which a host
or router needed to spend on the received packet
message processing, processing including packet
header analysis, extracted data from th e packet,
error checking or appropriate route finding, etc.
The message queuing delay is the queue time occurs
inside a router from input queue and output queue.
The length of time latency depends on network
traffic. When a large amount of communication
happens the queue overflow will occur which leads
to the loss of the packet, that means the delay is
infinity.
To sum up, the whole communication delay equals
to transmission delay plus propagation delay plus
processing delay and queuing delay.
D istributed cont rol and protection appli cation s
of dist r i b u t i o n network implem e n t e d through
communication network, the end-to-end transmission
delay of the communication network is required to
be certain. Therefore, the test of communication
performance of ethernet system is a necessary
step in building communicatio n system of smart
Paper No. CP0913 2/5 Page /1042
 2014 China International Conference on Electricity Distribution (CICED 2014) Shenzhen, 23-26 Sep. 2014

distribution network. D ue to the cost of message

protocol overhead encapsulation and decapsulation
in the end node, end-to-end transmission delay is
much higher than the average ethernet link delay.
Therefore, th e performance of communication
protocols is a crucial factor in network real-time
aspect. For example, processing delays of using
TCP and UDP are different. In order to meet the time
limit requirements of distributed con trol applications,
to carefully select the appropriate network protocol is
needed as well as guarantee th e system open[5] .

4 The realization of GOOSE over UDP

transmission mechanism
4.1 Several kinds of communication transmission
mechanism for distribution automation
TCP protocol is usually used for data communication
in th e traditional distribution network as shown in
Figure 1, the reliability of this way is high, but it need
to establish a connection before data transferring
and cannot guarantee the time latency. IEC 6 1850
GOOS E th rough priority setting in th e data link
layer to ensure GOOSE message preferred to send
than ordinary IP message under a heavy network
load. Since IE C 61850 GOOSE does not through
network lay e r as shown in Fig.2, the packet cannot
be transmitted cross a local area network. Due
to the simulation result of reference 2 that both
GOOSE message and IP message transmission
time is le ss than 0.5ms,which can meet the time
requirements(less than 4ms) of IEC 6 1850 on
the fast message. Therefore, mapping GOO S E
message t o UDP protocol t o b e transmitte d by IP
is considered, which is convenient for GOOSE to
implemented in distribution network automation,
to distinguish from IEC 61850 GOOSE, this way
is called GOOSE over UDP whose transmission
mechanism is shown in figure 3. Using the priority
setting of IP layer and MAC layer to ensure fast
message among STUs prior through.
"
Fig.2 Transmission mechanism of lEe 61 850 GOOSE
STU]
Fig.3 Transmission mechanism of GOOSE over UDP
User Datagram Protocol (UD P) is an important
transport layer p r o tocol of T CP / IP n etwork
arch itecture. UD P does not provide reliable data
transmission services, that is, when the sending
process using UDP socket to send packets, UDP
does not guarantee that th e message can be
received by the receiving process, an d received
packets may arrive out of order. Although UDP does
not provide reliable data transmission services,
it has become the first choice of m any real-time
applications due to its efficient data transmission.
Such as routing tables update, network management
and multimedia applications. UDP is better at control
the data to be send and the transmission time, in
addition, UDP can transmit data without establishing
a connection, does not need to maintain connection
state during the data transmission process, does not
have to verify the integrity of data and confirm data
has received or not.

GOOSE retransmis sion mechanis m can be used

to ensure the reliability of packet transmission in
GOOSE over U DP mechanism. N etwork traffic
will be overloaded if the retrans mission times
is too many, reliable t ransmission o f G O OSE
messages can be ensured by packets retransmitting
and g radually increas ing the time intervals in

(CICED 2014 Session3) Paper No. CP0913 3 /5 Page 11043

 2014 China International Conference on Electricity Distribution
retransmission mechanism as shown in Fig.4.
��'1!'ti=�'1 ' Event
Trigger
Fig.4 Retransmission mechanism of GOOSE packet
The real-time requirement which is l e s s than
100m s o n G O OSE o v e r UDP in d i s t r i b u t i o n
automation i s lower than IEC 61850 i n substation
a u t o ma t i on, GOO SE p a c k e t t r a n s m i s s i o n
i n t e r v a l s a r e r e c o m m e n d e d a s f o l l ows :
T O- r e t r a n s m i s s i o n i n t e r v a l u n d e r s t e a d y
state (l o n g time no i n c i d e n t)
be less t h a n 60S , 20S c a n b e c h o s e n;
(TO): retransmission interval under steady state
is shortened by event should be less than TO;
T 1: A f t e r the incident happened,
s h o r t e s t t ran s m i s s i o n t i m e i n t e r v a l s h o u l d
b e l e s s t h a n lOOm s , 2 0 m s i s s e l e c t a b l e;
T2: T2 > T1, 50m s is r e c o m m e n d e d;
T3: T 3> T2 , lOOms could be chosen.
4,2 Real-time realization of fast spee d packet
In orde r to ensure fast and reliable tran smission of
rapid packet from distributed control applications in
data mixed transmission network, switches which
could support VLAN priority are used, the priority
field in th e VLAN tag of the MAC layer and TOS
field of IP layer are used to set GOOSE over U DP
packets to high priority.
Destination Source tvlAC Tag of
Type Data
tvlAC Addless Address VLAN
46-1500
6 bytes 6 by1es 4 bytes 2 bytes
by1es
Flg.S Format of MAC message
Ethernet frame format among switches which could
support VLAN is shown in Fig.5, th ree bits in the
MAC packet's VLAN field to set priority level, IP
packet format is shown in[6] Fig.6, IP packets can
achieve the Type of Service( T OS) functionality for
upper protocols, TOS is consisted of 8 bits which is
used to indicate the quality of service, the value of
TOS is usually specified by the application. Currently,
these fields are ignored by almost all networks, partly
because the function itself is difficult to achie ve under
the required quality, on the other hand, unfairness
is caused if the quality requirements cannot be met.
Therefore, the realization of TO S control becomes
extremely complex, resulting in T OS is unapplied
in the whole internet. Setting different priority levels
for different data packets combining with features
of distribution automation, GOOSE messages
which will be mapped to the UD P protocol are set
to high priority while other packets that through
TCP protocol are set to low priority, therefore real­
time requirements of higher priority packets can be
(CICED 2014 Session3)
(CICED 2014 ) Shenzhen, 23-26 Sep. 2014
guaranteed.
Fig. 6 Format of IP packet
4,3 Test result
With the development of communication technology
and cost reduction of communication equipments,
the conditions to build a wide area IP communication
n etwo rk w hich covers the main station, slave
stations and distribution terminals for distribution
s h oul d automation are possessed now. Using IP network
can fully satisfy the needs of distribution automation,
more standard communication equipments can
be used, the network construction of distribution
th e automation could be simplified. The application of
IP network in distribution automation could facilitate
communication among distribution terminals and
the realization of distribution te rminal access to the
master station, which will promote the development
of smart distribution grid.
The real-time performance of the network in which
GOO S E ove r UD P transmission mechanism is
applied is tested by a three switches cascaded
system that is constructed based on STUs, the test
result shows that the time latency in this system
is within 4ms-8ms, wh ich could fully satisfy th e
requirements for distributed control applications of
distribution network. The configuration of GOOSE
FCS
over UD P mechanism is based on IP addresses,
4 bytes
project implementation is simple, fast peer-to-peer
real-time data transmission between STUs among
different LANs can be achieved.
5 Conclusion
T h e c o m p o s i ti o n of c o m m u ni c a t i o n d el a y is
described, the main factors which affecting the
end-to-end communication time are analyzed, the
conclusion that upper level protocols also have a
great impact on the communication delay is drawn.
GOOS E over U DP transmission mechanism is
proposed and the implementation method is given.
Setting GOO SE over UDP packets to high priority
in MAC and IP laye rs so that communication
data between STUs has hi gher priority than
communication data between STU and the main
station, hence, GOOSE over UDP packets could
p r i o r t h r o u g h switch es and r o u t e rs a n d fast
transmission of control information is ensured.
References
[1] Li Ti anyou, Jin Wenlong, Xu Bingyin, etc.
D istribution technology[M]. Beijing:China Electric
Paper No. CP0913 4/5 Page /1044
 2014 China International Conference on Electricity Distribution (CICED 2014) Shenzhen, 23-26 Sep. 2014

Power Press,2008.

[2] Han Guozheng. IEC6 1850-based communication

architecture for distribution automation[D], Shandong
UniversitY,2011.

[3] Xu Bingyin,Xue Yongduan,Li Tianyou,et al.A

wide area measurement and control system for
smart distribu tion grids and its protection and
control applications[J].Automation of Electric Power
Systems,2012,36 (18):2-9(in Chinese).

[4] Xie Xiren, Xie Jun. A textbook on computer

networks[M](3rd Edition).Beijing:Posts& Telecom
Press,2012.

[5] Wu Zaijun,Hu Minqiang,Du Yansen.Analysis of

realtime performance of communication network
in sUbstations[J].Automation of Electric Power
Systems,2005,29(8):45-49(in Chinese) .

[6 ] Takafumi Takeshita,Yukio Murayama, Toru

Arai, Yukio Karita. Mastering TCP/IP nyuumon
(Chinese Language Edition). Beijing: Post& Telecom
Press.2013.
Paper received:
Biography:
Fan Kaijun(1985-) ,Female, Juxian, Shandong
province, PhD student from Shandong University,
Research interest: Distributed control in smart
distribution network.

(CICED 2014 Session3) Paper No. CP0913 5/5 Page 11045