Measurement team

Hans Ludwing Reyes Chvez

hans@noc.redclara.net

Network Operation Center

May 2006
Curitiba, Brasil

Introduction

CLARA Network

Levels of the CLARA Network

Server

CLARA

Pop

BACKBONE
LA-NREN

LA-NREN

LA-Universidad
Universidad

Universidad

Architecture
Administration LAN

PoP
Two

CLARA
Backbone
PoP
One

LA-NREN

University A

University B

PoP
Three

Commodity
Internet
Connections

INTERNATIONAL CONNECTIVITY
OTRAS
NRENS

CUDI
GEANT

UNAM
CLARA
USA
ABILENENLR

Brasil- RNP

OTRAS
NRENS

OTRAS
NRENS-LA
GLO

NMS

NMS - Network Management Station

To get the alarms and event from the
CLARA Network
Used for the CLARA NOC and NEG
staff
Used to make the reports asked for the
directors of CLARA
HP-Open View - Network Node
Manager

Backbone Server
On each Pop of CLARA we have two high
performance server
The server are used to monitoring the performance of
the backbone CLARA network
Pop to Pot
Pop to NREN/ITN

The server are using open source tools

NMS

NOC-CLARA

CUDI

CLARA
Chile

RNP

NMS

NOC-CLARA

CUDI

CLARA
Chile

RNP

CLARA
LAYOUT
NMS

PSTN

OOB
WS1

WS2

CLARA
Backbone
Router

Infrastructure

Monitoring Infrastructure
Since the begin in every Pop CLARA have
two servers dedicated to have and support
tools to measurement
NMS - Network Management station
Dedicated to get alarms
Traps
Reports

Infrastructure - Cont.
With backbone servers we can have an
active/passive monitoring
With the NMS we can get alarms, traps
and the reports for the CLARA directors
The software for the NMS show a topology
maps view of the network what make more
easy and faster get the idea of how is
working the network in every moment

CLARA
Measurement Team

Main Objective
Development and deployment systems
to monitoring and measurement the
performance that give us the
information to know the real status of
the network, giving the possibility to
detect and correct situations that could
affect the network

Specific Objectives
1. Determinate the parameter to
measure
2. Deploy and install the tools for the
monitoring
3. Make reports of the status of the
network

How we are

Members from Clara Technical Forum

NOC of the LA-NRENs
CLARA NOC / NEG
People outside CLARA

*Formed on the second CLARA- TEC meeting in Veracruz on april 2005

Initial Works
Install and test of the more familiar tools:

Iperf/Netperf
Pchar
PathLoad
Cacti
Netflow
Logs
AMP
OWAMP

Status of the Team

The applications are ready to be used for the
NOC, NEG and LA-NRENs
Making tests for the EELA project
Almost of the test has been done between a
LA-NRENs and CLARA, last month to Spain,
soon to CERN
Many application of real-time are asking us
how we can monitoring and guaranty the
availability of the service.
Streamings of 10Mbps or more using Multicast
Access-grids
Virtual reality and remote instrumentation

Initial Works - cont.

Install and test of the more familiar tools:
Beacon Multicast
Beacon H323
Route proxy
Syslog
Naggios

Sample 1
Using Iperf to measure the TCP
transference rate from Sao Paulo to
Panama.
First using the default parameters for
TCP, just one TCP session
After making more bigger the TCP
window

Sample 1 - Cont.

eriko@server2-panama:~$ iperf -s -w 200k -i 10

-----------------------------------------------------------Server listening on TCP port 5001
TCP window size: 400 KByte (WARNING: requested 200 KByte)
-----------------------------------------------------------[ 4] local 200.0.206.229 port 5001 connected with 200.0.206.133 port 41138
[ 4] 0.0-10.0 sec 8.01 MBytes 6.72 Mbits/sec
[ 4] 10.0-20.0 sec 8.85 MBytes 7.42 Mbits/sec
[ 4] 20.0-30.0 sec 9.07 MBytes 7.61 Mbits/sec
[ 4] 30.0-40.0 sec 8.85 MBytes 7.43 Mbits/sec
[ 4] 40.0-50.0 sec 8.86 MBytes 7.43 Mbits/sec
[ 4] 50.0-60.0 sec 9.06 MBytes 7.60 Mbits/sec
[ 4] 60.0-70.0 sec 8.87 MBytes 7.44 Mbits/sec
[ 4] 70.0-80.0 sec 8.86 MBytes 7.43 Mbits/sec
.
[ 4] 10.0-20.0 sec 21.5 MBytes 18.0 Mbits/sec
[ 4] 20.0-30.0 sec 22.0 MBytes 18.4 Mbits/sec
[ 4] 30.0-40.0 sec 21.5 MBytes 18.0 Mbits/sec
[ 4] 40.0-50.0 sec 22.0 MBytes 18.4 Mbits/sec
[ 4] 0.0-54.8 sec 116 MBytes 17.8 Mbits/sec

Sample 1 - Cont.

eriko@server2-saopaulo:~$ traceroute 200.0.206.229

traceroute to 200.0.206.229 (200.0.206.229), 30 hops max, 38 byte packets
1 lan-saopaulo (200.0.206.130) 0.170 ms 0.112 ms 0.111 ms
2 saopaulo-tijuana.core.redclara.net (200.0.204.6) 166.664 ms 166.661 ms 166.609 ms
3 tijuana-panama.core.redclara.net (200.0.204.14) 228.940 ms 228.876 ms 228.880 ms
4 server2-panama (200.0.206.229) 229.336 ms 228.900 ms 229.040 ms
eriko@server2-saopaulo:~$

badwidth-delay product:
155 Mbps x 230 ms = 35650 kbits =
4.456 MB

Sample 1 - Cont.

TCP parameters
---------------------------------------------------------------eriko@server2-tijuana:~$ cat /proc/sys/net/core/wmem_max
131071
eriko@server2-tijuana:~$ cat /proc/sys/net/core/rmem_max
131071
eriko@server2-tijuana:~$ cat /proc/sys/net/ipv4/tcp_rmem
4096 87380 174760
eriko@server2-tijuana:~$ cat /proc/sys/net/ipv4/tcp_wmem
4096 16384 131072

Modify (Linux)
-------------------------------echo 8388608 > /proc/sys/net/core/wmem_max
echo 8388608 > /proc/sys/net/core/rmem_max
echo "4096 87380 4194304" > /proc/sys/net/ipv4/tcp_rmem
echo "4096 65536 4194304" > /proc/sys/net/ipv4/tcp_wmem

Sample 1 - Cont.
eriko@server2-panama:~$ iperf -s -w 2000k -i 10 > iperf-tcp.log
-----------------------------------------------------------Server listening on TCP port 5001
TCP window size: 3.91 MByte (WARNING: requested 1.95 MByte)
-----------------------------------------------------------[ 4] local 200.0.206.229 port 5001 connected with 200.0.206.133 port 41265
[ 4] 0.0-10.0 sec 88.8 MBytes 74.5 Mbits/sec
[ 4] 10.0-20.0 sec 112 MBytes 94.1 Mbits/sec
[ 4] 20.0-30.0 sec 112 MBytes 94.1 Mbits/sec
[ 4] 30.0-40.0 sec 112 MBytes 94.1 Mbits/sec
[ 4] 40.0-50.0 sec 112 MBytes 94.1 Mbits/sec
[ 4] 0.0-50.8 sec 547 MBytes 90.3 Mbits/sec

Sample 1 - Cont.

Sample 2
Using Iperf to make a transference from UNAM at Mxico city to the CLARA
Pop in Tijuana at least 10Mbps
Making the same that in the first sample with out change the default
TCP the transferences were below 2Mbps
After a TCP tuning we got transference of more of 10Mbps

Conclusions
A big issue in the CLARA are the slow
transferences on TCP but UDP dont have
problem and all the BW can be used
Increasing the TCP window and Multiples
session of TCP can help to make more faster
file transferences
We are looking for others solutions more plug
and play to solve the slow file transferences
Web100
gridFTP
Bulk transference protocol

Conclusions - Cont.
Join to others international measurement
projects
Join more LA-NRENs to be on the team

Thanks !!