Switch Testing

26601 W. Agoura Rd. Calabasas, CA 91302 (Toll Free US) 1.877.FOR.IXIA (Int'l) +1.818.871.1800 (Fax) 818.871.1805 www.ixiacom.com

Test Plan

Copyright 2006 by Ixia All rights reserved Ixia 26601 West Agoura Road, Calabasas, CA 91302 (877) FOR-IXIA This Test Plan contains a general outline for testing a particular technology. Not all the capabilities of Ixia technology have been exposed in this document. Please feel free to contact us if additional capabilities are required.

Switch Test Plan

Contents
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1. RFC 2889 Address Cache Size Test . . . . . . . . . . . 5 Objective . . . . . . . . . . . . . . . . . . . . . . . . . 5 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Input Parameters . . . . . . . . . . . . . . . . . . . 6 Methodology . . . . . . . . . . . . . . . . . . . . . . 7 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2. Data Integrity and Error Checking Test Objective . . . . . . . . . . . . . . . . . . Setup. . . . . . . . . . . . . . . . . . . . . Input Parameters . . . . . . . . . . . . . Methodology . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . ........8 ....... 8 ....... 8 ....... 9 ....... 9 . . . . . . 10

3. RFC 2544 Benchmark Tests . . . . . . . . . . . . . . . 11 Objective . . . . . . . . . . . . . . . . . . . . . . . . 11 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Input Parameters . . . . . . . . . . . . . . . . . . 12 Methodology . . . . . . . . . . . . . . . . . . . . . 13 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4. RFC 2889 Frame Error Filtering Test . . . . . . . . . 17 Objective . . . . . . . . . . . . . . . . . . . . . . . . 17 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Input Parameters . . . . . . . . . . . . . . . . . . 18 Methodology . . . . . . . . . . . . . . . . . . . . . 19 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Continued on next page

2006 by Ixia

p.3

www.ixiacom.com

Contents
5. RFC 2889 Fully Meshed Test . . . . . . . . . . . . . . . 21 Objective . . . . . . . . . . . . . . . . . . . . . . . . 21 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Input Parameters . . . . . . . . . . . . . . . . . . . 22 Methodology . . . . . . . . . . . . . . . . . . . . . 22 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 23 6. Layer 2-3 Stateless QoS Functional Test. . . . . . . 24 Objective . . . . . . . . . . . . . . . . . . . . . . . . 24 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Input Parameters . . . . . . . . . . . . . . . . . . . 26 Methodology . . . . . . . . . . . . . . . . . . . . . 27 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 28 7. Spanning Tree Network Convergence Performance Test . . . . . . . . . . . . . . . . . . . . . . . 32 Objective . . . . . . . . . . . . . . . . . . . . . . . . 32 Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Input Parameters . . . . . . . . . . . . . . . . . . . 34 Methodology . . . . . . . . . . . . . . . . . . . . . 34 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 36 8. OSPF Performance Test . . . . . . . . . . . . . . . . . . 38 Objective . . . . . . . . . . . . . . . . . . . . . . . . 38 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Input Parameters . . . . . . . . . . . . . . . . . . . 39 Methodology . . . . . . . . . . . . . . . . . . . . . 39 Results . . . . . . . . . . . . . . . . . . . . . . . . . 39

Switch Test Plan

Overview
Switching in this document refers to the function of Layer 2 and 3 devices that interconnect other network devices and computing equipment. While there is some overlap with the functionality of other devices, such as routers, switches operate at Layers 2 and 3 of the seven layer OSI Model. There are a great variety of switches available today. Figure 1 provides a view of the various features and functionality of switches, ranging from simple low-end switches to multi-function high-end switch products. High-end switches additionally provide much higher performance than low-end switches. The primary functions of a switch include: Forwarding based on Layer 2 and/or 3 information Forwarding with trafc prioritization (QoS) Internetworking with other switches and devices Switching between various network interfaces and speeds (e.g., 10/100/1000/10G Ethernet, ATM, T1, E1, etc.) Security features and ltering (e.g., VLAN, ACL, etc.)

2006 by Ixia

p.1

www.ixiacom.com

Switch Test Plan

LOW END

MEDIUM RANGE

MEDIUM HIGH RANGE

HIGH RANGE

Layer 2/3
MPLS GRE IGMP version 3

Layer 2/3
10 Gig Ethernet Jumbo Frames IPv4 IP Routing (RIP, OSPF, BGP) Multicast (PIM, DVMRP) Load Balancing and Redundancy

Cost

Layer 2
MAC Address Learning Packet Filtering Store and Forward STP, RSTP, MSTP, VLAN Tagging Quality of Service IGMP v1/v2 SNMP

10 Gig Ethernet Jumbo Frames IPv4 IP Routing (RIP, OSPF, BGP) Multicast (PIM, DVMRP) Load Balancing and Redundancy

STP, RSTP, MSTP, VLAN Tagging Quality of Service IGMP v1/v2 SNMP

STP, RSTP, MSTP, VLAN Tagging Quality of Service IGMP v1/v2 SNMP

MAC Address Learning Packet Filtering Store and Forward

MAC Address Learning Packet Filtering Store and Forward

MAC Address Learning Packet Filtering Store and Forward

Low

Switch Performance

High

Figure 1. Switch Categories and Features Reference Model Low-end switches are unmanaged, plug-and-play devices. Their purpose is to forward trafc with no advanced trafc analysis or scheduling. They are designed for small user groups, including home use. Medium range switches are more advanced, congurable devices capable of running Spanning Tree for loop prevention as well as supporting VLANs and trafc prioritization/scheduling. These devices also often support multicast protocols. Medium to high-end switches are deployed in larger networks and are distinguished by the addition of IP routing functionality. They are capable of effectively managing various types of trafc including a mix of data, voice, and video trafc, and forwarding trafc at line rate. They provide higher port density, higher scalability and performance, as well as offering a wider variety of interface modules and types. High-end switches are deployed in large scale networks where high performance and resiliency are critical requirements. They interconnect with many other switches and routers and can operate as the backbone of the network. These devices are highly scalable in both ports and protocols, providing a high level of reliability and performance.

2006 by Ixia

p.2

www.ixiacom.com

Switch Test Plan

This test plan provides a general framework and structure for custom test plan development that addresses the performance and functional tests requirements for the rst three categories of switches outlined above. It provides a starting point that can be extended to cover many other aspects of switch functionality. Ixias IP testing systems can be used to meet your specic switch testing requirements, and assist in the benchmarking and predeployment analysis of network devices and systems. Following is a description of the 8 test cases outlined in this test plan. Test Case RFC2889 Address Cache Size Data Integrity and Error checking Description Identies the switch address table size capacity. This is achieved using a binary search algorithm. beginning at half the size of the initial user-specied table size Veries the DUTs ability to forward frames under certain trafc rates without corrupting the payload. Frames are transmitted with a predened data pattern and it is veried that the DUT properly forwards the frames. Provides a benchmark performance analysis of the DUT using industry standard methodology. Four functional test areas are covered: Back-to-Back, Frame Loss, Latency, and Throughput. These tests measure forwarding performance and latency using linear or binary searches. Determines if the DUT correctly lters illegal frames such as undersized frames, oversized frames, frames with CRC errors, fragmented frames, alignment errors, and dribble errors. Determine the total number of frames that the DUT can handle when it receives frames on all its ports. Each port in the test sends frames to all other ports in an evenly distributed, round-robin fashion at a specic user dened rate.

RFC 2544 Benchmark

RFC2889 Frame Error Filtering RFC2889 Fully Meshed

2006 by Ixia

p.3

www.ixiacom.com

Switch Test Plan

Layer 2-3 Stateless QoS Functional Test Spanning Tree Network Convergence

Measures the baseline performance of the DUT with and without QoS when stateless trafc is injected into the network. Veries the DUTs Spanning Tree convergence performance. This test measures the network convergence based on the handling of Topology Changes Notications and Conguration BDPUs as well as trafc switchover. Measures the OSPF performance and scalability of a DUT. A dened OSPF topology is set up and the no-drop throughput and latency measured across it. The test supports both OSPFv2 and OSPFv3 protocols.

OSPF Performance

Table 1. Switch Test Cases

2006 by Ixia

p.4

www.ixiacom.com

Switch Test Plan

1. RFC 2889 Address Cache Size Test

Objective The purpose of this test is to determine the switchs address table size capacity. The size of the address table for each port or for the entire switch is found by starting with half of the size of the initial user-specied table size and using a binary search algorithm. Learned frames are transmitted between each iteration. Then generic frames are transmitted at a userspecied frame rate to see if the DUT has properly learned all of the addresses. If neither frame loss nor ooding is detected, the address table size is increased, and the test is repeated in a binary fashion until the address table size is determined. Setup The baseline setup for this test requires three test ports. The DUT receives the trafc on one port and forwards it back to the other two emulated test ports for analysis. See Figure 2. Ixias IxScriptMate RFC 2889 Address Cache Size Test script can be used to set up and execute this test.

2006 by Ixia

p.5

www.ixiacom.com

Switch Test Plan

Ixia Port 1

Ixia Port 2

p1 p2

p3

DUT

Ixia Port 3

Traffic is sent on port 1 to the DUT with number of addresses DUT forwards back the received traffic with the learned addresses on ports 2 and 3 The traffic received from the DUT is analyzed for learned address accuracy If no frame loss or flooding is detected, the table size is increased and a binary search is performed until the maximum table size is reached

Figure 2. Address Cache Size Test Setup Input Parameters Parameters Frame Size Trafc Rate Table Size Age Description The selected frame size used for the test Initial rate of Trafc to be sent from the transmit port The desired table size the user sets for this test The Age value that coincides with the address table aging parameter on the DUT

Table 2. RFC 2889 Address Cache Size Test Input Parameters Table
2006 by Ixia p.6 www.ixiacom.com

Switch Test Plan

Methodology 1. Congure to start the test with an initial frame size, trafc rate, and a desired table size. Refer to Table 2 above for the necessary Input Parameters. 2. Run the test. The trafc received by the DUT is forwarded back to the other test ports for the DUT learned addresses accuracy check.

Figure 3. RFC 2889 Address Cache Size Test Setup Results The results shown in Figure 4 indicate that the trafc was sent to the DUT and received back from the DUT at a line rate of 5% and at a desired table size of 200, with a nal address table size of 199. A total of 2000 frames were sent to the DUT at a rate of 10 frames for each learned address.

Figure 4. RFC 2889 Address Cache Size Results

2006 by Ixia

p.7

www.ixiacom.com

Switch Test Plan

2. Data Integrity and Error Checking Test

Objective The purpose of this test is to verify the ability of the DUT to forward frames at a certain trafc rate without corrupting the payload. This test consists of transmitting frames that contain some predened data pattern and verifying that the DUT forwards the frames properly. The test calculates the number of sequence errors and the number of data errors. The rst measurement is made when the trafc rate is set at one level, and the second measurement is made when the trafc rate is increased to another level. A comparison is made between the two measurements to identify any possible impact on the data integrity results. Setup The baseline setup for this test requires two test ports. Both ports send Layer 2 or 3 trafc to the DUT with a predened data pattern. The DUT receives the trafc and forwards it back to the same two emulated test ports for analysis. See Figure 5. Ixias IxScriptMate MATS Data Integrity Test script can be used to set up and execute this test.
Ixia Port 1

Ixia Port 2

p1 p2

DUT

Traffic is sent by the test port with payload and predefined data pattern DUT receives the frames and sends back to the test ports for analysis, data integrity and sequence checking

Figure 5. Data Integrity Test Setup

2006 by Ixia p.8 www.ixiacom.com

Switch Test Plan

Input Parameters Parameters Frame Size Trafc Rate Data Pattern Description The selected frame size used for the test Initial Trafc rate that the transmit port(s) will send User selected data pattern, for example AllOnes

Table 3. Data Integrity Test Input Parameters Table Methodology TEST 1 Initial trafc rate 1. Congure to start the test with an initial, maximum trafc rate, e.g., 50%. 2. Enter the appropriate test parameters refer to Table 3. 3. Run the test for the specied duration for all frame sizes. The trafc received by the DUT is forwarded back to the same transmitting test ports for analysis. The emulated test ports check for the validity of the frames and perform data integrity on the payload and sequence frames checking. TEST 2 Increased trafc rate 1. Increase the initial maximum trafc rate (e.g., 75%) and rerun the test.

Figure 6. IxScriptMate Data Integrity Test Setup

2006 by Ixia

p.9

www.ixiacom.com

Switch Test Plan

Results The results indicate that the trafc that was sent to the DUT and received back from the DUT at a line rate of 50% showed no errors in data, frame sequence, or any trafc loss. See Figure 7. However, as the trafc line rate was increased to 75%, both trafc loss, as well as sequence errors, were observed. In addition, as the frame size was increased at the new higher trafc rate, sequence errors were also increased, though slightly. See Figure 8.

Figure 7. Data Integrity and Frame Loss/Error Count Report (trafc rate at 50%)

Figure 8. Data Integrity and Frame Loss/Error Count Report (trafc rate set at 75%)

2006 by Ixia

p.10

www.ixiacom.com

Switch Test Plan

3. RFC 2544 Benchmark Tests

Objective These test cases address four performance benchmark tests dened by RFC 2544: Back-to-Back, Frame Loss, Latency, and Throughput. These tests determine throughput and latency characteristics of the device under test using linear or binary search algorithms. An overview of these tests is listed below: BACK-to-BACK Starting with a maximum trafc rate, this test determines the maximum duration that the DUT can receive and forward without frame loss. Frames are sent at a user-specied rate. A binary search algorithm is used to obtain the longest duration by the DUT without any loss. FRAME LOSS Starting with the initial frame rate, the test transmits a specied number of frames to the DUT. The DUT receives the frames and then forwards them back to the other test ports, which in turn calculate the number of frames received and analyze the measured frame loss. A binary search algorithm is used to obtain the highest trafc load that the DUT can handle without any frame loss. THROUGHPUT Starting with an initial frame rate, the test transmits a specied number of frames to the DUT. The DUT forwards the frames back to the other port. A binary search algorithm is used to obtain both the rate and the frame size at which the DUT provides the best throughput. LATENCY Starting with a maximum trafc rate (where the DUT does not lose frames) the test compares the transmit timestamp of the tagged frames with the receive timestamp. The difference between the two timestamps is the measured latency.

2006 by Ixia

p.11

www.ixiacom.com

Switch Test Plan

Setup The baseline setup for these test requires two test ports directly connected to the DUT and generating trafc at various frame sizes and trafc rates. See Figure 9. Ixias IxScriptMate RFC 2544 test suite can be used to set up and execute this test.
Ixia Port 1

Ixia Port 2

p1 p2

DUT

Traffic is sent by the test port to the DUT port p1 The DUT receives the traffic and forwards back on port 2 to the other test port for Back-to-Back, Frame Loss, Latency and Throughput analysis

Figure 9. RFC 2544 Benchmark Tests Setup Input Parameters Parameters Frame Size Trafc Rate Description The selected frame size(s) used for the test Initial Trafc rate of the transmit port

Table 4. RFC 2544 Benchmark Tests Input Parameters Table

2006 by Ixia

p.12

www.ixiacom.com

Switch Test Plan

Methodology TEST 1 Back-to-Back 1. Set up the test parameters for this test. Refer to Table 4 for the Input Parameters. 2. Start the test and run for all selected frame sizes. This test performs a binary search to determine the longest duration the DUT experiences in forwarding frames without any loss. 3. See Figures 11 and 12 for results analysis. TEST 2 Frame Loss 1. Set up the test parameters for this test. Refer to Table 4 for the Input Parameters. 2. Start the test and run for all selected frame sizes. The test performs a binary search for the highest trafc load that the DUT can handle with the least frame loss. 3. Once the test concludes, note the frame loss values as both the frames size and the trafc rate changes. See Figures 13 and 14 for results analysis. TEST 3 Throughput 1. Set up the test parameters for this test. Refer to Table 4 for the Input Parameters. 2. Start the test and run for all frame sizes. The test performs a binary search for the highest trafc load that the DUT can handle with the best throughput. 3. See Figure 15 for results analysis. TEST 4 Latency 1. Set up the test parameters for this test. Refer to Table 4 for the Input Parameters. 2. Set the trafc rate for a relatively low rate to ensure the least possible trafc loss and more accurate latency measurements. 3. Start the test and run for all frame sizes. The test performs a binary search for the highest trafc load that the DUT can handle with the least latency. 4. See Figures 16 and 17 for results analysis.

2006 by Ixia

p.13

www.ixiacom.com

Switch Test Plan

Figure 10. IxScriptMate RFC 2544 Benchmark Tests Setup Results The results for each of the 4 test cases are given below. The Back-to-Back test concludes the maximum number of back-toback frames that the DUT is capable of forwarding without any frames loss for each of the frame sizes. See Figures 11 and 12. The Frame Loss test concludes that frame loss is experienced when the trafc rate increases above the 50% mark, regardless of the frames size. See Figures 13 and 14 The Throughput test concludes that the best aggregate throughput for the DUT is experienced when frame size is at a minimum of 64 bytes size. The results show the best trafc that the DUT is able to forward without any data loss for each of the frame sizes. See Figure 15. The Latency test concludes that the least average latency for the DUT is observed when frame size it at a minimum of 64 bytes size. The best throughput is noted when frame size is at a minimum of 64 bytes size. See Figures 16 and 17.

2006 by Ixia

p.14

www.ixiacom.com

Switch Test Plan

Figure 11. RFC 2544 Back-to-Back Iterations Statistics Results

Figure 12. RFC 2544 Back-to-Back Summary Results per Frame Size

2006 by Ixia

p.15

www.ixiacom.com

Switch Test Plan

Figure 13. RFC 2544 Frame Loss Iteration Statistics Results

Figure 14. RFC 2544 Frame Loss Aggregate Results per Frame Size

Figure 15. RFC 2544 Throughput Aggregate Results per Frame Size

Figure 16. RFC 2544 Latency Aggregate Results per Frame Size

Figure 17. RFC 2544 Latency Statistics Results per Frame Size
2006 by Ixia p.16 www.ixiacom.com

Switch Test Plan

4. RFC 2889 Frame Error Filtering Test

Objective This test determines if the DUT correctly lters illegal frames, such as undersized frames, oversize frames, frames with CRC errors, fragmented frames, alignment errors and dribble errors. The results show the type of error transmitted, the number of transmitted frames, inter-frame gap, and the number of errored frames at each frame size. Setup The baseline setup for this test requires three test ports. One emulated port sends Layer 2 trafc to the DUT. The DUT receives the trafc and lters any of the illegal frames or errors and forward back only acceptable frames to the test ports for analysis. See Figure 18. Ixias IxScriptMate RFC 2889 Frame Error Filtering Test script can be used to set up and execute this test.

2006 by Ixia

p.17

www.ixiacom.com

Switch Test Plan

Ixia Port 1

Incoming Traffic

Ixia Port 2

p1 p2

p3

DUT

Ixia Port 3

Outgoing/ Filtered Traffic

Traffic is sent by on test port to the DUT port p1 Traffic received by the DUT is filtered and only acceptable frames are forwarded back on p2 and p3 to the other test ports for analysis and any possible errors

Figure 18. RFC2889 Frame Error Filtering Test Setup Input Parameters Parameters Frame Size Illegal Frame Types Trafc Rate Description The selected frame size(s) used for this test Selected illegal frames and types to send to the DUT Initial Trafc rate that the transmit port(s) will send

Table 5. RFC2889 Frame Error Filtering Input Parameters Table

2006 by Ixia

p.18

www.ixiacom.com

Switch Test Plan

Methodology 1. Set up the test parameters for this test. Refer to Table 5 for the Input Parameters. 2. Start the test and run for all selected illegal frame types and maximum trafc rate. See Figure 19. The trafc goes through the DUT and is forwarded back to the emulated test ports after all illegal and unaccepted frames are ltered by the DUT. 3. Once the test concludes note the Frame Error Filtering results. See Figure 20.

Figure 19. IxScriptMate RFC2889 Frame Error Filtering Test Setup

2006 by Ixia

p.19

www.ixiacom.com

Switch Test Plan

Results The only error that was found and detected by the test ports in this test was for a frame size of 1,519, which was allowed to be forwarded by the DUT. See Figure 20.

Figure 20. IxScriptMate Frame Error Filtering Report

2006 by Ixia

p.20

www.ixiacom.com

Switch Test Plan

5. RFC 2889 Fully Meshed Test

Objective The purpose of this test is to determine the total number of IP frames that the DUT can handle when it receives frames on all its ports. Each port in the test sends frames to all other ports in an evenly distributed, round-robin type fashion at a specic userdened rate. Setup The baseline setup for this test requires three test ports. All ports send Layer 2 or 3 trafc to the DUT with an initial trafc rate. Trafc is forwarded back to the same three ports for analysis. This test requires VLAN and IP addresses to be congured on the DUT. Ixias IxScriptMate RFC 2889 Fully Meshed Test script can be used to set up and execute this test.
Ixia Port 1

Traffic

VLAN (a) IP1

Ixia Port 2

p1

p2

VLAN (b) IP2 p3

DUT

Ixia Port 3

VLAN (c) IP3

Traffic is sent by the test ports to the DUT at various traffic rates and frame sizes Traffic is forwarded back and analyzed by the test ports to determine the number of frames that the DUT can handle on all its ports

Figure 21. IxScriptMate Fully Mesh Test Setup

2006 by Ixia

p.21

www.ixiacom.com

Switch Test Plan

Input Parameters Parameters Frame Size Trafc Rate Trafc Data Type DUT setup Description The selected frame size(s) used for the test Initial trafc rate that the transmit port(s) will send The user selected Data Type, for example IP DUT is congured for VLANs with IP address association

Table 6. RFC 2889 Fully Meshed Test Input Parameters Table Methodology 1. Set the initial Trafc Rate, Frame Sizes, and Data Type for the test. Refer to Table 6 for the Input Parameters. 2. Run the test for the desired duration for all selected frame sizes and trafc rate. See Figures 22 and 23. 3. Once the test concludes, note the ports statistics results. See Figure 24.

Figure 22. IxScriptMate RFC 2889 Fully Meshed Test Trafc Setup (1)

2006 by Ixia

p.22

www.ixiacom.com

Switch Test Plan

Figure 23. IxScriptMate RFC 2889 Fully Meshed Test Trafc Setup (2) Results The results indicate that the DUT experienced frame loss for each of the selected frame sizes when the trafc rate was set at 50%. See Figure 24. This degradation in performance is possibly related to the buffering algorithm on the DUT, which had slowed down due to the intensive load of frames received and the associated processing requirement.

Figure 24. RFC 2889 Fully Meshed Port Statistics per Frame Size Test Report

2006 by Ixia

p.23

www.ixiacom.com

Switch Test Plan

6. Layer 2-3 Stateless QoS Functional Test

Objective The purpose of this test is to measure the baseline performance of the DUT with and without QoS when stateless trafc is injected into the network. Stateless trafc is of type Layer 2 -3 data and does not emulate true user application trafc. This test veries that the latency and the packet loss on the egress trafc port degrades signicantly when QoS is enabled on the receiving DUT. The rst step is to take measurements and collect statistics when QoS is disabled on the DUT. The second step is to take measurements and collect statistics when QoS with IP Precedence classifying and marking are enabled on the DUT. Setup The baseline setup for this test requires four test ports. Three ports are used to generate Layer 3 trafc connected to three DUT ports. These connections are considered the ingress ports to the DUT or the network. Each port carries a separate stream with a specic IP Precedence marked value. The fourth test port is connected to a fourth DUT port to evaluate the outgoing network trafc (egress) based on the QoS service characteristics and settings. See Figure 25. Ixias IxScriptMate QoS Many-to-One test can be used to setup and execute this test.

2006 by Ixia

p.24

www.ixiacom.com

Switch Test Plan

Ixia Port 1

Priority 0

Ingress
p1

Ixia Port 2

Ixia Port 4
p2

p4
DUT

Priority 5

p3

Egress

Ixia Port 3

Priority 6

Traffic is received by the DUT over three different interfaces p1, p2 and p3 DUT prioritizes the traffic according to the traffic classification and any policies The resultant traffic (Egress) is measured for traffic latency and packet loss with and without Qos enabled on the DUT

Figure 25. QoS Many-to-One Test Setup

2006 by Ixia

p.25

www.ixiacom.com

Switch Test Plan

Input Parameters Two sets of parameters are required prior to running the Layer 2/3 QoS functional test. One set of parameters is for the test tool and the other for the DUT. Parameters Frame size Duration Trafc Rate DUT-QoS DUT-Line Speed DUT-QoS Type DUT-QoS Policies DUT-Queue Type Description Packet frame size can bet set as xed or random Test Duration to run ranges from hours down to seconds Trafc rate per priority level Administrative DUT QoS setting (enabled or disabled) The link / interface speeds of the DUT ports DUT QoS type settings: COS, ToS IP Precedence, or DSCP DUT QoS Policies applied to the ingress trafc Queuing mechanism such as Weighted Random Early Detection (WRED) and Weighted Round Robin (WRR) Queuing

Table 7. QoS Many-to-One Input Parameters Table

2006 by Ixia

p.26

www.ixiacom.com

Switch Test Plan

Methodology TEST 1 QoS is disabled on the DUT 1. With QoS disabled on the DUT, congure the network according to Figure 25. 2. Set up the simulated trafc rate per type. Refer to Figure 26. Refer to Table 7 for the test Input Parameters. 3. Start the trafc, and run for the test duration. The trafc is received by the DUT and is not prioritized or classied. See Figure 27. Note the packet loss and the latency measurements in Figure 28 and 29. TEST 2 QoS is enabled on the DUT 1. Enable QoS on the DUT, and rerun the same test. 2. The trafc that is received (ingress) by the DUT is classied, prioritized and processed accordingly. See Figure 30. The resultant trafc (egress port) is measured for packet loss and latency. 3. Observe the new packet loss and latency measurements in Figures 31 and 32.

Figure 26. IxScriptMate QoS Many-to-One Test Setup

2006 by Ixia

p.27

www.ixiacom.com

Switch Test Plan

Results The results show some packet loss but no particular order in latency is shown when QoS is disabled. See Figure 27. The lower priority trafc (priority 0) still shows the highest packet loss.

Figure 27. IxScriptMate Log with QoS Disabled on the DUT

Figure 28. Receive and Loss Rate for All Three Streams

2006 by Ixia

p.28

www.ixiacom.com

Switch Test Plan

Figure 29. Latency for all Three Streams

Figure 30. IxScriptMate Log with QoS Enabled on the DUT

2006 by Ixia

p.29

www.ixiacom.com

Switch Test Plan

Figure 31. Receive and Loss Rate for all Three Streams

Figure 32. Latency for all Three Streams

2006 by Ixia

p.30

www.ixiacom.com

Switch Test Plan

Figure 33. QoS Many-to-One Test Per Port Statistics

2006 by Ixia

p.31

www.ixiacom.com

Switch Test Plan

7. Spanning Tree Network Convergence Performance Test

Objective This test veries that whenever the Path Cost to root changes, a bridge link goes down, or a bridge stops sending BPDUs during trafc generation on a switched LAN, that the Spanning Tree topology is recalculated to update all bridges on the network with the latest BPDU topology notication and changes. This test also measures the network convergence based on the DUT performance and handling of the Topology Changes Notications and Congurations BDPUs, as well as trafc switchover. This test case validates the following: DUTs Spanning Tree recalculation based on new Root Path Cost, a bridge link failure, or a bridge stops sending BPDUs Network topology changes and convergence due to the occurrence of any of the previously listed conditions Trafc switchover from one emulated bridge port to another due to the occurrence to any of the previous listed conditions Any trafc forwarded by the emulated bridge from Host A to Host B through the DUT bridge will be halted until the complete Spanning Tree is recalculated, the new Spanning Tree topology has stabilized, and all network bridges ports have reached their nal state. After the Spanning Tree is stabilized, the trafc is switched over from one path to another. This switchover mechanism should take about twenty-eight seconds for the Spanning Tree to complete, and is virtually immediate for the Rapid Spanning Tree protocol, since the later is designed with less port states to cycle through before the Spanning Tree network topology is stabilized.

2006 by Ixia

p.32

www.ixiacom.com

Switch Test Plan

Setup The baseline setup for this test requires three test ports. Each of the rst two test ports emulates a bridge connected to two separate physical ports on the DUT running Spanning Tree Protocol. In addition, a third DUT bridge port is connected to a non-Spanning Tree emulated test port for sending and receiving trafc. At startup, the emulated bridge is the root bridge for the network (set the emulated bridge ID to be the lowest ID by changing priority and/ or MAC address). Ixias IxRouter application can be used to set up and execute this test.
Traffic Path 1 Ixia Port 1
Host B Emulated Root
p1 BR1 p2 BR2

Ixia Port 3
Host A

DUT

Ixia Port 2 Path 2

The DUT learns about the emulated LAN hosts A and B Traffic initially travels over Path 1 (DUT Port 1 is in Forwarding state) Traffic switches over to Path 2 (after the Path cost to root changes, DUT Port 1 is in Blocking state, DUT Port 2 is in Forwarding state)

Figure 34. Multiple Spanning Tree Emulated Bridges Connected to the DUT

2006 by Ixia

p.33

www.ixiacom.com

Switch Test Plan

Input Parameters Parameters Root ID Description Contains the bridge ID of the root bridge. The root ID consists of the Priority, System ID and MAC Address. After convergence, all Conguration BPDUs in the bridged network should contain the same value for this eld. The cumulative cost of all links leading to the root bridge. Bridge mode type can be Spanning Tree or Rapid Spanning Tree.

Root Cost Bridge Mode

Table 8. Spanning Tree Network Convergence Input Parameters Methodology TEST 1 Trafc switchover due to Path Cost change 1. Set BR1 and BR2 Sending Root bridge MAC address to CC CC CC CC CC CC and priority 4096. The root is the imaginary emulated root bridge. BR1 and BR2 bridge ports are in Root Forwarding. One of the DUT bridge ports is in Alternate / Blocking state. Set up two trafc streams on BR1 and BR2 for the emulated Host B, and stream 3 for the emulated Host A. Stream 1 on BR1 is set with the Host B MAC address value for the DUT to learn Host B MAC address from BR1. Stream 2 on BR2 is set with the Host B MAC address value for the DUT to learn Host B MAC address from BR2. Set up trafc stream 3 on the emulated trafc receive / generation port for the DUT to learn Host A MAC address. Start trafc streams enabling the DUT to learn the MAC address of the emulated LAN nodes. Host A <- -> Host B trafc is going over one of the available DUT / emulated bridge Paths. See Figure 34. Select the emulated bridge that is forwarding the trafc, and change its Path Cost to Root from 0 to 3, forcing the Spanning Tree to be recalculated.

2. 3. 4.

5. 6. 7. 8.

2006 by Ixia

p.34

www.ixiacom.com

Switch Test Plan

9.

The trafc is temporarily halted due to the new topology change occurrence.

10. Once the Spanning Tree is stabilized and all ports have reached their nal states, the trafc will switch over to the other Path. See Figure 34. 11. Any other path from any of the bridges in the network to the root bridge (DUT) that is not needed in this switched network will be set to blocking state, avoiding redundant path to the Root and possible looping condition. TEST 2 Trafc switch over due to link down 1. Given that the trafc has been handled by one of the DUT ports that is connected to the emulated bridge port via one of the paths, select this emulated bridge port and simulate a cable disconnect. The same behavior as in previous step is observed: The Spanning Tree is recalculated based on the new topology change and the network converges. The trafc is once again switched over to the other available path. This process will only take few seconds.

2.

3.

TEST 3 Trafc switch over due to one bridge stopping BPDUs 1. Given that the trafc has been handled by one of the DUT ports that is connected to the emulated bridge port via one of the paths, select this emulated bridge and stop its Spanning Tree protocol. The same behavior as in previous step is observed: The Spanning Tree is recalculated based on the new topology change, and the network converges since one of the bridge ports has removed itself from the Spanning Tree topology. The trafc is once again switched over to the other available path. This process will only take a few seconds.

2.

3.

2006 by Ixia

p.35

www.ixiacom.com

Switch Test Plan

Results The success of this test depends on the convergence of the Spanning Tree and trafc switchover. This process will only take about 15-20 seconds for Spanning Tree mode (STP), and is virtually immediate for Rapid Spanning Tree mode (RSTP). The results shown below are captured for Spanning Tree mode (STP). The verication is calculated before and after the change of the Spanning Tree due to the new Path cost bridge parameters change. NOTE: The initial Spanning Tree state shows that the lowest cost to the root is the preferable path. To avoid looping, the Spanning Tree Protocol will block the other path from forwarding trafc. The other port will be set as Alternate/Blocking on the DUT. Before switchover, the bridge ports state show the following: Ixia BR1 port 1 Designated/Forwarding Ixia BR2 port 2 Designated/Forwarding DUT indicates the imaginary bridge with MAC is the Root (that is CC CC CC CC CC CC) DUT port 1 is Root/Forwarding DUT port 2 is Alternate/Blocking The trafc passes through Ixia port 1 as shown in Figure 34 prior to switchover. After switchover, the bridge ports state should show the following: Ixia BR1 port 1 Designated/Forwarding Ixia BR2 port 2 Designated/Forwarding The imaginary bridge with MAC is the Root (shown in the DUT) DUT port 1 is Alternate/Blocking DUT port 2 is Root/Forwarding

2006 by Ixia

p.36

www.ixiacom.com

Switch Test Plan

The trafc is shown passing through Ixia port 2 as illustrated in Figure 35 below, identifying the drop down of packets received on BR1, the delay (about 28 seconds), then the trafc picking up with the BR2.

Figure 35. Trafc Switch over from BR1 to BR2

2006 by Ixia

p.37

www.ixiacom.com

Switch Test Plan

8. OSPF Performance Test

Objective The OSPF performance test is designed to measure the nodrop throughput and latency by setting up dened routes and a topology and then measuring the no-drop throughput and latency between advertised ports. The test can be executed with either the OSPFv2 or OSPFv3 protocols. Setup This test requires two ports to be connected to the DUT. Each test port simulates routers and networks behind the routers on each side of the DUT. See Figure 36. Ixias IxScriptMate OSPF Performance Test can be used to setup and execute this test.
Ixia Port 1
OSPF Routes

Ixia Port 2
Traffic p1
DUT
OSPF Routes

p2

OSPF Routes

OSPF Routes

Simulated OSPF Routers Advertising Routes

Traffic is sent by the emulated test ports to the DUT, simulating routers and networks behind the routers The Traffic is forwarded back by the DUT to the test ports for performance and no-drop throughput analysis

Simulated OSPF Routers Advertising Routes

Figure 36. OSPF Performance Test Setup

2006 by Ixia

p.38

www.ixiacom.com

Switch Test Plan

Input Parameters Parameters Frame Size Trafc Rate OSPF Parameters DUT setup Description The selected frame size(s) used for the test Initial Trafc rate that the transmit port(s) will send OSPF area ID, number of emulated routers, number of emulated routes, Inter-area or External type routes DUT is congured for OSPFv2 or OSPFv3 operation

DUT OSPF Area DUT interfaces are set for OSPF area 0 (backbone) Table 9. OSPF Performance Input Parameters Table Methodology 1. Congure two test ports for OSPF. Refer to Table 9 for the Input Parameters. This test sets up a routing infrastructure and topology where several routers on each side of the DUT have been simulated with hundreds of routes behind each router. 2. Once route verication is successful, start trafc across the learned routes. 3. Observe the results once the test concludes. See Figures 37 and 38. Results This test shows the DUTs ability to handle OSPF routed trafc in addition to learning and announcing all the OSPF learned routes by using either a linear or binary search function. Refer to Figure 37 for the OSPF performance statistics per port describing Frame Loss, Latency, and Throughput per port. Figure 37 indicates that as the frame size increases from 64k to 128k, the no drop rate decreases from 58.33% down to 54.41%. The more interesting results appear in Figure 38, which illustrates that as the trafc is increased above the 55-56% mark, a noticeable increase in trafc loss is experienced. Such results and measurements indicate that the DUT is experiencing packet processing and route forwarding performance degradation.

2006 by Ixia

p.39

www.ixiacom.com

Switch Test Plan

Figure 37. OSPF Performance per Port Statistics

Figure 38. OSPF Performance Iterations Statistics

2006 by Ixia

p.40

www.ixiacom.com

About Ixia
Ixia is a leading provider of performance test systems for IP-based infrastructure and services. Its highly scalable solutions generate, capture, characterize, and emulate network and application trafc, establishing denitive performance and conformance metrics of network devices or systems under test. Ixias test systems are used by Network and Telephony Equipment Manufacturers, Semiconductor Manufacturers, Service Providers, Governments, and Enterprises to validate the functionality and reliability of complex IP networks, devices, and applications. Ixias Triple Play test systems address the growing need to test voice, video, and data services and network capability under realworld conditions. Ixias vision is to be the worlds pre-eminent provider of solutions to enable testing of next generation IP Triple Play networks. Ixias test systems utilize a wide range of industry-standard interfaces, including Ethernet, SONET, ATM, and wireless connectivity, and are distinguished by their performance, accuracy, reliability, and adaptability to the industrys constant evolution.

Contact Ixia

For more information, contact Ixia or visit our Web Site at http://www.ixiacom.com.

Ixia Worldwide Headquarters

Corporate Center 26601 W. Agoura Rd. Calabasas, CA 91302 (Toll Free North America) 1.877.367.4942 (Outside North America) +1.818.871.1800 (Fax) 818.871.1805 www.ixiacom.com Info: info@ixiacom.com Investors: ir@ixiacom.com Renewals: renewals@ixiacom.com Sales: sales@ixiacom.com Support: support@ixiacom.com Training: training@ixiacom.com

Ixia USA Sales

Phone: 1.866.355.4942

Email: sales@ixiacom.com Email: salescanada@ixiacom.com Email: saleschina@ixiacom.com

Ixia Canada Sales Ixia China Sales

Phone: 1.877.367.4942 Phone: +86.10.84549199

Ixia Europe, Middle East, & Africa Sales Phone: +44.1753.722056 Email: salesemea@ixiacom.com Ixia India Sales
Phone: +91.80.25633570 Email: salesindia@ixiacom.com Email: salesjapan@ixiacom.com Email: salesoceania@ixiacom.com Email: salessouthkorea@ixiacom.com Email: salesfederal@ixiacom.com

Ixia Japan Sales

Phone: +81.3.5365.4690

Ixia Oceania Sales Ixia South Korea

Phone: 1.818.292.1561 Phone: +82.11.897.1326

Ixia Federal Sales

Phone: 1.703.822.7527

 1998-2006 Ixia. All rights reserved. This publication may not be copied, in whole or in part, without Ixias consent. Ixia and its licensors retain all intellectual property rights in all products identied in this publication. Such products may be covered by one or more patents and/or pending patent applications, including but not limited to the following U.S. patents: 6,717,917; 6,408,335; 6,397,359; 6,061,725; 5,937,165; 5,881,237; and 5,838,919. All software and related documentation identied in this publication is licensed, not sold, pursuant to a separate license agreement between Ixia and the recipient. The recipients use of such software and documentation is subject to the terms of that agreement. Restricted Rights Legend Use, duplication, or disclosure by the U.S. Government is subject to the restrictions set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 and FAR 52.227-19. THIS PUBLICATION IS PROVIDED AS IS AND WITHOUT ANY WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED. IXIA SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT. THE INFORMATION HEREIN IS FURNISHED FOR INFORMATIONAL USE ONLY, IS SUBJECT TO CHANGE BY IXIA WITHOUT NOTICE, AND SHOULD NOT BE CONSTRUED AS A COMMITMENT BY IXIA. IXIA ASSUMES NO RESPONSIBILITY OR LIABILITY FOR ANY ERRORS OR INACCURACIES CONTAINED IN THIS PUBLICATION. Ixia, the Ixia four petal logo, and IxScriptMate are either trademarks or registered trademarks of Ixia in the United States and/ or other countries. All other trademarks belong to their respective owners.