An Empirical Model of TCP Performance

Allen B. Downey
Olin College of Engineering

TCP Performance
Goal: model and predict transfer times.


Understand TCP performance

Improve TCP?
Improve the network?

Interactive applications.
Predict duration of downloads.
Select mirror site.

Distributed applications.
Resource selection.
Scheduling.
2

Goals
Complementary questions:


What do we need to
know about a
network path?

What can we
measure about a
network path?

Minimize
measurement.

Complementary goals:


Maximize accuracy.

Two approaches


History-based.
Pro: Good statistical description.
Con: Availability of data?
Passive: Might not have seen what you need.
Active: Obtrusive?

Model-based.
Pro: small measurements parameters
model predictions
Con: single-value predictions.
Con: incomplete models, so far.
4

TCP Models


Several models for mice (never leave slow start).

Several models for elephants (ignore slow start).

For medium size (bdp < size < 10bdp), performance

depends on slow start and steady state.

bdp commonly 4 KB 128 KB.

Maybe 40% of TCP transfers are in this range.

Hybrid model


Explicit model of slow start.

History-based prediction of steady state throughput.

ss0

ss1

ss2

ssn

ca

Model parameters
So what do we need to know?


State transition probabilities.

Distribution of rtt.

cw1 , cw2 , cw3 ...

Distribution of steady-state throughput.

Can we measure these parameters?

Measurement


Application-level HTTP timing (instrumented wget).

Pro: easy to implement.
Con: some timing inaccuracy, lost information.

Timing chart
server 7 timing chart

100

size (1000 B)

80

Plot bytes read

vs. time.

Immediately, we
can estimate rtt,
cw, bw and bdp.

And we can infer

TCP state.

60
40
20
0

100

200
300
time (ms)

400
9

Estimating parameters


Divide timing chart into rounds.

Measure window size for each round.

Pattern match on window sizes:

2, 4, 8, 16, 32 ...
2, 4, 6, 4, 5, 6 ...
2, 4, 6, (long pause) 11, 6 ...
3, 6, 12, 15, 15, 15 ...
3, 6, 12, 51, 17, 63 ...

10

Measurement


Measurements:
100,000 byte transfers.
100 transfers, with 100s between.

HTTP downloads:
2 URLs provided by collaborators.
11 URLs culled from proxy cache logs.

Diverse network paths:

rtt from 7 to 270 ms.
bw from 0.350 to 100 Mbps.
11

Window sizes
server 7 window sizes
observed window (packets)

4
3
2
1

40
30

This is the sort

of thing I
expected.

Too bad its the

exception.

20
10
0

20

40
60
measurement

80
12

Window sizes
server 3 window sizes
4
3
2
1

observed window (packets)

15

10

20

40
60
measurement

cw2 is
sometimes 3,
sometimes 4.

cwn+1 is
2 cwn m,
where m is
0, 1, 2, ...

10 out of 13 are
similar.

80
13

Non-deterministic slow start



Partial explanation in the paper.

Putting the empirical in empirical model.

Models that omit this behavior cant work for

shortmedium transfers (25 rounds of slow start).

14

Estimating Parameters


R: Distribution of rtt.

W : Distribution of window size.

T : Distribution of throughput.

State transition probabilities.

ss0

ss1

ss2

ssn

ca
15

Generating predictions
Monte Carlo simulation of finite state model:
1. Start at ss0 .
2. Choose a state transition at random.
3. If in slow start, choose rtt from R and cw from W.
4. If in congestion avoidance, choose throughput from T.
5. Update total time and total data transmitted.
6. If total data > size, return total time.
7. Otherwise go to step 2.

16

Validation


Randomly partition 2 datasets of 50 measurements.

Estimate parameters and generate distributions from

one subset.

Compare to actual times from other subset.

Agreement indicates that the model is sufficiently

detailed, and that the estimated parameters are
consistent.

17

Example #1
server 7 tts (cdf)

Pr (transfer time < x)

1.0
0.8
0.6
0.4

Deterministic
slow start.

Bandwidth
limited.

1000
5000
20000
50000
100000

0.2
0.0

62

119

231
x (ms)

447

866
18

Example #2
server 1 tts (cdf)

Pr (transfer time < x)

1.0

Nondeterministic
slow start
multimodal
distributions.

Modes at
multiples of rtt.

0.8
0.6
0.4

1000
5000
20000
50000
100000

0.2
0.0

563

1119

2227 4431
x (ms)

8815
19

Example #3
server 2 tts (cdf)

Pr (transfer time < x)

1.0

Buffer-limited.

0.8
0.6
0.4

1000
5000
20000
50000
100000

0.2
0.0

18

29

46
x (ms)

74

119
20

Example #4
server 9 tts (cdf)

Pr (transfer time < x)

1.0
0.8
0.6
0.4

Congestion
limited.

Underestimating
variability?

1000
5000
20000
50000
100000

0.2
0.0

121

419

1450 5011 17317

x (ms)
21

Evil case #1
server 3 tts (cdf)

Pr (transfer time < x)

1.0
0.8
0.6
0.4

Up to 50,000
bytes, not bad.

Anything over
60,000, way off!

1000
5000
20000
50000
100000

0.2
0.0

101

190

355
x (ms)

665

1244
22

Server performance


Server imposes 50 ms delay after 40 packets.

Model includes processing time in round one, but not

afterwards.

Reminder: real world resists modeling.

23

Evil case #2
server 6 tts (cdf)

Pr (transfer time < x)

1.0

Actual times are

sharply
multimodal.

Model smoothes
the modes.

0.8
0.6
0.4

1000
5000
20000
50000
100000

0.2
0.0

19

41

91
x (ms)

201

443
24

Multiple paths
server 6 timing chart

100

size (1000 B)

80

Akamai-style
content delivery
multimodal
rtt.

60
40
20
0

100

200
300
time (ms)

400
25

Headlines


Hybrid model captures transition from slow start to

steady state.
Lots of parameters, but easy to estimate.

Non-deterministic slow start sighted.

TCP characteristic or Linux bug?

26

Ongoing work


Followup paper:
TCP Self-Clocking and Bandwidth Sharing.

Network monitoring/predicting tool.

Collect active and passive measurements.

Maintain database.
Generate vizualizations.
Answer queries.

27

Had enough?


Followup paper and additional data available from

http://allendowney.com

Contact me at
downey@allendowney.com

28