Testing: Revit Architecture 2008 and 2009 with Riverbed

Steelhead Appliance and Steelhead Mobile

Executive Summary

Revit® Architecture from Autodesk is building design software which enables architects to
visualize ideas on the fly with 3D views and fully rendered scenes. More information on Revit
Architecture can be found at the following URL:

http://usa.autodesk.com/adsk/servlet/index?siteID=123112&id=8479253

Application performance testing of Revit Architecture using Steelhead and Steelhead mobile
shows significant improvement for both the cold and warm pass.

Revit 2008 Results Summary

Total Data Total Time to Data Time

Transferred Complete Reduction Improvement
Baseline 396.2 Megabytes 68.0 minutes
Steelhead Cold 76 Megabytes 20.0 minutes 80.8% 3.4 x
Steelhead Warm 6 Megabytes 14.6 minutes 98.5% 4.7 x

SHMobile Cold 166 Megabytes 28.9 minutes 58.1% 2.4 x

SHMobile Warm 8 Megabytes 14.6 minutes 98.0% 4.7 x

Note: the SHMobile data transferred is the sum of the both clients (individual data stores vs. single data store in the
SH appliance).

Revit 2009 Results Summary

Total Data Total Time to Data Time

Transferred Complete Reduction Improvement
Baseline 200 Megabytes 23.5 minutes
Steelhead Cold 20 Megabytes 5.7 minutes 90% 4.2 x
Steelhead Warm 4 Megabytes 3.9 minutes 98% 6.0 x

Performance Testing Overview:

The purpose of this test is to determine the benefits of the Steelhead appliance and Steelhead
Mobile software on Revit Architecture 2008 software in a WAN environment.

The testing simulates two Revit users working on separate local workstations. Each workstation
has the Revit Architecture 2008 software installed. Each user opens a common Central File on
a server located across a WAN, saves a Local copy to their computer, performs some work
(which involves getting permissions and borrowing elements from the Central File), and then
publishes their changes to the Central File. The tests use a custom API tool

1
(WorkScheduler.exe) so that the Saves to Central are near-simultaneous, simulating typical
real-life Central File conflicts

The simulation of the 2 users is done using a script provided by Autodesk. The script directs
each of the 2 users perform the following six operations. Operations 1 and 5 do not access files
across a network, resulting in consistent times for all the tested topologies.

1. Save as Local_File_User.rvt
2. Make some changes to User local file
3. User Save to Central
4. User Reload Latest
5. Make some changes to User Local File
6. User Save to Central

The WAN environment used in this case is a simulated T1 with 100 ms of latency. A baseline
test was run without any Steelhead appliances, and then with the Steelhead appliances and
finally with the Steelhead mobile client software.

The server and workstations are running Windows XP with service pack 2 as the OS. The client
workstations are running the June 2007 release of Revit Architecture 2008.
Steelheads 300s with version 4.1 software are used for Steelhead Mobile testing version 1.1 of
the software was used.

Topology 1: Revit Architecture 2008 without the Steelheads.

The Revit operations would not complete without adjusting the synch timer of the software
application in this topology. The default value of the timer is 600 seconds. It was adjusted to
1200 seconds.

2
The following is a graph of the baseline performance of the Revit Architecture software across a
WAN. The graph combines the 6 operations for each of the 2 users. For individual breakdown of
test results refer to Appendix A.

W AN Utilization (bi-directional)

250

200
Throughput (kbyte)

150

100

50

0
1 331 661 991 1321 1651 1981 2311 2641 2971 3301 3631 3961
Time (seconds)

Baseline

___ Baseline 396.2 Megabytes 68 minutes

Topology 2: Revit Architecture 2008 with Steelheads Appliances

In the second topology the same tests were performed as in topology 1 with the addition of the
Riverbed Steelhead appliances. The test was performed with both cold and warm data store on
the Steelhead. The following topology was used for the testing.

The following graph shows the comparison of baseline performance of the Steelheads with cold
and warm data store passes. This is the combined stats of the 6 different functions that were
run in the test script. For detailed breakdown of each individual test result refer to Appendix A.

3
 WAN Utilization (bi-directional)

250

200
Throughput (kbyte)

150

100

50

0
1 208 415 622 829 1036 1243 1450 1657 1864 2071 2278 2485 2692 2899 3106 3313 3520 3727 3934
Time (seconds)

Baseline Cold Warm

___ Baseline 396.2 Megabytes 68 minutes

___ Cold 76 Megabytes 20 minutes
___ Warm 6 megabytes 14 minutes 35 seconds

Topology 3: Revit Architecture 2008 with Steelhead Mobile

The third topology performed the same tests using Steelhead Mobile software on the clients and
the Steelhead appliance at the server site. The tests were performed with both cold and warm
data store on the Steelhead. The following topology was used for the testing.

4
The following is a graph comparing the baseline, cold and warm performance of the Steelhead
Mobile software. Results of SH Mobile and SH appliance was similar for warm transfer. In the
cold performance the topology using just Steelhead appliances performed better. This is
expected due to the shared datastore on the client side when the Steelhead appliance was
used, vs. individual data stores for the Steelhead Mobile client. For detailed breakdown of test
results refer to Appendix A.

WAN Utilization (bi-directional)

300

250
Throughput (kbyte)

200

150

100

50

0
1 331 661 991 1321 1651 1981 2311 2641 2971 3301 3631 3961
Time (seconds)

Baseline Cold Warm

___ Baseline 396.2 Megabytes 68 minutes

___ Cold 166 Megabytes 28 minutes 51 seconds
___ Warm 8 megabytes 14 minutes 36 seconds

5
Topology 4: Revit Architecture 2009 with Steelhead Appliances

Testing was also performed with Revit 2009 and Steelhead appliances to ensure continued
interoperability between the 2 products. It was also mentioned by several Revit customers that
the average size of a shared Revit file is 150 MB so testing was modified to use a 200 MB file to
simulate a more real life environment. Testing was performed with a simulated 5mb link with
100 ms of delay. The testing with Revit 2009 was streamlined to test only the operations that
are WAN intensive, namely, Save_to_Central and Reload_test.

User1___ Baseline 1412 seconds

User1___ Cold 90% data reduction 340 seconds
User1___ Warm 98% data reduction 234 seconds

User2___ Baseline 1411 seconds

User2___ Cold 90% data reduction 339 seconds
User2___ Warm 98% data reduction 223 seconds

6
Appendix A – Detailed performance results

Revit Architecture Performance Tests

Baseline - No Steelheads
Operation User 1 Run 1 Run 2 Run 3 Avg
Save as Local_File_User1.rvt 19 seconds 19 seconds 19 seconds 19
Make some changes to User1 712 seconds 710 seconds 710 seconds 711
User1 Save to Central 859 seconds 797 seconds 823 seconds 826
User1 Reload Latest 223 seconds 223 seconds 223 seconds 223
Make some changes to User1 Local File 63 seconds 62 seconds 62 seconds 62
User1 Save to Central 616 seconds 604 seconds 608 seconds 609

Operation User 2 Run 1 Run 2 Run 3 Avg

Save as Local_File_User2.rvt 14 seconds 14 seconds 14 seconds 14
Make some changes to User2 67 seconds 77 seconds 67 seconds 73
User2 Save to Central 860 seconds 795 seconds 860 seconds 838
User2 Reload Latest 224 seconds 222 seconds 222 seconds 223
Make some changes to User2 Local File 62 seconds 61 seconds 61 seconds 61
User2 Save to Central 617 seconds 603 seconds 617 seconds 612

Steelheads Cold Transfer

Operation User 1 Run 1 Run 2 Run 3 Avg
Save as Local_File_User1.rvt 19 seconds 18 seconds 19 seconds 18
Make some changes to User1 149 seconds 144 seconds 146 seconds 146
User1 Save to Central 269 seconds 289 seconds 267 seconds 275
User1 Reload Latest 26 seconds 27 seconds 27 seconds 27
Make some changes to User1 Local File 56 seconds 56 seconds 56 seconds 56
User1 Save to Central 299 seconds 252 seconds 173 seconds 241

Operation User 2 Run 1 Run 2 Run 3 Avg

Save as Local_File_User2.rvt 17 seconds 14 seconds 14 seconds 15
Make some changes to User2 66 seconds 65 seconds 66 seconds 66
User2 Save to Central 267 seconds 288 seconds 266 seconds 273
User2 Reload Latest 27 seconds 27 seconds 28 seconds 27
Make some changes to User2 Local File 55 seconds 55 seconds 55 seconds 55
User2 Save to Central 301 seconds 254 seconds 174 seconds 243
Steelheads Warm Transfer
Operation User 1 Run 1 Run 2 Run 3 Avg
Save as Local_File_User1.rvt 19 seconds 19 seconds 19 seconds 19
Make some changes to User1 147 seconds 147 seconds 146 seconds 147
User1 Save to Central 163 seconds 185 seconds 166 seconds 171
User1 Reload Latest 27 seconds 27 seconds 27 seconds 27
Make some changes to User1 Local File 56 seconds 56 seconds 56 seconds 56
User1 Save to Central 122 seconds 122 seconds 122 seconds 122

Operation User 2 Run 1 Run 2 Run 3 Avg

Save as Local_File_User2.rvt 14 seconds 14 seconds 14 seconds 14
Make some changes to User2 66 seconds 65 seconds 66 seconds 66
User2 Save to Central 186 seconds 163seconds 164 seconds 171
User2 Reload Latest 26 seconds 27 seconds 28 seconds 27
Make some changes to User2 Local File 55 seconds 55 seconds 55 seconds 55
User2 Save to Central 123 seconds 122 seconds 123 seconds 123

7
Steelhead Mobile Cold Cold
Transfer
Operation User 1 Run 1 Run 2 Run 3 Avg
Save as Local_File_User2.rvt 22 seconds 19 seconds 19 seconds 20
Make some changes to User2 402 seconds 390 seconds 398 seconds 397
User2 Save to Central 322 seconds 323 seconds 340 seconds 328
User2 Reload Latest 113 seconds 112 seconds 111 seconds 112
Make some changes to User2 Local File 56 seconds 55 seconds 62 seconds 58
User2 Save to Central 325 seconds 320 seconds 325 seconds 323

Operation User 2 Run 1 Run 2 Run 3 Avg

Save as Local_File_User2.rvt 13 seconds 12 seconds 12 seconds 12
Make some changes to User2 67 seconds 70 seconds 70 seconds 69
User2 Save to Central 322 seconds 303 seconds 320 seconds 315
User2 Reload Latest 112 seconds 110 seconds 110 seconds 111
Make some changes to User2 Local File 56 seconds 55 seconds 55 seconds 55
User2 Save to Central 326 seconds 323 seconds 323 seconds 324

Steelheads Mobile Warm Transfer Run 1 Run 2 Run 3 Avg

Operation User 1
Save as Local_File_User1.rvt 22 seconds 20 seconds 22 seconds 21
Make some changes to User1 218 seconds 238 seconds 262 seconds 239
User1 Save to Central 162 seconds 178 seconds 160 seconds 167
User1 Reload Latest 26 seconds 26 seconds 28 seconds 27
Make some changes to User1 Local File 59 seconds 56 seconds 58 seconds 57
User1 Save to Central 125 seconds 122 seconds 125 seconds 124

Operation User 2 Run 1 Run 2 Run 3 Avg

Save as Local_File_User2.rvt 13 seconds 14 seconds 14 seconds 13
Make some changes to User2 66 seconds 65 seconds 66 seconds 66
User2 Save to Central 161 seconds 178 seconds 160 seconds 166
User2 Reload Latest 27 seconds 27 seconds 28 seconds 27
Make some changes to User2 Local File 58 seconds 55 seconds 58 seconds 57
User2 Save to Central 125 seconds 123 seconds 126 seconds 125
LAN Transfer
Operation User 1 Run 1 Run 2 Run 3 Avg
Save as Local_File_User1.rvt 20 seconds 20 seconds 16 seconds 19
Make some changes to User1 215 seconds 211 seconds 218 seconds 214
User1 Save to Central 283 seconds 275 seconds 276 seconds 278
User1 Reload Latest 12 seconds 13 seconds 12 seconds 12
Make some changes to User1 Local File 33 seconds 31 seconds 31 seconds 32
User1 Save to Central 101 seconds 98 seconds 98 seconds 99

Operation User 2 Run 1 Run 2 Run 3 Avg

Save as Local_File_User2.rvt 16 seconds 19 seconds 18 seconds 17
Make some changes to User2 144 seconds 138 seconds 140 seconds 139
User2 Save to Central 283 seconds 273 seconds 276 seconds 277
User2 Reload Latest 12 seconds 13 seconds 13 seconds 12
Make some changes to User2 Local File 32 seconds 31 seconds 31 seconds 31
User2 Save to Central 101 seconds 98 seconds 97 seconds 98