Documente Academic
Documente Profesional
Documente Cultură
www.veritest.com • info@veritest.com
Network Appliance™
FAS3050, R200, and EMC Centera G4
Performance Comparison
Test report prepared under contract from Network Appliance
Executive Summary
Key Findings
! "#$%$ & $$
'() ) *+ , ! "#$%$ + #53 2,
6
+ 7 " 8) *+ +5
& $$6 , 8 +%3
, , - ! "#$%$
.
- ( ) *+ 3
'/ $$#
&- 0 ! "#$%$
"0 1' -
2 $" , - ! "#$%$ 5$3
) *+7 & $$ 9+3
- , - ! "#$%$ %
The back-end storage architecture ) *+
is not the only factor when
evaluating performance. Archival
, ! "#$%$ #%3 2, 6
+ 7
performance also depends on the " 8) *+
application use and application , - ! "#$%$
server configuration. For this ) *+ 3
4- - , - ! "#$%$ +53
comparison, we generated 80.4 ) *+
million e-mails using Microsoft ! "#$%$ - / - 0
LoadSim 2003, populating eight ) *+ - -
It should be noted that we used a Network Appliance FAS3050 and a FAS3020 as the primary back-end
storage for the Exchange and SQL databases. These were mutually exclusive arrays, separate from the
archive storage platforms. The Exchange host servers connected to the FAS3050 and the FAS3020 via Fibre
Channel using a Brocade 3800 Silkworm Fibre Channel switch. A detailed diagram of the performance test
bed can be found in “Testing Methodology.”
The EMC Centera G4 configuration followed the EMC Centera Best Practices Guide. A total of 16 G4 nodes
were deployed. We assigned 4 nodes access roles, with the remaining 12 nodes assigned a storage role.
Each storage node contained four 500GB SATA drives, for a total of forty-eight 500GB drives. All nodes were
dedicated to their assigned role. This 3:1 storage node-to-access-node ratio exceeds the EMC best practices
for small object workloads (<100K avg). The Centera was deployed using the default CPM data protection
scheme as well as the Storage Strategy Performance setting to ensure optimal write performance in small
object environments such as the e-mail archival environment.
®
We compared the Centera G4 against two platforms from Network Appliance: the NearStore R200 and a
single-node FAS3050. The R200 uses a single controller, or node. as well. We configured the R200 with one
aggregate (or pool) of forty-eight 320GB SATA drives and four dedicated Gigabit Ethernet interfaces. The
FAS3050 was also configured with four dedicated Gigabit Ethernet interfaces and a single aggregate of
twenty-four 250GB SATA drives. Aside from capacity differences, both EMC and Network Appliance platforms
used 7200 RPM SATA drives. The Enterprise Vault archive servers communicated to the Centera via the
®
Centera access APIs. We used a standard CIFS or Windows share connection to the FAS3050 and R200 for
their respective tests.
We ran two scenarios. In the first scenario, we conducted what was termed a 100% deployment, composed of
eight Exchange 2003 servers and eight Enterprise Vault servers, along with one Microsoft SQL server. We
believed that the eight Exchange 2003 servers would generate a sufficient workload for the eight Enterprise
Vault servers. In the second scenario, we conducted a 50% deployment using four Exchange 2003 servers
and four Enterprise Vault servers, with one Microsoft SQL server.
To create our simulated e-mail environment, we used the industry-standard Microsoft Exchange Server 2003
Load Simulator (LoadSim), available from Microsoft at
http://www.microsoft.com/downloads/details.aspx?familyid=92EB2EDC-3433-47CA-A5F8-
0483C7DDEA85&displaylang=en. Using the LoadSim default message mixture, we generated 80.4 million e-
mail messages across 15,000 mailboxes, for an average of 5,360 e-mail messages per user. Each Exchange
Server also contained 5,000 unique e-mails (5 mailboxes with 1,000 messages each), generated using the
Mailsend.exe tool and containing a 50/50 split of embedded text and attached text with a unique string used
for retrieval by the Symantec Enterprise Vault Discovery Accelerator application. We stored these e-mails on
the FAS3050 and FAS3020 Fibre Channel connected arrays mentioned earlier. For the archive storage
hardware, we used an authorized service partner (Unisys) and the publicly available best practices for the
installation and configuration of the Centera G4. We also used the publicly available best practices
documentation from Network Appliance to provision the FAS3050 and the R200. We archived approximately
1.7 million e-mails from a population of 80.4 million for each execution of the Enterprise Vault archiving tasks
during the 100% deployment testing. The archival volume was half this value when running the 50%
deployment tests.
Overall we found that a single-node FAS3050 with twenty-four 250GB 7200RPM drives scaled performance
at a rate 42% greater than a 16-node Centera G4 with forty-eight 500GB 7200RPM drives, even as we
continued to fill the file systems on the FAS3050 past 50% full. Moving from the 50% deployment up to the
100% deployment, the FAS3050 archival rate increased from 157MB/minute to 261MB/minute, a
104MB/minute improvement. The Centera G4 moved from 116MB/minute to 189MB/minute, an improvement
of only 73MB/minute.
250
104 MB/Minute
200
73 MB/Minute
150
100
50
0
FAS3050 Centera
Please refer to “Testing Methodology,” later in this report, for complete details of how we conducted the
performance testing on the FAS3050, R200, and Centera G4.
Monitoring the archival process over the course of one hour, we found that the Network Appliance R200
(single node) achieved the highest archival performance rate, averaging 275MB/minute ingestion of content
storage—45% faster than the Centera G4 and 5% faster than the FAS3050. The FAS3050 (single node)
averaged 261MB/minute, while the Centera G4 ingested content at a rate of 189MB/minute, putting the
FAS3050 ingestion rate at a level 38% greater than that of the Centera G4. Moving forward, we considered
this the baseline archival rate for each platform.
300
250
200
150
100
50
With the EMC implementation of a RAIN architecture, the storage node containing a single failed disk must be
brought offline prior to replacing the failed disk. Each Centera storage node contains four 500GB SATA
drives. In consultation with EMC technical resources, we received the following instructions for conducting a
drive failure/pull test:
If a single drive in a storage node is pulled. it cannot immediately be returned into the Centera. The
drive must be returned to EMC for repair and a spare drive obtained.
Upon receipt of the replacement drive, the procedure for the single-drive replacement requires:
In order to simulate a drive failure given this constraint, all drive failure test data used in this report were
obtained by powering off a Centera storage node from the front panel and unplugging the dual power
supplies. The second storage node powered off in the same manner. Powering the storage nodes back up
brought them back online.
A single-drive failure on the Network Appliance systems resulted in little or no performance degradation. The
R200 decreased 6.5% during this drive failure test. The FAS3050 actually showed a slight statistical
improvement in throughput capacity (up 1.5%), albeit at a higher utilization rate. A storage node failure on the
Centera G4 resulted in a 22% drop in performance.
In an archival test while experiencing a single-drive failure, the FAS3050 had an ingestion rate 80% greater
than the Centera G4 (265MB/minute vs. 147MB/minute), while the R200 showed a 74% advantage over the
Centera G4.
250
200
150
100
50
0
R200 FAS3050 Centera
Double-Disk Failure
Today, the likelihood of experiencing a media/bit error during reconstruction of an eight-drive RAID 4/5 set
with 500GB SATA drives has reached 28% (see Appendix C, “Application Data Integrity with RAID-DP,” for
supporting information). With that in mind, we constructed a dual-drive failure scenario. To determine
performance impact to each system, we again measured archival rates using EV Archival Throughput as
reported on the Enterprise Vault saveset SQL table.
In a two-drive failure scenario, the FAS3050 again maintained an archival rate within 5% of its baseline.
However, the R200 dropped 36% from its baseline archival rate, while the Centera fell 41% from its baseline.
Relative to each other under a two-drive failure scenario, the FAS3050 archived 124% (2.24 times) faster
than the Centera G4. The R200 experienced an archival rate that was 56.36% faster than the Centera G4.
Again, it’s important to note that the FAS3050 used half the data drives of the Centera G4.
250
200
150
100
50
0
R200 FAS3050 Centera
Baseline Single Drive Failure Dual Drive Failure
During this test, we observed the largest impact to the FAS3050 in any test to this point; its performance
dropped 22% from baseline archival rate. Likewise, the R200 archival rate dropped to its lowest level, off 52%
from its baseline archival rate. The R200 showed an archival rate 18% below that of the Centera G4 during
this index rebuild test. The Centera G4 dropped 17% from its baseline. Relative to the FAS3050, the
FAS3050 maintained a 30% performance advantage over the Centera G4. Another way to look at this is that
the FAS3050 during an index rebuild statistically outperformed the Centera G4 baseline.
The following figure summarizes all measured archival rates, including the index rebuild test.
Monitoring the archival process over the course of one hour, we found that the FAS3050 averaged 157
MB/minute, while the Centera G4 ingested content at a rate of 116MB/minute, putting the FAS3050 ingestion
rate at a level 35% greater than that of the Centera G4. Moving forward, we considered this the baseline
archival rate for each platform.
300
250
200
150
100
50
FAS3050 Centera
Single-Disk Failure
In order to test a drive failure condition, we physically pulled a single-disk drive from the drive shelf enclosures
on the FAS3050. We powered off a single storage node (vs. an access node) on the Centera G4. To
determine the performance impact on each system, we measured archival rates using EV Archival
Throughput as reported on the Enterprise Vault saveset SQL table.
As in the 100% deployment scenario, a single-drive failure on the FAS3050 showed no impact on its overall
archival performance. A storage node failure on the Centera showed a subsequent drop in archival
performance of 11%. The FAS3050 ingested content at a rate 54% greater than the Centera G4.
250
200
150
100
50
0
FAS3050 Centera
Double-Disk Failure
We then conducted a dual-drive failure test for the 50% deployment test environment. To determine
performance impact to each system, we again measured archival rates using EV Archival Throughput as
reported on the Enterprise Vault saveset SQL table.
In a two-drive failure scenario, the FAS3050 maintained an archival rate within 5% of its baseline. With the
failure of two storage nodes, the Centera experienced a 12% decrease in performance relative to its baseline.
Relative to each other, the FAS3050 archived e-mail 48% faster than the Centera G4.
250
200
150
100
50
0
FAS3050 Centera
Baseline Single Drive Failure Dual Drive Failure
During the Discovery Accelerator search and retrieval of the 20,000 unique e-mails, the archival rate for the
FAS3050 fell 7% from its baseline rate, dropping from 157MB/minute to 145-MB/minute. The Centera G4 fell
20% from its baseline. Relative to each other, we found that during the discovery process the FAS3050
ingested content at a rate 58% greater than that of the Centera G4. The discovery process itself ran well for
both systems, completing in 57 minutes on the FAS3050 and 1 hour 5 minutes on the Centera.
The index rebuild dropped the content archival rate 24%, from 157MB/minute to 118MB/minute. Once again,
by way of comparison, the FAS3050 during an index rebuild statistically outperformed the Centera G4
baseline. The Centera G4 archival performance dropped 14% during the index rebuild. Relative to the
FAS3050 performance during its index rebuild, the Centera G4 was 18% slower.
The following figure summarizes all measured archival rates in each test.
For testing purposes, we created a typical medium-sized enterprise Microsoft Exchange e-mail environment
with 80+ million e-mails, spread evenly across 15,000 mailboxes, and archived a percentage of those e-mails
to three types of NAS storage solutions, using the Symantec Enterprise Vault application. We then measured
the archival throughput performance of Enterprise Vault for each of the NAS storage devices while the
archival tasks were running. Each test was conducted independently. We generated two deployment levels,
based upon differing numbers of available Enterprise Vault and Exchange servers.
We first deployment level consisted of four Enterprise Vault Servers and four Microsoft Exchange 2003
Servers configured in a one-to-one relationship. This deployment is referred to as the 50% deployment level.
The second deployment level consisted of eight Enterprise Vault servers and eight Microsoft Exchange 2003
servers, also configured in a one-to-one relationship. This deployment is referred to as the 100% deployment
level.
For the Microsoft Exchange servers, 15,000 users were generated using the LoadSim 2003 application to
populate the Exchange data evenly across the available Exchange Servers. We then created five unique
mailboxes per Exchange server, and used the mailsend.exe utility to generate 1,000 unique e-mails for each
of those accounts. Therefore, for the 50% testing there were 20,000 unique e-mails available for Discovery
Accelerator search and retrieval, and 40,000 unique e-mails available at the 100% level.
We started by determining the baseline throughput when only archival tasks were executing. We then
introduced scenarios that one would expect to encounter in a real-world deployment, and measured the
Enterprise Vault archival throughput during those situations. Each deployment was configured with a single
Windows 2003 Active Directory server to manage the AD domain and a single SQL Server 2000 to contain
the database information for the Enterprise Vault application. Further, we broke the Enterprise Vault archival
tasks into two rounds of execution.
The first round archived e-mail for 35 LoadSim-generated mailboxes per Exchange Server, along with the 5
unique mailboxes on each Exchange Server. For this first round of archiving tasks, we gave Enterprise Vault
45 minutes to ramp up. We then measured for 60 minutes to determine the baseline archival throughput. To
simulate a single-disk failure, we removed a single-parity disk from the active aggregate on the NetApp NAS
devices, and powered off a single storage node on the Centera G4. We then measured the Enterprise Vault
archival throughput for 60 minutes to monitor for any performance change. We then simulated a double-disk
failure by removing a second disk in the aggregate on the NetApp NAS devices, and by powering off a
second storage node on the Centera G4. Again, we measured the archival throughput to monitor for any
change in performance. We then reinserted the disks on the NetApp NAS devices, powered on the downed
nodes on the Centera G4, and waited for the archival tasks to finish before moving on to the next scenario.
A second Enterprise Vault archiving task was executed using 40 newly enabled LoadSim-generated
mailboxes for each Exchange server. We again allowed Enterprise Vault 45 minutes to ramp up, and
monitored for 60 minutes to verify the baseline archival throughput rate. We then executed a Discovery
Accelerator search and retrieval to retrieve the unique e-mails previously archived, while the current archiving
Server performance impact was measured through Perfmon, to monitor the relative health and performance
of the Enterprise Vault, Exchange, SQL, and domain controller servers. During the Centera G4 testing, the
Centera Viewer utility was used to monitor the performance of the G4. During the FAS3050 and R200
®
testing, the native Data ONTAP command sysstat was used to monitor the performance of the
FAS3050/R200 devices. To monitor the performance of the Brocade 3800 Silkworm FCP switch, we ran the
portperfshow command continuously to real-time monitor the throughput for each Exchange server to
verify that there were no performance bottlenecks coming from the Exchange storage. We used the NetGear
administration GUI to monitor the Gigabit Ethernet switch to make sure that no errors or performance
bottlenecks were experienced on that device as well.
The following code shows the SQL query used to retrieve the EV archival throughput rate.
select "Archived Date" = cast (archiveddate as smalldatetime),
"Hourly Rate" = count (*)*60,
"Av Size" = sum (itemsize)/count (*)
from saveset
group by
cast (archiveddate as smalldatetime)
order by
"Archived Date"
desc
Networking Equipment
Brocade SilkWorm 3800
Fibre Channel Switch Firmware v3.1.3
Gig-E Non Blocking Managed
Switch
NetGear GS748T
The dramatic increase in disk sizes, the relatively slower growth in disk bandwidth, the construction of disk
arrays containing larger numbers of disks, and the use of less reliable and poorer performing varieties of disk
such as ATA combine to increase the rate of double-disk failures. The use of algorithms can protect against
double-disk failures and ensure adequate data integrity. For a complete discussion of algorithms that protect
against two-disk failures, refer to the Usenix publication “Row-Diagonal Parity for Double Disk Failure
Correction” http://www.usenix.org/publications/library/proceedings/fast04/tech/corbett/corbett.pdf (Peter
Corbett, Bob English, Atul Goel, Tomislav Grcanac, Steven Kleiman, James Leong, and Sunitha Sankar).
• Disk drive replacement: The failure of a disk drive in a single-parity RAID group presents a substantial risk
of data loss during the RAID rebuild/reconstruction process. Some disk replacements result from transient
drive, array, or loop problems and thus are not hard failures; but any event that causes a RAID rebuild
creates a risk. How often drives are replaced is typically a function of many factors, including physical
environment, I/O workload, drive age, drive production batch, drive design generation, etc. Experience to
date shows that SATA drives can be every bit as reliable as FC drives when they are used for their
appropriate workloads, but there is some indication that SATA drive failure rates may increase more than
FC as the drives age. More important than debating how often one or the other drive type fails is the
understandings that drive failures will occur, thus presenting risk of data loss without double-parity
protection.
• Bit error likelihood: Typical non-recoverable (or uncorrectable) error rate (UER) specs are 1 error in 10^15
bits read for FC drives, and 1 in 10^14 for SATA drives.
Note:
• The specs assume that full data recovery procedures are available, but these entail more read retries
than RAID array system timeout limits allow, which may result in not achieving the spec error rate (that is,
a higher incidence of errors may result). The 0.2% rate applies to 300GB FC drives; it is the number of
bits per drive divided by the UER spec of 1 in 10^15. The same calculation for a 500GB SATA drive with
a UER spec of 1 in 10^14 results in a 4% rate.
• Double-parity RAID bit error likelihood: A more complex calculation that incorporates the probability of a
second drive failing during RAID reconstruction of another drive in the same 16-drive RAID-DP group and
a bit error occurring while both drives are in rebuild.
The rate of single-disk failures per year squared times the reconstruction time in years times the
number of two-disk combinations in the array.
This result is (0.029 failures/year)^2 x 7.9E-4 years x 16*15/2 combinations, which equals 0.08E-
3 failures per year.
The probability of another media error in the array is the number of bits left to read times the error
rate per bit: 500GB*8 * 14 * 1E-14 = 0.56
Multiplying, the result is 0.08E-3 * 0.56 = 0.045E-3 data loss failures per year.
This is quite a lot higher than 1 in 30 million, so it is a conservative estimate; however, it is a
significant improvement in data protection relative to single-parity RAID.