Optimizing Streaming Performance - Developer Zone - National Instruments Page 1 of 7

Document Type: Tutorial

NI Supported: Yes
Publish Date: Sep 24, 2008

Optimizing Streaming Performance

Overview

There are numerous applications that require streaming large amounts of data at high speeds. Some examples include
RF recording and playback as well as acoustic imaging. While modern rotating medium hard drives allow for high
performance streaming, there are still limitations that might prevent hard drives from achieving optimal speeds. This
article will discuss various strategies to maximize your streaming performance.

Table of Contents

1. Introduction
2. Fragmentation
3. Cluster Size
4. File Placement
5. File I/O In LabVIEW
6. Other Considerations
7. Conclusion

Introduction

Figure 1 shows the main components of a mechanical hard drive. The components that mainly define the data streaming
performance of a mechanical hard-drive are the platter, the actuator and the read/write head. Platter(s) consists of
magnetic disks that spin at high speeds on the spindle and the actuator controls the position of the actuator arm. The
head on the actuator arm is used to read and write data to the platter. For such mechanical hard-drives, operating
systems such as Windows allocate file space from the outer rim inwards. This is also true for most RAID arrays.

Figure 1: Components of a mechanical hard-drive

Figure 2 illustrates how the read and write speeds drop off as the data is being read toward the inner diameter of the
platter. Angular velocity is proportional to the radius; at larger radii the angular velocity will be higher resulting in higher
data rates at the outer rim. Testing on a western digital hard drive showed performance Improveat your
62 MB/sec for a file at Login
ni.com experience. the or Create a user profile.
outer rim of the disk, and falls to 36 MB/sec at the inner rim.

http://zone.ni.com/devzone/cda/tut/p/id/7677 10/21/2009
Optimizing Streaming Performance - Developer Zone - National Instruments Page 2 of 7

Figure 2: Change in read and write performance of a mechanical hard-drive as the data is written from the outermost rim
to the innermost (X-Axis "0" -> Outermost Rim, "100" -> Innermost Rim)

Often times it is difficult to attain the specified read and write rates from a hard-drive or a RAID array. There are four
main reasons that might cause this:

1. Fragmented hard drive

2. How the disk is formatted and cluster unit size

3. File placement on disk

4. LabVIEW file I/O used (Binary vs Win32)

5. Maximizing Windows drive performance

Following sections explain these reasons in detail and also recommend some strategies on optimizing each category.

Fragmentation

A highly fragmented hard drive could result in lowered read and write speeds. Data is stored on the hard drive as
clusters and when a file has to be written, the operating system stores the file in whichever free clusters it can find. This
might create a file split up into clusters scattered throughout the drive platter. This requires the hard drive head to move
to different locations on the disk to perform reads and writes. It takes time to write to these scattered free clusters, and
when a file has to be read, it takes additional time for the head to get data from the clusters. Windows comes with a
defragment utility that can be accessed by going to Start > Programs > Accessories > System Tools > Disk
Defragmenter. This utility reallocates data from a particular fragmented file on to a contiguous block of clusters. Figure 3
shows an example of an highly fragmented hard-drive that will exhibit poor data streaming dates.

Figure 3: Example of a highly fragmented file

http://zone.ni.com/devzone/cda/tut/p/id/7677 10/21/2009
Optimizing Streaming Performance - Developer Zone - National Instruments Page 3 of 7

Cluster Size

How the disk is formatted also affects performance. When formatting a hard drive an NTFS logical partition, a master file
table is created on the drive that contains all the attributes for the drive volume. One parameter important when working
with large files is “Allocation unit size” which refers to the cluster size on the disk. When a file is saved to disk it usually
takes up one or more clusters. When using RAID, larger cluster sizes result in better sequential I/O performance by
reducing fragmentation on the drives and allowing larger files to be read in bigger chunks. Table 1 below gives an
example of how the sequential read and write rates of an 8 drive RAID array are affected with different cluster sizes.

Cluster size Write Speed Read Speed

64 kilobyte 452 MB/s 445 MB/s

512 byte 402 MB/s 306 MB/s

Table 1: Effect of cluster size on data read and write performance

The formatting options for a logical partitions can be accessed by right-clicking on My Computer and choosing Manage.
Go to Storage > Disk Management, right click on the hard drive and choose Format. Use the quick format option.
Formatting the drive will erase ALL data and should be done right in the beginning of setting up the system. Ideally, a
separate logical partition should be used for data storage than the one use to load the OS. Following figures illustrate
this procedure.

Figure 4: Formatting a logical partition to set allocation size

Figure 5: Select the highest allocation unit size available for the parition

http://zone.ni.com/devzone/cda/tut/p/id/7677 10/21/2009
Optimizing Streaming Performance - Developer Zone - National Instruments Page 4 of 7

My Profile
Another way to access formatting options is by right-clicking the relevant drive in Windows File Explorer, | RSS | Privacy | Legal | Contact NI
and choosing
© 2009 National Instruments Corporation. All
“Format”. However, there are less cluster size options here than in Disk Management.
rights reserved. | E-Mail this Page

Figure 6: Formatting partition and setting allocation unit size through Windows File Explorer

File Placement

File placement is also an important factor in hard disk streaming speed. If possible, a separate data drive formatted for a
large cluster size is ideal. Having a drive dedicated to data allows you to reformat the partition at will when needed. If
this is not possible, create a partition at the beginning of your drive for your data. This will allow all data to be written and
read from the fastest part of the disk, resulting in maximum performance when accessing and writing your data.

Figure 7: Placement of a file in the specific logical partition affects sequential read/write performance

File I/O In LabVIEW

LabVIEW offers the ability to write and read data to disk in a binary file format. Using the binary format LabVIEW buffers
and caches the data before writing. This limits streaming performance and it is difficult to write/read more than 100MB/s
using the binary file format in LabVIEW. In order to overcome this limitation, National Instruments has a Win32 File I/O
API available. LabVIEW 8.6 will have native support for the high speed Win32 File I/O API. If using LabVIEW 8.5 or
earlier, the Win32 Vis can be downloaded here. The writes using the Win32 API are non-buffered and is useful for

http://zone.ni.com/devzone/cda/tut/p/id/7677 10/21/2009
Optimizing Streaming Performance - Developer Zone - National Instruments Page 5 of 7

sequential reads and writes of large amounts of data at high data rates. It also disables Windows caching of data and
uses queues to increase efficiency.

Table 2 below gives a comparison of the performance between binary and Win32 file I/O.

Drives/Controller Type Binary Win32

1 (SATA) Write 46 58

4 (ICH7) Write 109 240

4 (Promise) Write 144 243

8 (RocketRAID) Read 58 438

12 (RocketRAID) Read 38 723

12 (RocketRAID) Write 68 617

Table 2: Sequential write performance difference between Binary File I/O & Win 32 File I/O

Other Considerations

Windows OS often times has background services running that access the hard-drive for file I/O. These services are
scheduled to run at specific time intervals and will cause a degradation of read/write streaming performance of the
application. If there is a separate hard-drive or RAID partition that is allocated to data streaming then these background
services can be disabled for this hard-drive. This will allow us to avoid unpredictable degradation in read/write
performance. Virtual memory, Recycle Bin & System Restore are few of these background services that can be turned
off for specific drives to increase and maintain high data streaming rates. For optimum performance Write Caching must
also be turned on for all drives. With Write Caching enabled, instead of writes being made immediately when demanded,
data is written to the disk cache and then written to the disk when the cache is filled. This results in less number of writes
and seeks. Figures 8 through 10 illustrate how to disable few of these background services.

http://zone.ni.com/devzone/cda/tut/p/id/7677 10/21/2009
Optimizing Streaming Performance - Developer Zone - National Instruments Page 6 of 7

Figure 8: Turning off System Restore can help in avoiding unpredictable streaming performance degradation

Figure 9: Disabling Recycle Bin

http://zone.ni.com/devzone/cda/tut/p/id/7677 10/21/2009
Optimizing Streaming Performance - Developer Zone - National Instruments Page 7 of 7

Figure 10: Turn on Write Caching to increase write performance

Conclusion

Disk performance is important when streaming data to and/or from disk. There are a number of ways to ensure that you
are maximizing the potential of your hard drives. Keeping the drive defragmented is important as this ensures that the
drive is not constantly writing or reading a file on different locations on the disk. Maximizing the cluster size will result in
better sequential I/O performance. You should also allocate your data partition at the beginning of the drive to take
advantage of the higher read and write speeds at that location. Using the high speed Win32 File I/O API that is natively
supported in LabVIEW 8.6 and higher will also maximize data streaming performance. Implementing these steps will
ensure maximum streaming performance from your National Instruments Data Storage hardware.

Reader Comments | Submit a comment »

Legal
This tutorial (this "tutorial") was developed by National Instruments ("NI"). Although technical support of this tutorial may
be made available by National Instruments, the content in this tutorial may not be completely tested and verified, and NI
does not guarantee its quality in any way or that NI will continue to support this content with each new revision of related
products and drivers. THIS TUTORIAL IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND AND SUBJECT
TO CERTAIN RESTRICTIONS AS MORE SPECIFICALLY SET FORTH IN NI.COM'S TERMS OF USE
(http://ni.com/legal/termsofuse/unitedstates/us/).

http://zone.ni.com/devzone/cda/tut/p/id/7677 10/21/2009