MCS SNR MIMO RI BLER 64QAM 16QAM QPSK CQI L1 DL RSSI RSRQ RSCP CINR ACK NACK PDSCH

HS-PDSCH HSSCCH MAC-HS E-DPCH FTP UDP HTTP RTT TCP dBm dB Redefining Research Mbps ms MOS LTE HSPA+ DC-HSDPA TD-LTE EV-DO Rev A/B Mobile WiMAX MCS MCS SNR MIMO RI BLER 64QAM 16QAM QPSK CQI L1 DL RSSI RSRQ RSCP CINR ACK NACK PDSCH HS-PDSCH HS-SCCH MAC-HS E-DPCH FTP UDP HTTP RTT TCP dBm dB Mbps ms MOS LTE HSPA+ DCHSDPA TD-LTE EV-DO Rev A/B Mobile WiMAX MCS MCS SNR MIMO RI BLER 64QAM 16QAM QPSK CQI L1 DL RSSI RSRQ RSCP CINR-20ACK NACK PDSCH HS-PDSCH HS-SCCH -30 -40 -50 MAC-HS E-DPCH FTP UDP HTTP RTT TCP dBm dB Mbps ms MOS LTE HSPA+ DC-HSDPA TD-LTE EV-DO Rev A/B Mobile WiMAX MCS MCS SNR MIMO RI BLER 64QAM 16QAM QPSK CQI L1 DL RSSI RSRQ RSCP CINR ACK NACK PDSCH HS-PDSCH HS-SCCH MAC-HS E-DPCH FTP UDP HTTP RTT TCP dBm dB Mbps ms MOS LTE HSPA+ DC-HSDPA TD-LTE EV-DO Rev A/B Mobile WiMAX MCS MCS SNR MIMO R BLER 64QAM 16QAM QPSK CQI L1 DL RSSI RSRQ RSCP CINR ACK NACK PDSCH HS-PDSCH HS-SCCH MAC-HS E-DPCH FTP UDP HTTP RTT TCP dBm dB Mbps ms MOS LTE HSPA+ DC-HSDPA TD-LTE EV-DO Rev A/B Mobile WiMAX MCS MCS SNR MIMO RI BLER 64QAM 16QAM QPSK CQI L1 DL RSSI RSRQ RSCP CINR ACK NACK PDSCH HS-PDSCH HS-SCCH MAC-HS E-DPCH FTP UDP HTTP RTT TCP dBm dB Mbps ms MOS LTE HSPA+ DC-HSDPA TD-LTE EV-DO Rev A/B Mobile WiMAX MCS MCS SNR MIMO RI BLER 64QAM 16QAM QPSK CQI L1 DL RSSI RSRQ RSCP CINR ACK NACK How not to spend your summer vacation PDSCH HS-PDSCH HS-SCCH MAC-HS E-DPCH FTP UDP VOLUME 2 PREVIEW EDITION HTTP RTT TCP dBm dB Mbps ms MOS LTE HSPA+ DCHSDPA TD-LTE EV-DO Rev A/B Mobile WiMAX MCS MCS
8,000 kbps 7,000
Other Nokia Kyocera Palm LG
100% 90% 80% 70%

Other

HTC

Motorola

Samsung

Apple

HSPA Demo Phone - Cat 8 (1Rx)

Other RIM

October 19, 2011, Vol. 7 No. 13 PREVIEW

6,000 5,000

60% 50%

4,000 3,000

Total DC-HSDPA PHY Layer Throughput (Kbps)

52.9%

35.7%

40% 30%

2,000 1,000 0

PHY Layer Primary Throughput (Kbps) PHY Layer Secondary Throughput (Kbps)
38:09.0 37:09.0 36:45.0 36:57.0 37:45.0 37:57.0 37:33.0 37:21.0

HSPA Demo Phone - Cat 8 (1Rx)

20% 10% 0%

18.5%

36:09.0

35:45.0

36:33.0

35:57.0

36:21.0

NETWORK BENCHMARK TESTS

THE MOTHER OF ALL

38:21.0

17:29.0 17:37.0 17:45.0 17:53.0 18:01.0 18:09.0 18:17.0 18:25.0 18:33.0 18:41.0 18:49.0 18:57.0 19:05.0 19:13.0 19:21.0 19:29.0 19:37.0 19:45.0 19:53.0 20:01.0 20:09.0 20:17.0 20:25.0 20:33.0 20:41.0 20:49.0 20:57.0 21:05.0 21:13.0 21:20.0 21:27.0 21:35.0

InItIal feedback receIved In the fIrst few hours after releasIng volume 1 of our three-part serIes Included the followIng comments:
This is a real differentiatorwell worth it. Marketing Director, infrastructure supplier Very detailed and above and beyond my expectations. CTO, tier two operator Great report as always. CTO, leading mobile operator High impact stuffNobody even comes close to delivering this kind of data. Nice, nice move. Marketing Director, infrastructure supplier YOU kick $@ttThis is great stuff. Managing Director, financial institution Great report again. Senior Technical Fellow, infrastructure supplier Great stuffwell thought out and most of all fair. Network Services, leading mobile operator

This document contains a highly-redacted executive summary, a complete table of contents, and our test methodology (Chapter 8 from the main report) for a signals ahead research product that we published on October 18th. All three reports were done with the support of Accuver, who provided us with access to its complete set of network benchmarking tools and post-processing software. Volume 1 of our special three-part series was published in late September and Volume 3 will be published in November. Additionally, this report preview provides a summary of past topics that we have covered in Signals Ahead and a list of likely topics that we plan to tackle in the coming months. The 66-page report contains 53 figures and tables, with many of the figures and tables consisting of multiple parts. This report can be purchased separately for $1,495 or it is included with any paid corporate subscription to signals ahead.

In Volume 1 (Network and Technology Performance) of our special the leading next-generation mobile broadband wireless technologies performed, as exemplified by deployments in North America. To a large

three-part series of reports, we provided a thorough examination of how

network as possible. If for no other reason, this activity is prohibitively

at all times of the day and night trying to suck as much capacity from the

since no one in their right mind drives around large metropolitan areas

degree, the results contained in that report were somewhat contrived

expensive in many markets due to existing rate plans and data usage caps.
what really matters to consumers and [hopefully] operators is the impact of the underlying network/ technology on the typical user experience while consuming mobile data. Instead, what really matters to consumers and [hopefully] operators is the impact of the underlying network/technology on the typical user experience while consuming mobile data. In the infancy of a new commercial network launch, technology neophytes and pundits flock to their speed testing website de jour (e.g., www.speedtest.net) and they bask in the glory of never-before-seen data rates. However, this activity and the use of more sophisticated network benchmark studies quickly give way to more typical user behavior. Long-term sustained FTP downlink transfers get replaced by interactions with video delivery services (Netflix, YouTube, etc), content providers (e.g., Apples iTunes) or service enablers (e.g., Googles email). Uplink FTP transfers give way to more challenging scenarios involving the transmitting of video content from the mobile device to the network (e.g., Skype video). Ongoing 32 byte ping tests to a test server get replaced by basic web browsing of popular mobile web sites where the throughput potential of the network and in-network latency take a backseat to the end-to-end latency of the entire connection and how quickly the mobile device and the host site can communicate with each other and pass data back and forth. In Volume 2 (Quantifying the User Experience), we expose readers to a large number of neverbefore-seen test results that we collected during the data collection portion of this exercise how we spent our summer vacation. Although our conclusions from the first report remain intact, there are several nuances to those observations which need to be considered. To summarize, throughput still matters, but only to a certain point since most applications require only a fraction of the throughput offered by todays mobile broadband networks while in many cases the chokepoint in the broadband connection is the wired Internet/host server and not the operators next-generation mobile broadband network. Latency, including both the time to connect to the network and start receiving data, as well as the subsequent interactions between the device and the network/host server, can never be too low. In that regard, too much attention is paid to the my pipe is bigger than your pipe mantra when the real focus should be on delivering a more compelling user experience, regardless of how it is achieved. Our ability to collect and analyze the user experience data would not have been possible without the support of Accuver, who allowed us to use its suite of network drive test tools, including its recently released XCAL-MO network benchmarking tool and XCAL-M drive test solution, as well as its XCAP post-processing software to analyze the results. We have used the Accuver tools

throughput still matters, but only to a certain point while latency can never be too low.

we were able to capitalize on many of the unique features associated with the suite of accuver tools when we conducted our user experience tests.

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

operators should re-evaluate how they promote and market their services, starting by replacing the focus on speed with the focus on fast.

several times in the past for various Signals Ahead reports and we have grown quite fond of their capabilities and their ease of use. In particular, we were able to capitalize on many of the unique features associated with the suite of test tools when we did our user experience tests. We look forward to leveraging other innovative capabilities associated with their suite of tools that pertain to quantifying the user experience in the near future. For the first time, we are now able to offer their tools with our services for commissioned-based projects on behalf of operators, government regulators, vendors, trade associations or other interested parties on a global basis. We look forward to discussing such opportunities with anyone that is interested. As we discuss in Chapter 3 of this report, operators should re-evaluate how they promote and market their services. To some extent it is a bit too late since theyve sold consumers on the concept that speed is all that matters. However, it is possible to replace speed with fast, and then equate fast with the time required to connect to the network, as well as the time required to access and/ or download content. As an analogy, operators dont advertise if they use half-rate AMR, full-rate AMR or wideband AMR, but they do advertise voice quality, call completion rates, dropped call rates, etc. In a similar fashion, handset manufacturers dont promote lower current drain; instead they advertise longer battery lives. Those metrics truly define the user experience. In this context, an improved user experience can be achieved with a better handset design and an improved operating system GUI that make it easier to enter commands or navigate the web, just as it can be achieved by improvements to the network. Further, operators do not necessarily need to rush out and deploy the latest technology on the market to make their networks faster. Instead, these operators could take an existing technology and simply make it better and more efficient. This goal could be achieved through a combination of network optimization initiatives, by taking full advantage of the networks capabilities, including the widespread use of handsets that support the same set of robust features (e.g., Enhanced_FACH, CPC, CELL_PCH/URA_PCH, Cat 14, etc), and by moving popular content closer to the edge of the network. Chapter 2 of this report contains the introduction and Chapter 3 contains the key conclusions and observations from our tests. It is by far the most important chapter in this report. The remaining chapters and the appendix document the basis for our conclusions. Chapter 4 presents results from numerous tests involving HTTP sessions where we analyze how long it takes to load a popular web page for a given combination of throughput and latency, not to mention other considerations. Chapter 5 explains why throughput doesnt [always] matter and Chapter 6 explains why latency [almost] always matters. Chapter 7 focuses on bandwidth-intensive applications, such as Gmail, iTunes, Netflix, YouTube and to a lesser extent Skype Video. Chapter 8 provides our test methodology and Chapter 9 provides some closing remarks. The Appendix provides results from a few test scenarios that didnt find their way into the main body of the report. Our three-part series of reports is included with a subscription to Signals Ahead or it can be purchased on an individual basis the former option is far more economical since it includes at least 14 additional Signals Ahead reports. A summary of Volume 1 and Volume 3 follows in the subsequent paragraphs.

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

volume 1 (network and technology performance)

Next-generation network technologies are not created equal. On paper, there exist meaningful performance differences, some of which are due to channel bandwidth considerations, but also to the underlying technology itself. Further, operator deployment philosophies and the maturity of the solutions can have an over-arching impact on the results. As operators around the globe struggle to make crucial strategic decisions regarding their network technology evolution, it is imperative that they fully understand and appreciate the potential of these technologies as well as their limitations. While this report is intended to address the needs of operators worldwide by focusing on the performance of the technologies, as a secondary feature it also provides valuable insight into the performance of each major network deployment in the United States. Everyone claims that they have the best network, but only one operator can be right. Specific topics addressed in Volume 1 include, but are not limited to, the following:
Application Mean, Geo

volume 1 is critical for operators around the globe who are currently making strategic decisions regarding their network technology evolution.

and/or Physical Layer Throughput

median and CDF plots comparisons, including versus LTE (2x10MHz) with 2x2 MIMO

plots of throughput for all test scenarios using Google Earth

Technology

DC-HSDPA LTE

(2x20MHz) versus LTE (2x10MHz) WiMAX versus HSPA+, LTE and DC-HSDPA versus HSPA+ versus LTE, HSPA+, etc normalized for channel bandwidth and duplex scheme

Mobile

DC-HSDPA EV-DO

Single User Spectral Efficiency Results LTE with MIMO really outperform narrow bandwidth solutions network performance with multiple devices and HSPA+ devices in the same 10MHz channel allocations

Throughput Does LTE

DC-HSDPA

Side-by-Side operator network coverage maps for drive routes used in each market throughput throughput Latency based on time of day and network loading

Downlink Uplink

Network Variance

LTE network deployment philosophies (LTE cell site density relative to the legacy network) and

their implications for coverage and capacity HSPA+ versus AT&T LTE

AT&T Verizon AT&T Mobile

Wireless EV-DO versus Verizon Wireless LTE LTE versus Verizon Wireless LTE WiMAX (2500MHz) versus LTE (700MHz)

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

volume 2 (Quantifying the user experience)

Although mobile operators, industry pundits and most well-informed consumers understand the notion that a higher megabit-per-second throughput is preferable, the typical consumer is generally clueless when it comes to understanding what these obscure marketing messages really mean for the mobile Internet experience. Most operators recognize that they need to move away from the my pipe is bigger than your pipe marketing mentality, but it is easier said than done. Further, it is readily apparent that the capabilities of these next-generation networks frequently exceed the requirements of the application and/or the capabilities of the Internet itself. Very few applications and/or web site servers support high double-digit megabit-per-second throughput. Instead, it may actually be the combination of relatively high throughput and low network latency offset by transport latency that really defines the user experience. But to what degree do these relationships provide the most benefit to the user? Mobile video, be it YouTube or Netflix, is driving mobile data growth and the capabilities of nextgeneration networks will only serve as an impetus to even higher data usage. The perceived quality of the video playback also matters, not only for consumers, but also for mobile operators, content owners, and video hosting services. As higher resolution video formats with higher encoding rates become more mainstream, this issue becomes even more important, especially if the performance of next-generation networks fails to keep up with the requirements of the video content. This report is critical for operators trying to understand how to market their broadband wireless service offering as well as how they should prioritize their network optimization activities in order to achieve the best possible user experience for their subscribers. In addition to mobile operators, this report provides invaluable insight to application developers and content providers who require a greater appreciation for how network performance characteristics impact the user experience. Specific topics addressed in Volume 2 may include, but are not limited to, the following:
Quantifying Popular Results

volume 2 is critical for operators trying to understand how to market their broadband wireless service offering as well as how they should prioritize their network optimization activities.

the user experience based on HTTP web page download times

websites, including Yahoo, CNN, iTunes, Amazon, YouTube, etc. down to the millisecond, based on device/chipset signaling messages comparisons

Network/technology Determining Determining 3

Comparisons based on network loading same location over a 12-15 hour period of time if perceived differences in network/technology performance have more to do with network loading than the actual capabilities of the network/technology itself how the combination of throughput and latency impact the HTTP web page download time results axis plot, showing maximum achievable throughput and network latency versus webpage load time and required throughput matters most latency or throughput the crossover point when higher throughput become irrelevant the user experience based on downloading Google email attachments the user experience based on downloading video and audio content from iTunes the crossover point when higher throughput become irrelevant DC-HSDPA really offer a quantifiable benefit over HSPA+

Which Does

Determining Quantifying Quantifying Determining Netflix

video streaming requirements

Determining the chokepoints in the network (from end user to the original source of the content),

how they vary as a function of loading, and their impact on the user experience

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

volume 3 (detailed performance analysis)

In our third and final installment we delve much deeper into the KPIs that we captured with the Accuver suite of drive test tools. As we have witnessed in the past there are discernible differences in how each vendor implements a technology. Frankly, some vendors do a much better job than their peers. Given that we collected network performance data in a number of key markets and that we knowingly included virtually every single vendor + technology combination that exists in North America, this report provides invaluable competitive intelligence for vendors, subsystem suppliers and mobile operators. Further, by peeling back the layers of the proverbial technology onion it becomes possible to gain a greater appreciation for how each technology delivers its results. Specific topics addressed in Volume 3 may include, but are not limited to, the following:
Modulation MIMO

volume 3 provides invaluable competitive intelligence while also allowing readers to obtain a greater appreciation for how each technology delivers its results.

Utilization (QPSK, 16QAM, and 64QAM) by primary and/or secondary carriers as appropriate RI 1 and RI 2 how MIMO performs at 700MHz Codes (average, % > 10, distribution) by primary and secondary carriers

CQI (average and median) by primary and secondary carriers Scheduling Success Rate by primary and secondary carriers

HS-PDSCH HS-SCCH Average

PHY Layer Served Rate by primary and secondary carriers PHY Layer Scheduled/Served Rate DC-HSDPA only

Maximum UL

Transmit Power (average and median)

We will also leverage the capabilities of the XCAP-M tool to analyze these KPIs by several different means, potentially including, but not limited to the following:
MAC-HS MAC-HS Reported

Throughput versus RSCP scatter plot Throughput versus Reported CQI Values scatter plot Throughput versus Cell ID (real time) Throughput versus # of Assigned HS-PDSCH Codes (real time) Throughput (primary, secondary, and combined)

CQI Values versus 64QAM Availability scatter plot

MAC-HS MAC-HS MAC-HS HSPA+

MAC-HS Throughput versus DC-HSDPA MAC-HS Throughput (primary, secondary and combined) versus CINR (scatter plot and real-time) versus Cell ID (e.g., handover performance)

CINR versus RSSI (scatter plot and real-time)

Throughput Throughput

CINR versus RSSI (scatter plot and real-time) CINR versus Modulation Scheme and/or MCS Transmit Power versus Cell ID Throughput versus Transmit Power Scheme (antenna 1 and antenna 2) October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

UL UL

Modulation

potential topics for the coming year include:

The The

Mother of all Network Drive Tests (LTE, DC-HSDPA, HSPA+, Rev A and Mobile WiMAX) challenges of delivering video in a mobile network modules/netbooks network performance (throughput and latency) impacts the user experience network performance benchmark results

How

Embedded TD-LTE

CoMP and LTE Advanced Green financial implications and challenges Smartphone signaling implications and LTE chipset performance benchmark test results

Going

LTE The

impact of Type 3i receivers on UE performance (includes chipset benchmark tests of leading solutions) happened to IMS? Americas (MIMO) network performance benchmark results

Whatever LTE 4G

World and GSMA MAC impact of latency Safety Options with 700MHz Rev B network performance benchmark results

HSPA+ The

Public

EV-DO LTE

chipset landscape

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

HTTP R HSDPA T SNR MI RSRQ R MAC-HS MOS LT WiMAX QPSK C HS-PDSC TCP dBm EV-DO R BLER 64 CINR A E-DPCH HSPA+ D MCS MC DL RSSI HS-SCC Mbps m A/B Mo 16QAM PDSCH HTTP R HSDPA T SNR MI
9,000 6,000 3,000
35:09.0

-60

35:21.0

RTT TCP dBm dB Mbps ms MOS LTE HSPA+ TD-LTE EV-DO Rev A/B Mobile WiMAX MCS IMO RI BLER 64QAM 16QAM QPSK CQI L1 DL RSCP CINR ACK NACK PDSCH HS-PDSCH HS-S Coming Soon! S E-DPCH FTP UDP HTTP RTT TCP dBm dB Mbp TE HSPA+ DC-HSDPA TD-LTE EV-DO Rev A/B M X MCS MCS SNR MIMO RI BLER 64QAM 16Q CQI L1 DL RSSI RSRQ RSCP CINR ACK NACK PD SCH HS-SCCH MAC-HS E-DPCH FTP UDP HTTP m dB Mbps ms MOS LTE HSPA+ DC-HSDPA TD Rev A/B Mobile WiMAX MCS MCS SNR MIM 4QAM 16QAM QPSK CQI L1 DL RSSI RSRQ ACK NACK PDSCH HS-PDSCH HS-SCCH MA H FTP UDP HTTP RTT TCP dBm dB Mbps ms MO DC-HSDPA TD-LTE EV-DO Rev A/B Mobile Wi CS SNR MIMO RI BLER 64QAM 16QAM QPSK C I RSRQ RSCP CINR ACK NACK PDSCH HS-PD CH MAC-HS E-DPCH FTP UDP HTTP RTT TCP dB ms MOS LTE HSPA+ DC-HSDPA TD-LTE EV-DO obile WiMAX MCS MCS SNR MIMO RIsubscription) 64Q BLER pre-order your report lIcense now (included as part of a Signals Ahead QPSK CQI L1 DL RSSI RSRQ RSCP CINR ACK N HS-PDSCH HS-SCCH MAC-HS E-DPCH FTP RTT TCP dBm dB Mbps ms MOS LTE HSPA+ TD-LTE EV-DO Rev A/B Mobile WiMAX MCS IMO RI BLER 64QAM 16QAM QPSK CQI L1 DL
5,000
70%

4,000 3,000

Total DC-HSDPA PHY Layer Throughput (Kbps)

60% 50%

52.9%

35.7%

40% 30%

2,000 1,000 0

PHY Layer Primary Throughput (Kbps) PHY Layer Secondary Throughput (Kbps)
36:45.0 36:57.0 37:09.0 37:33.0 37:45.0 37:57.0 37:21.0 38:09.0

HSPA Demo Phone - Cat 8 (1Rx)

20% 10% 0%

18.5%

35:45.0

35:33.0

35:57.0

36:09.0

36:33.0

36:21.0

-50

-40

-30

-20

lIcense costs

38:21.0

(pre-publishing and post-publishing prices)

contact InformatIon

Volume 1 avaIlable now! Network and Technology Performance ($1,995) Volume 2 avaIlable now! Quantifying the User Experience ($1,495) Volume 3 Detailed Performance Analysis ($1,295, $1,495) full report all 3 volumes ($3,300, $3,995) 9

You may call us at +1 (510) 273-2439 or email us at information@signalsresearch.com and we will contact you for your billing information or respond to any further inquiries that you may have. Subscription information for our Signals Ahead research product, which includes these reports, can be found on the last page of this report. You can also visit our website at www.signalsresearch.com or write us at

Signals Research Group, LLC 10 Ormindale Court Oakland, CA 94611 October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

17:29.0 17:37.0 17:45.0 17:53.0 18:01.0 18:09.0 18:17.0 18:25.0 18:33.0 18:41.0 18:49.0 18:57.0 19:05.0 19:13.0 19:21.0 19:29.0 19:37.0 19:45.0 19:53.0 20:01.0 20:09.0 20:17.0 20:25.0 20:33.0 20:41.0 20:49.0 20:57.0 21:05.0 21:13.0 21:20.0 21:27.0 21:35.0

In Case You Missed It

9/27/11 The Mother of all Network Benchmark Tests

In Volume 1 of a special three-part series of reports we leverage the capabilities of Accuvers complete suite of drive test tools to analyze the performance of all next-generation wireless technologies as exemplified by operator deployments in North America. In addition to looking at basic parameters, such as throughput and latency, the report looks at end-user spectral efficiency and the different network deployment philosophies of the leadng operators (e.g., their cell grid density). networks In this report we discuss the recent trends impacting the various mobile platforms that exist and what has transpired since our piece from three years ago on Web 2.0. We address the state of the mobile platforms that exist, provide our thoughts on the current and future prospects and look at the various trends that are driving the industry. provide the key takeaways from the LTE World Summit, held in Amsterdam. Spectrum fragmentation tops the list of key LTE topics, although a growing focus on the use of 1800MHz for those operators that have access to it is encouraging. VoLTE, or the lack thereof, is still on everyones minds, but in the interim CSFB isnt even working as promised. Finally, there was a lot of talk about Mobile WiMAX, but the emphasis seemed to be on how to best move away from the technology and adopt TD-LTE. collide In addition to covering the basics of heterogeneous networks (HetNet), a key LTE-Advanced (R10) feature, we present a compelling series of analytical studies which demonstrate the need for macro network offload, starting as early as 2015. We also get into the technical details of how HetNet works, including discussions on eICIC, ABS and the importance of interference cancellation in the handset. Finally, we look at what is being done with legacy 3G femtocells to limit interference-related problems that they introduce, both with the macro network and between each other. Soy Sauce In collaboration with Spirent Communications we provide results from the industrys only independent performance benchmark tests of HSPA+/HSPA chipsets. In the most recent benchmark study we tested 16 different device configurations, representing chipsets from 9 different suppliers, including new entrants, such as Samsung (HSPA+), Intel (HSPA+), MediaTek

and HiSilicon. We provide the results, based on a total of 42 HSPA+ test scenarios and 26 HSPA test scenarios.
3/15/2011 Looking beyond HSPA+: keeping up with the

7/6/11 Mobile Platforms the center of mobile

Joneses Based on interviews with 3GPP member companies and a thorough review of 3GPP submissions, we offer an in-depth look at the future of HSPA+ (Release 11 and beyond). Ultimately, we conclude that many of the features that are being incorporated into LTE will find their way into HSPA+, thus blurring the performance differences between the two technologies. Latency and the impact of new features on legacy devices are two areas of prime importance where HSPA+ could face challenges relative to LTE. the Pleasure, Part II In collaboration with Accuver, who provided us with its XCAL-W drive test tool and XCAP-W post-processing software, we provide results and analysis from an extensive drive test of Telstras DC-HSDPA network. We compare DC-HSDPA with HSPA+ performance in a number of side-by-side tests. the Pleasure, Part I In collaboration with Accuver, who provided us with its XCAL-W drive test tool and XCAP-W post-processing software, we provide results and analysis from an extensive drive test of Telstras DC-HSDPA network. We compare DC-HSDPA with HSPA+ performance in a number of side-by-side tests. tion with Sanjole we examine how some of todays commercial LTE eNodeBs allocate network resources when serving multiple devices. We determine that while LTE may deliver a compelling user experience, it is largely due to an empty network and the large channel bandwidths, and that further improvements are necessary if LTE is going to support multiple users in an efficient manner.

1/12/2011 DC-HSDPA: Double the Bandwidth, Double

6/8/2011 United we stand, fragmented we fail We

1/12/2011 DC-HSDPA: Double the Bandwidth, Double

5/16/2011 HetNet: When big cells and small cells

12/10/2010Can you schedule me now? In collabora-

12/3/2010 A Perspective from LTE Americas and the

4/26/2011 Chips and Salsa XIII: Now Seasoned with

GSMA Mobile Asia Congress We provide and discuss various data points which stem from our participation at the LTE Americas event in Dallas and the GSMA MAC event in Hong Kong. We provide an LTE market update, including TD-LTE, discuss the debate about a smart or dumb pipe strategy, and the impact of smartphones and social networking services, including the use of cloud computing, intelligent networks, network offloading and data caching.

Come join us!

4g world, Chicago, IL Oct. 24-27 lte americas, Dallas, TX Nov. 8-9 Invited speaker rcr wireless oc event, Orange County, CA Nov. 10 Invited Speaker consumer electronics show, Las Vegas, NV Jan. 10-13 mobile world congress, Barcelona, Spain Feb. 20-23

10

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

table of contents of volume 2

1.0 Executive Summary 3 2.0 Introduction 10 3.0 Key Conclusions and Observations 4.0 HTTP Web Browsing Analysis

12 20

4.1 downtown san francisco July 29th, 0430 - 1730 20 4.1.1 Throughput and Latency Measurements (Downtown San Francisco July 29th, 0430 1730) 21 4.1.2 HTTP Web Page Load Time Results (Downtown San Francisco July 29th, 0430 1730) 23 4.2 http web page load time results (srg headquarters deep within the oakland hills august 8, 1800-2200) 4.3 downtown oakland august 14th, 1330 - 1530

27 28 28 29 31 31

4.3.1 Throughput and Latency Measurements (Downtown Oakland August 14th, 1330-1530) 4.3.2 HTTP Web Page Load Time Results (Downtown Oakland August 14th, 1330-1530) 4.4 santa clara august 29th august 31st 4.4.1 Throughput and Latency Measurements (Santa Clara August 29th August 31st)

4.4.2 HTTP Web Page Load Time Results (Santa Clara August 29th August 31st) 32 4.5 ev-do rev a and lte 33 4.5.1 Throughput, Latency and HTTP Web Page Load Time Results EV-DO Rev A and LTE (Kansas City, June 17th, 1700-1900) 33 4.5.2 Throughput, Latency and HTTP Web Page Load Time Results EV-DO Rev A and LTE (SRG Headquarters, August 8th, 1800-2200) 34 5.0 Why Throughput Doesnt [Always] Matter 6.0 Why Latency [Almost] Always Matters 6.1 web page 101

36

40

40

6.2 tracing the real latency to various web sites by network 43 6.3 connection time latency

46 47 49 52 56 58 61 62

6.4 low latency and its Impact on the user experience 7.1 itunes and gmail

7.0 The User Experience and High-Bandwidth Applications 49

7.2 youtube and netflix

7.3 time of day considerations 8.0 Test Methodology 10.0 Appendix 1

7.4 skype video 57

9.0 Concluding Remarks

1 1

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

Index of figures
figure 1. Downlink Application Layer Median Throughput San Francisco (0430 1730) figure 3. Network Latency San Francisco (0430 1730)

21 21

figure 2. Downlink Application Layer Maximum Throughput San Francisco (0430 1730)

22

figure 4. Amazon Web Page Load Times by throughput + latency combinations 24 figure 5. Yahoo Web Page Load Times by throughput + latency combinations 24 figure 6. iTunes Web Page Load Times by throughput + latency combinations 26 figure 7. iTunes Web Page Load Times by operator
26 28 29 30

figure 8. CNN Web Page Load Times by throughput + latency combinations (SRG HQ) 27 figure 9. Throughput and Latency Results by Operator (Oakland) figure 10. CNN Web Page Load Times by throughput + latency combinations (Oakland)

figure 11. Amazon Web Page Load Times by throughput + latency combinations (Oakland) 30 figure 12. Yahoo Web Page Load Times by throughput + latency combinations (Oakland) figure 13. Throughput and Latency Results by operator (Santa Clara) 31 figure 14. Yahoo Web Page Load Times by throughput + latency combinations (Santa Clara) 32 figure 15. Amazon Web Page Load Times by throughput + latency combinations (Santa Clara) figure 16. Throughput and Latency Results EV-DO Rev A and LTE (Kansas City)
32

33

figure 17. Web Page Load Times by throughput + latency combinations (Kansas City) 34 figure 18. Throughput and Latency Results EV-DO Rev A and LTE (SRG Headquarters)
35 35

figure 19. Wikipedia Web Page Load Times by throughput + latency combinations (SRG Headquarters) figure 21. DC-HSDPA MAC Layer Throughput HTTP Web Browsing (San Francisco, July 29th, 0500) figure 23. HTTP Application Layer Throughput by technology (Oakland, August 14th) figure 24. LTE Application Layer Requirements an entire user experience test script figure 26. Connection Latency by technology

figure 20. iTunes Web Page Load Times by throughput + latency combinations (SRG Headquarters) 35
36

figure 22. DC-HSDPA MAC Layer Throughput HTTP Web Browsing (San Francisco, July 29th, 1700) 37
38 39

figure 25. The Theoretical Impact of Varying Throughput and Latency on Web Page Load Times 41
46 47 47

figure 27. The Impact of Ultra Low Latency and Low Throughput YouTube figure 28. The Impact of Ultra Low Latency and Low Throughput PornHub figure 29. The Impact of Ultra Low Latency and Low Throughput MSN

48 50

figure 30. Gmail and iTunes Throughput Requirements Clearwires 2x20MHz LTE Network 49 figure 31. Application Layer Throughput while using iTunes with and without a background FTP session figure 33. Gmail and iTunes Throughput Requirements AT&Ts LTE Network (Houston, September 7th) figure 35. Netflix Throughput Requirements T-Mobile DC-HSDPA Network (Oakland, August 14th) figure 32. Gmail and iTunes Throughput Requirements T-Mobiles HSPA+ Network (San Francisco, July 29th) 51

51

figure 34. YouTube and Netflix Throughput Requirements Verizon Wireless LTE Network (Oakland, August 14th)

52

53 55

figure 36. YouTube and Netflix Throughput Requirements AT&T LTE Network (Houston, September 7th) 53 figure 37. Netflix Throughput Requirements T-Mobile DC-HSDPA Network (Santa Clara, August 29th, 2100) figure 38. iTunes Throughput Requirements as a Function of Time of Day Verizon Wireless LTE Network (San Francisco, July 29th) 56

1 2

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

figure 39. Skype Downlink Throughput Requirements Device 1 and Device 2 on the Verizon Wireless LTE Network (Oakland, August 18th) 57 figure 40. Skype Uplink Throughput Requirements Device 1 and Device 2 on the Verizon Wireless LTE Network (Oakland, August 18th) 57 figure 41. XCAL-M Drive Test Tool in Action DL performance 58 figure 42. XCAL-M Drive Test Tool in Action UL performance
59 62 63 64 64

figure 43. iTunes Web Page Load Times by throughput + latency combinations (SRG Headquarters) 62 figure 44. iTunes Web Page Load Times by operator (SRG Headquarters) figure 45. Amazon Web Page Load Times by operator (SRG Headquarters)

figure 46. Yahoo Web Page Load Times by throughput + latency combinations (SRG Headquarters) 63 figure 47. LTE Application Layer Throughput HTTP Web Browsing (San Francisco, July 29th, 0601) figure 48. LTE Application Layer Throughput HTTP Web Browsing (San Francisco, July 29th, 1635)

figure 49. Gmail Throughput Requirements as a Function of Time of Day Verizon Wireless LTE Network (San Francisco, July 29th) 65

Index of tables
table 1. URL Analysis Size and HTTP Requests
42 43

table 2. HTTP Activity Count by URL 42 table 3. Real World Latency Calculations to Target URLs by network/technology table 4. Trace Route results by operator, technology and URL
44

13

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

8.0 test methodology

For the drive tests that we have been conducting this summer we primarily used the Accuver XCAL-MO network benchmarking tool along with the Accuver XCAL drive test tool to collect the underlying performance indicators and to conduct the user experience tests. For purposes of our tests, we limited the XCAL-MO to only four dongles one dongle for each network/technology that we wanted to test. In theory we could have installed multiple dongles for each network/ technology. We used the Accuver XCAP post-processing tool to analyze the data and to help us create the figures which appear in this summary report. Thanks to a combination of the powerful tool and countless hours spent on the road, we are convinced that we have witnessed network performance both good and bad that would have otherwise not been observed. Figure 41 and Figure 42 illustrate a typical user display that we used when collecting the data. We have included two figures since they also help prove that we observed downlink data rates greater than 61Mbps (Figure 41) and uplink data rates in excess of 23Mbps (Figure 42).

we used the accuver Xcal-mo and Xcal-m tools to collect the underlying performance indicators and the accuver Xcap-m post-processing tool to do the analysis of the data that we collected.

figure 41. Xcal-m drive test tool in action dl performance

Source: Accuver XCAL and SRG

14

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

figure 42. Xcal-m drive test tool in action ul performance

Source: Accuver XCAL and SRG

Each operator provided us with at least two dongles, although in the case of operators, such as Clearwire, with multiple network/technology deployments (e.g., 2x20MHz LTE and 1x10MHz Mobile WiMAX), we received multiple dongles. Unlike the drive test results which we presented in Volume 1, the results shown in this report were collected from a stationary position. We took this approach in order to reduce the variability in performance that can occur when moving throughout the network. Eliminating, or at least reducing, this variability was important since the objective of the user experience tests was to determine various user experience metrics for a known set of inputs primarily throughput and latency. Testing in each market took place from as early as 4AM local time until the early evening hours. We also did a lot of user experience testing during the dead of the night when we suspect the networks were wide open. Since we were using test equipment we had the ability to determine whether or not network loading was impacting the results. Suffice it to say that in the early morning hours network loading was not a concern for any of the networks. Later in the day, network loading impacted the performance of certain networks/technologies while it was not even a consideration with other networks/technologies. We take this phenomenon into consideration when doing our analysis. Each user experience test in involving HTTP web browsing consisted of the following series of tests:
A A

60 second multi-session FTP test to determine the maximum throughput potential of the network at the time. 60 second latency test in which we pinged a local server to determine the median latency. a series of popular websites and downloading their contents with an Internet Explorer browser. In order to ensure a degree of consistency, we downloaded each web page up to 50 times and then used the median download time. The download time calculations leveraged very detailed information captured by the Accuver drive test tools. October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

Accessing

15

60 second multi-session FTP test to determine the maximum throughput capabilities of the network at the time. The results from this test were combined with the results from the first FTP test to determine the median throughput. It was our assumption that this potential throughput was relatively constant throughout the entire test, although this assumption may not have always been correct.

like all Signals Ahead reports, we received no sponsorship or funding from the participating companies, in order to maintain our independence.

For the high-bandwidth application tests, such as iTunes, Netflix and YouTube, we conducted a multi-session FTP test to determine the maximum throughput potential of the network. This test was followed by interacting with these applications (watching a video, downloading an email attachment) and using the Accuver tools to record the underlying interactions between the mobile device and the network. Like all Signals Ahead reports, we received no sponsorship or funding from the participating companies, in order to maintain our independence. As such, we foot the bill for all of our travel expenses not to mention an inordinate amount of time and effort collecting the data and writing these series of reports. We also could not have done this report without the support of Accuver who provided us with its suite of drive test tools and post-processing software. SRG takes full responsibility for the analysis and conclusions that are documented in this report and in our forthcoming report that analyzes and compares technology- and vendor-specific performance characteristics.

michael thelander
Michael Thelander is the CEO and Founder of Signals Research Group. In his current endeavor he leads a team of industry experts providing technical and operator economics analysis for clients on a global basis. Mr. Thelander is also responsible for the consultancys Signals Ahead research product, including its widely acclaimed Chips and Salsa series of reports that focus on the wireless IC industry. Previously, Mr. Thelander was an analyst with Deutsche Bank Equity Research. Prior to joining Deutsche Bank, Mr. Thelander was a consultant with KPMG (now known as BearingPoint) and a communications officer with the United States Army. Mr. Thelander has also published numerous articles for leading trade publications and engineering journals throughout his career. He has been an invited speaker at industry conferences around the world and he is frequently quoted by major news sources and industry newsletters, including The Economist, The Wall Street Journal, Investors Business Daily, Reuters, Bloomberg News, and The China Daily. Mr. Thelander earned a Masters of Science in Solid State Physics from North Carolina State University and a Masters of Business Administration from the University of Chicago, Graduate School of Business.

16

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW

Signals Ahead Subscription

The Signals Ahead newsletter is available on a subscription basis. We offer four distinct packages that have been tailored to address the needs of our corporate users. The Group License includes up to five users from the same company. The Global License is the most attractive package for companies that have several readers since it is offered to an unlimited number of employees from the same organization. Finally, the Platinum package includes the Global License, plus up to five hours of analyst time. Other packages are available. Corporate Rates (18 issues) Group License ($3,995) Global License ($7,995) Payment Terms American Express Check Purchase Order Name: Affiliation: Mailing Address: Platinum ($9,495)

Visa MasterCard Check Number PO Number Title: Phone: ( )

Credit Card #

Exp Date

/ /

Mailing Address Signals Research Group, LLC ATTN: Sales 10 Ormindale Court Oakland, CA 94611 Our fax number is (510) 338-1284. Alternatively, you may contact us at (510) 273-2439 or at information@signalsresearch.com and we will contact you for your billing information. We will not process your payment until after the trial subscription period is completed. Terms and Conditions: Any copying, redistributing, or republishing of this material, including unauthorized sharing of user accounts, is strictly prohibited without the written consent of SRG.

please note disclaimer: The views expressed in this newsletter reflect those of Signals Research Group, LLC and are based on our understanding of past and current events shaping the wireless industry. This report is provided for informational purposes only and on the condition that it will not form a basis for any investment decision. The information has been obtained from sources believed to be reliable, but Signals Research Group, LLC makes no representation as to the accuracy or completeness of such information. Opinions, estimates, projections or forecasts in this report constitute the current judgment of the author(s) as of the date of this report. Signals Research Group, LLC has no obligation to update, modify or amend this report or to otherwise notify a reader thereof in the event that any matter stated herein, or any opinion, projection, forecast or estimate set forth herein, changes or subsequently becomes inaccurate. If you feel our opinions, analysis or interpretations of events are inaccurate, please fell free to contact Signals Research Group, LLC. We are always seeking a more accurate understanding of the topics that influence the wireless industry. Reference in the newsletter to a company that is publicly traded is not a recommendation to buy or sell the shares of such company. Signals Research Group, LLC and/or its affiliates/investors may hold securities positions in the companies discussed in this report and may frequently trade in such positions. Such investment activity may be inconsistent with the analysis provided in this report. Signals Research Group, LLC seeks to do business and may currently be doing business with companies discussed in this report. Readers should be aware that Signals Research Group, LLC might have a conflict of interest that could affect the objectivity of this report. Additional information and disclosures can be found at our website at www.signalsresearch.com. This report may not be reproduced, copied, distributed or published without the prior written authorization of Signals Research Group, LLC (copyright 2011, all rights reserved by Signals Research Group, LLC).

17

October 19, 2011 | Signals Ahead, Vol. 7, Number 13 PREVIEW