Documente Academic
Documente Profesional
Documente Cultură
w h i t e c r o s s
w h i t e
p a p e r
maintain customers that fit best with the business model the
table of contents
2 3 4
4 5 5 5 6
7
7 7 7
8
8 9 9
NARUS IBI platform data collection and session reconstruction WhiteCross servers (WXDES) data analysis and transformation WhiteCross and NARUS solutions information presentation
12 13
WhiteCross
executive summary
As fewer new customers sign up for IP services, Internet Protocol Service Providers (IPSPs) will have to shift focus away from simply capturing a fair share of new subscribers to managing and capitalizing on existing customers by offering new billing plans and differentiated services, reducing and managing subscriber churn, and setting and enforcing acceptable usage policies. Little, however, has been spent implementing the Internet Business Infrastructure (IBI) needed to support this shift. Fundamental to all IBI solutions is an understanding of subscriber usage of IP services through Usage Pattern Analysis (UPA). UPA is the analysis of subscriber usage information captured at the application layer, combined with offline data such as user demographics, billing data, and customer service interaction to create service and subscriber usage profiles. The NARUS and WhiteCross solution is based on the By understanding service and subscriber usage profiles, IPSPs can make more informed and effective decisions regarding major IBI concerns such as billing, churn management, new service offerings, and acceptable use policies and enforcement. The elements of an effective UPA solution include data collection, data analysis and transformation, and information presentation and delivery. The data collection platform needs to capture detailed information that completely characterizes the applications and services used and the quality of the users experience. The data analysis platform must correlate this information with offline data sources such as user demographics, billing data, and customer support history and proNARUS IBI Platform and the WhiteCross Data Exploration Server (WX/DES). The NARUS IBI Platform captures application-level subscriber usage information directly off highspeed networks, in real-time. The WX/DES provides a scalable platform with the power to perform rapid analysis of large volumes of usage data at the speed of thought. The resulting Usage Pattern Analysis (UPA) solutions are offered through a service bureau. The solutions and services include an out-of-the-box solution, project-based services, and ongoing, continuous custom reporting and analytic services. The result is a unique set of tools that enable IP service providers to manage their businesses and keep up with the fast-moving pace of the IP services market. NARUS and WhiteCross have partnered to deliver unique IBI solutions by leveraging core capabilities from both companies. The result of the partnership is a suite of business intelligence applications and services through which IPSPs can understand their customers usage patterns, and use that knowledge to prepare, implement and monitor the success of new services and billing plans, manage customer churn, set and enforce usage policies, and increase overall profitability. vide a data analysis capability that is flexible, scalable, and powerful enough to handle large volumes of data quickly (for train of thought analyses). Finally, the UPA solution must present and deliver the results as actionable information; that is, the solution must present answers, not just more data for the user to analyze.
WhiteCross
introduction
Until recently, IPSPs have focused almost exclusively on building out their physical infrastructure. As more and more individuals and businesses flocked to the Internet, IPSPs focused on adding capacity, increasing bandwidth, and rushing into new geographic areas. Services were undifferentiated, and the industry took a one size fits all approach to pricing and bundling of services. The environment in which IPSPs operate, however, is changing dramatically: s As fewer new customers sign up for IP services, and as the market for IP service providers becomes saturated, IPSPs are forced to shift their focus away from simply capturing a fair share of new customers. Instead, they have to actually steal share away from competitors. s As new players enter the market with new business models, no IPSP can afford to offer the same, undifferentiated services. Instead, they must possess a dynamic understanding of what services subscribers want and how much they are willing to pay for those services. s As more and more mission-critical systems are deployed across IP networks, IPSPs cannot afford to offer one grade of best effort service. Instead, IPSPs will have to offer quality of service (QoS) and delivery options to meet the different needs of different subscribers. s IPSPs business problems can no longer be solved by simply throwing additional bandwidth on the network. Instead, IPSPs must be able to identify new business models, new markets, and new service bundles to offer subscribers. In other words, the focus of the IPSP market is shifting away from the physical infrastructure towards the Internet Business Infrastructure (IBI). Reap the Rewards
TURE VALUE
Profitability Analysis
D E SI
OWLEDGE KN
CUSTOMER USAGE DATA
Service Definition
G N A N D IMPLE M
Billing
C AP
Provisioning
Churn Management
IBI Solutions
IBI is the set of capabilities that enable IP service providers to manage customers, services, and business models. IBI tools allow IPSPs to define and deploy new services, offer flexible and differentiated billing options, provide proactive customer care, detect and defeat fraud, and manage customer churn. Figure 1 illustrates how IBI solutions allow ISPs to increase profitability. Fundamental to all IBI tools is an understanding of subscriber usage of IP services through Usage Pattern Analysis (UPA). UPA is the analysis of subscriber usage information captured at the application layer, combined with offline data such as user demographics, billing data, and customer service interaction. This white paper discusses the importance of UPA to IPSPs who want to differentiate their services in the highly competitive market for providing IP services to consumers and businesses. It describes why UPA is critical to understanding customers needs and their satisfaction with current IP service offerings. It also describes why business models based on Usage Pattern Analysis are essential for driving profitable growth in the areas of customer acquisition and retention. Finally, it describes how the WhiteCross-NARUS UPA solutions deliver an especially efficient and effective IBI solution.
OWLEDGE KN
E NT
Fraud Detection
Customer Care
M ANAGE
WhiteCross
BILLING
As stated in the introduction, many IPSPs have employed the same business model to date: flat rate, one-service-fitsall model. So, while segmentation exists among access providers in the form of wholesale vs. retail, dial-up vs. broadband, consumer vs. business, the same model is generally used for all segments: flat-rate, best-effort access to the entire (and identical) set of IP services. Today, IPSPs offering differentiated services are the exception, not the rule. As the market becomes increasingly saturated and correspondingly more competitive, IPSPs are looking to use new and creative billing plans as a competitive advantage. Residential customers are starting to ask IPSPs why they should pay $19.95 or more per month when all they use is email and the web. Business clients want to know if the IPSP can offer higher quality of service for certain applications like video conferencing over IP. A major barrier to implementation of usage-based billing is the fear of the unknown. Specifically, without an under-
for IPSPs that are analyzing current flat-rate models in anticipa66 60 54 48 43 Traffic (Mbytes)
Mbytes
Mbytes
0 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 6.0 6.5 7.0 7.5 8.0 8.5 9.0
Figure 2
9.5
WhiteCross
tion of moving to a usage-based model. The report is broken out into two charts. The first chart shows the top 5% of users. The second chart shows the bottom 95%. In this example, the report shows that the top 4% of web users generated at least 10 megabytes of traffic per user. It also shows that users in the top 5% range generate eight times as much web traffic (per user) as users in the bottom 50%. If the IPSP was using a flat rate billing model, this report could be used to identify this top 5% as the "costliest" segment of subscribers. This report also provides critical input for creating new, usage-based billing plans by showing IPSPs how service usage is distributed across the active subscriber base. That is, this report identifies the extent to which different segments of active subscribers are using the service.
CHURN MANAGEMENT
A second challenge is protecting high-value customers from competitors i.e. customer retention. Customers rarely register their dissatisfaction with a service directly, preferring to express it through a more indirect and costly mechanism, i.e. by switching to a competitor. Churn is one of the costliest problems faced by IPSPs today. Many IPSPs give it scant consideration, preferring instead to replace lost customers through enticements (lower flat-rate prices) or mergers and acquisitions. As the market moves towards saturation, however, these methods will prove ever more costly experience shows that once high-value customers are lost, they are expensive to replace or entice back. Highest profits continue to be earned from existing customers who prefer to remain with their current service provider subject to careful management of their inertial hurdle the level at which their dissatisfaction overrides their resistance to change. Fortunately, customer dissatisfaction shows up early in usage. Careful UPA will highlight subtle changes in customer behavior and potential dissatisfaction with services reduced session frequency, shorter sessions, and other changes in the usage profile all indicate potential dissatisfaction. The answer to the churn problem is in the detail; monitoring detailed usage activity provides IP service providers the vehicle for designing truly proactive customer responses.
streaming news video usage is trending upwards across the user base. The IPSP may want to partner with CNN or other news providers to offer that coverage directly on the IPSPs homepage, thus keeping subscribers on the site longer and, ultimately, increasing advertising revenue. Additionally, the IPSP may see that most subscribers logged on to the network from noon to one oclock PM are browsing e-commerce sites. The IPSP may want to dynamically change its homepage content during that hour to focus primarily on its e-commerce offerings before shifting back to streaming video news clips for the 6:00 to 9:00 PM time slot. Used in this fashion, UPA enables an IPSP to optimize the attractiveness of its services across the boundaries of time, geography, and customer segments.
WhiteCross
resources and can accelerate churn. With an effective UPA solution, potential spammers are more easily identifiable through statistical analysis of current and historic email usage of the potential violators. A second example is home server usage. Particularly on broadband residential IP networks, subscribers that are only paying for Internet access may be hosting servers, a direct violation of the service agreement. Through proper monitoring using UPA, an IPSP has a much greater chance of identifying these policy violators and either charging them for the hosting service or terminating their service agreement. Figure 3 is another example of a NARUS Intelligence report that can be used to help detect fraud. This report identifies the extent of potential account fraud by showing the percentage of users with simultaneous logins. The report identifies the percentage of users who potentially gave out their personal account name and password to others.
maximum number of simultaneous logins, and total traffic generated in terms of megabytes) to cut down on this type of fraud. By identifying these users and taking action, the IPSP can reduce costs and free up resources for other, legitimate users.
SUMMARY
An effective Usage Pattern Analysis solution provides the insight and actionable information needed to implement and manage new and creative billing plans, new and better service offerings, improved churn management systems and policies, and enforceable acceptable-usage policies.
11% 89%
Simultaneous Segment
Users with No Simultaneous Logins Users with Simultaneous Logins
Number of Users
11,856 1,450
Figure 3
Simultaneous Logins
An IPSP can use this report to understand the scope of account fraud, and then use the NI report that identifies these users (along with that users total connection time,
WhiteCross
DATA COLLECTION
An effective UPA solution is built on a foundation of a robust, scalable, and complete data collection methodology. The solution must capture subscribers detailed network usage at the application level (layer 7 of the OSI Model) to completely characterize the applications and services used and the quality of the users experience. This data collection must be scalable to meet the needs of even the largest IPSPs, and, since UPA is the foundation of all IBI solutions, data collection must be revenue-grade: All applications and services must be visible, and no data can be lost.
actively engaged on the network for one hour per day, and assume that each generates 120 records per day (records per day can vary dramatically depending on the granularity of data captured; 120 records per day is an average number of records). This means that the subscribers on the IPSPs network will generate 12,000,000 records per day. To work with 60 days of IP data, the IPSP would need a UPA solution that could handle 720,000,000 records in a timely fashion. This is a conservative estimate; by changing just a few assumptions, the numbers of records are counted in billions.
WhiteCross
interface to the network to monitor the data stream and parse each packet to extract packet header and payload information in order to reconstruct user sessions in real-time. The Analyzers extract information identifying the user, the application (e.g. web browsing, messaging, streaming media, IP telephony) and the content (URL, email, video, multi-point conference) of each session. The Analyzers are non-intrusive, passive devices that process traffic without introducing any latency or impacting network throughput. They are typically installed at major traffic collection points within a providers network, i.e. at primary points of presence (POPs).
with data from network and application devices using NARUS Virtual Analyzers. The Virtual Analyzers interface with network devices (e.g. routers, switches, gateways) and application devices (servers) and gather information using industry standard or vendor proprietary methods. Virtual Analyzers are available for collecting data from a wide range of devices and techniques including SNMP; log files, Cisco NetFlow, and others.
WhiteCross EasyStart
WhiteCross Xcellerate
Analyzers, NARUS LogicServers turn network-level events into events that have meaning in terms of a service providers offerings. For example, a streaming media session might generate records that record the total elapsed time, or the number of bytes transferred, or simply the fact that a specific piece of content was accessed. With the flexibility to apply different business rules, these real-time aggregators can process the same stream of events in different ways to
meet the needs of different applications, such as fraud detection, policy monitoring or billing. Feeding this stream of events to a WhiteCross Data Exploration Server (WX/DES) provides IP service providers a potent weapon for analyzing, understanding and managing customers and networks in broadband environments.
WhiteCross
the proven capability to analyze usage patterns to the finest level of detail across thousands of services, millions of customers and billions of events.
WhiteCross
experiencing. They also identify the most active users e.g. show usage patterns for the top X% of active users, broken out by service.
Service
200,000
Video Conferencing 2 1
VDOLive NETBIOS-SSN TELNET KERBEROS DOMAIN FTP IMSP IMAP2 AUTH SMTP HTTPS AOL TCP POP3 HTTP
Percent Service Percent of Total Traffic Email File Transfer 8 10 News Groups 15 10 Streaming Audio 15 30 Streaming Video 5 15 Web Browsing 40 90 VoIP
10
5 5
Figure 5
In this example, newsgroups traffic and streaming traffic are equal (15%), but twice as many users utilize streaming audio (30%) as utilize newsgroups (15%). Additionally, email traffic (10%) accounts for a smaller percentage of total traffic than either, yet 85% of active subscribers use the service. With an understanding of the value of a service relative to other services, the IPSP executive can move on to the set of reports available for that service. These reports will show how usage of that service is distributed over the subscriber base, how it varies over time or by region, and (in the case of web browsing and streaming audio and video) where users are going for content. WX/Reporting Services (WX/RS) WX/RS is an on-going service designed to deliver regular tailored reports, models, or analyses to IPSPs. Typical areas covered: s Usage Reporting These reports determine which activities customers are interested in, which information, applications, and services get their attention, and where they find the most value in service offerings. They are designed to provide the information needed to optimize pricing models and shift to value-based pricing. Usage Reports provide information on services that customers are using (e.g. email, web browsing, file transfer, streaming media), the amount of bandwidth
Figure 7
Y
SMTP
1 0
IMSP
Hour
1 2
1 4
KERBEROS
1 6
1 8
2 0
VDOLive
2 2
Figure 6
s Segmentation Reporting These reports are designed to improve targeting of customers and services. The reports identify significant groups by geographic region, access type, bandwidth consumption, etc. and identify strong correlations between services usage. Combined with Usage Reports and creative billing plans, segmentation reporting allows an IPSP to truly differentiate services and offer the most effective, targeted billing plans to subscribers. Figure 7 shows a decision tree extracted from an investigation looking at attributes that are common to people who shop at a website.
N (2,212,492) 66.4% Y (1,117,572) 33.6% Total 3,330,064 SEGMENT Type A Type D N (318,127) 29.4% Y (762,651) 70.6% Total 1,080,778 32.5% ASTROLOGIST N N (277,793) 34.6% Y (524,235) 65.4% Total 802,028 24.1% GENDER Y N (21,925) 10.2% Y (192,993) 89.8% Total 214,918 6.5% GAMBLER N Y Total Y (2,848) 61.8% (1,758) 38.2% 4,606 0.1% TECHIE N Y Total N Y Total <NULL> (78,613) 92.4% (6,479) 7.6% 85,092 2.6% N Y Total Type B Type C N (1,894,365) 84.2% Y (354,921) 15.8% Total 2,249,286 67.5% GENDER F (278,382) 67.4% (134,604) 32.6% 412,986 12.4% INCOME N (75,765) 94.1% (4,721) 5.9% 80,486 2.4% ACCESS M N (1,537,370) 87.8% Y (213,838) 12.2% Total 1,751,208 52.6% FOOD_DRINK
N (40,334) 14.5% Y (238,416) 85.5% Total 278,750 8.4% FOOD_DRINK N N (18,409) 28.8% Y (45,423) 71.2% Total 63,082 1.9% GENDER
HEALTH FITNESS
WhiteCross
10
s Customer Profiling These reports are designed to score customers according to their value to the company. The reports identify profitable versus unprofitable customers, customers likely to churn, customers likely to commit fraud, and customers abusing acceptable use policies. The reports show an individual users behavior over a specified period (e.g., which services they used, when, and how much). The reports drill down to which services an individual user prefers, and which times of the week are peak times for them. s Content Reporting These reports are designed to generate important metrics for web content: popularity of web sites, stickiness, long-term appeal, etc. These reports also identify the characteristics of users using the sites from an analysis of data such as raw hits, hits by user segment, amount of time spent at site, number of visits over specific time periods, sites visited per session, pages viewed, content categories, etc. Figure 8 is an example report showing the daily popularity of a page by the hour.
areas within the network, optimally configure and manage all aspects of network operations, and make the most cost-effective decisions regarding future equipment investments. WX/Analytical Services (WX/AS) WX/AS is an on-going service for proactive investigation of time critical business developments or changes in market or customer behavior. Typical areas covered: s Pricing These analyses measure the impact of pricing on usage behavior, assist with the development of models for optimization of pricing, analyze impact of repricing, measure profitability by customer/day/time/etc s Campaign Targeting These analyses identify key customer segments, analyze the impact of special offers, identify most/least profitable customers, and upselling/cross-selling opportunities s Churn These analyzes build profiles of normal
usage to predict future customer behavior and to flag patterns likely to predict churn, identify loyal customer patterns and the impact of enticements.
140000
120000
Page Impressions
100000
80000
60000
40000
20000
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Hours of Day
Figure 8
s Network Usage Reporting These reports provide network managers and architects with complete visibility into their networks. They are used to assist in reducing costs by optimizing usage of available network resources. The reports help diagnose potential problem
11
WhiteCross
summary
As the focus of IPSPs shifts from building out the physical network to building Internet Business Infrastructure, IPSPs will need fundamental information for effective decision making regarding the implementation, monitoring, and managing of IBI solutions. Therefore, IPSPs will need and demand an effective Usage Pattern Analysis solution. The system will have to be flexible, scalable, powerful, and robust. Data collection should include complete (application-level) subscriber usage information and be capable of capturing the quality of the users experience. The engine that processes the data should be powerful enough to handle even the largest data sets for fast analysis and be able to correlate this data with off-line data sources such as user demographics, billing data, and customer support history data. Finally, an effective UPA solution will offer several service options including a predefined reporting package, as well as customized applications and ad hoc analyses. WhiteCross and NARUS have teamed to bring together the benefits of their complementary core technologies to IP service providers. The combined WhiteCross/NARUS offering has the power to: s Collect complete, application-layer subscriber usage data with minimal impact on the network and IT resources and staff. s Aggregate this usage data into meaningful business information. s Create an enriched data ensemble by correlating this information with off-line line data sources such as user demographics, billing data, and customer support history data. s Perform train-of-thought analysis on this data ensemble scaling to billions of rows/sec. WX/NARUS ASP Services enables business managers to make better informed and better targeted decisions, at a speed consistent with the pace of markets driven by the new communications technologies, despite the exponential growth of the data on which those decisions must be based. It allows unlimited exploration (unconstrained ad-hoc analysis) and data mining of large data volumes captured from IPSP networks in Internet time. WhiteCross and NARUS delivers these capabilities through its joint ASP Services. WX/NARUS ASP Services represent a joint vision of providing companies with a UPA solution that affords them the ability to analyze the huge volumes of IP data collected, relate it to other data sources about customers, discover more actionable opportunities, and enable faster, better-refined decisions all in a proven, secure ASP environment at a cost and time commitment that is easily justified.
WhiteCross
12
business scenarios to identify critical opportunities and return high priority results. The key deliverable for extracting maximum information value is the design and construction of an initial or expanded data ensemble and associated analyses and reports. Once the scope of work for the WX/XC engagement is complete, it is time to move into an on-going service mode, either via a WX/Reporting Service or a WX/Analytical Service.
13
WhiteCross
reports and analyses from their desktops for ad-hoc updates as well as receive regularly scheduled delivery of updates. WX/Analytical Service (WX/AS) is an on-going, continuous service providing the use of WX/NARUS technology and experts for the proactive investigation of time-critical business developments and changes in market or customer behavior. WX/NARUS provides the capability to execute complex analyses at the direction and discretion of the client or at the initiative of WX/NARUS. WX/NARUS provides a forward-looking competitive intelligence function through this service.
WX/AS projects include ad-hoc analysis, data mining, and data exploration projects to investigate particular business issues or opportunities for maximizing the yield from customer information assets. These may be based on WX/NARUS existing portfolio of applications such as Churn or Pricing, or be custom developed to meet a clients specific needs. Examples include: special customer segmentation exercises; churn analyses, profit sensitivity to pricing changes, and preemptive competitive analyses.
WhiteCross
14
388 Market Street Suite 1450 San Francisco, CA 94111 USA Tel: 415 249 0124 Fax: 415 908 6889 Waterside Park Cookham Road Bracknell RG12 1RB United Kingdom Tel: 44 (0) 1344 300770 Fax: 44 (0) 1344 301424 info@whitecross.com www.whitecross.com