OceanStor 9000

— storage Systems for Big Data

应用不动 性能翻番

HUAWEI TECHNOLOGIES CO., LTD.

Contents

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 4

Arrival of Big Data

1960s
Mainframe server 1980s
Personal computer 1990s
Internet 2000s 2010
Mobile Internet Big Data

An era of producing, sharing, and using data on a large scale is coming...

— Kenneth Cukier (co-author of Big Data: A Revolution That Will Transform How We Live, Work, and Think)

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 5

Big Data from Genetic Analysis

Gene Genetic
Higher
sequencing Import data analysis
analysis
faster
efficiency

Data sharing Data storage

Access data Higher
faster capacity

Genetic analysis is a process of producing, sharing, analyzing, and storing very large
amounts of gene data. Genetic analysis is a typical Big Data processing workflow.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 6

 Easier Said than Done: Big Data Challenges for Genetic
Analysis Organization CIOs
Gene sequencing Gene data sharing Gene data analysis 1. Three copies of the same
data are stored, wasting 2/3
Gene
of the space.
samples 2. The same data is replicated
Gene data Gene data
replication replication
twice, wasting double the
Local disks of Shared NAS storage Hadoop time.
a sequencer Gene data storage storage
Gene data
storage system
Why would this happen? No single storage system perfectly adapts to all aspects of Big Data processing.

Sequencer Shared NAS Hadoop

Local Storage Storage Storage

Quick read/write √ X √

Interworking analysis X X √

Data sharing X √ X

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 7

Difficulties in Different Industries — Similar Big Data
Processing Workflows
From 2D to 3D, accuracy
From low to ultra-high From SD to HD, the bit- 20M transactions per
increased dozens of
resolution, accuracy rate increased 4 to 8 day, the data must be
times, data increased 20
increased nearly 100 times, data increased 10 saved no less than 15
times, and the efficiency
times and the data times, and the total years and the total
improved dozens of
increased 100 times amount exceeded 10 PB amount exceeded 10 PB
times
Aeronautics and Energy Finance and
M&E Securities
Astronautics Exploration

Data Data
Source Analysis
Massive
Data Data Data
Sharing Storage

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 8

Contents

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 9

OceanStor 9000: A Storage System for Big Data

•Efficient integration of storage

and analysis

•Flexible integration of storage

and archiving

OceanStor 9000 Big Data Storage •Integration of diversified data

华为全融合大数据存储系统
System
types
Archiving Zone
•Ultimate scalability

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 10

 Integration Scalability

All-in-One Architecture
Video Bill Query Net-Surfing Precision Business
Media and HPC Web Disk
Surveillance Behavior Marketing Promotion
Entertainment
Analysis

Application
Layer
Files Objects Query/Retrieval Data Analysis
NFS CIFS FTP Object SQL
HDFS
Distributed Database

Data
Processing Distributed File System
Layer
Archiving

Hardware Node Node Node Node Node Node Node Node Node Node Node

Layer
Node Node Node Node Node Node Node Node Node Node

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 11

 Integration Scalability

Storage and Analysis Integration, Improves

Processing Efficiency

Data Data Data Data Data Data

production sharing analysis production sharing analysis

Replicate Replicate
Storage
Importing Storage Storage
data to sharing analysis resource pool
storage

Three copies of the same data are stored, Only one copy of the same data is stored,
wasting 2/3 of the space. saving 2/3 of the space.
The same data is copied twice, wasting double Data replication is not required, improving
the time. efficiency by two times.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 12

 Integration Scalability

Storage and Archiving Integration, Simplifies Data

Lifecycle Management

External
Application
retrieval system
Storage resource pool
Internal
Archiving retrieval engine
flow

Storage Archiving
Storage Archiving zone zone
device device

Separate management, requiring an Unified management, without using an

independent retrieval system, and wasting independent retrieval system or participation
application layer resources. of the application layer.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 13

 Integration Scalability

Integration of Diversified Data Types, with Support for

Full Data Analysis

Analysis of genetics, videos, Analysis of genetics, videos,

CDR query, BI CDR query, BI
behaviors, etc. behaviors, etc.

Hadoop Hadoop SQL

SQL
File
File
NFS/CIFS NFS/CIFS
Unstructured storage
Unstructured Structured storage
storage
Database
Database Storage resource pool
Structured
storage

Different systems process various types of data and One system processes multiple data types and
support different analysis modes. supports various analysis modes.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 14

 Integration Scalability

Multiple Key Technologies, Building Scale-Out Capability

Dynamic load-balancing eliminates hot spots,

optimally utilizing the resources

The industry’s only dynamic sub-tree cutting

technology, managing large files

All-IP interconnection, fully distributed

architecture, providing flexible scalability

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 15

 Integration Scalability

Highest Storage Performance in the Industry

Performance more than three
times higher than competitors
Unit: OPS
5,030,264 Finance: 0.45 million electric bills can be
5,500,000
5,000,000 archived in one second. All China Construction
4,500,000 Bank transactions can be archived each day.
4,000,000
3,500,000
3,000,000
Satellite mapping: 34,523 HD geographical
2,500,000 images can be written in one second. HD
2,000,000 1,512,784 geographical images of the entire country of China
1,500,000 1,112,705 can be written in two minutes.
1,000,000
500,000
EMC Isilon NetApp Huawei Broadcasting and TV: 1,750 HD channels
are concurrently supported. HD channels of the
Operation Per Second (OPS) refers to the number of requests concurrently
processed per second. entire continent of Europe are supported.
http://www.spec.org/sfs2008/results/sfs2008.html

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 16

 Integration Scalability

Largest File System in the Industry

Petroleum exploration: Total data volume of

BGP CNPC over the next four years

Genetic analysis: Total data volume of BGI

over the next five years
Satellite mapping: Total data volume of
Actual test: 288 nodes largest file system Satellite Surveying and Mapping Application
Center, State Bureau of Surveying and Mapping
(SBSM) over the next six years

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 17

 Integration Scalability

On-Demand Expansion and Purchase

Performance

Capacity

 On-demand expansion and purchase of  Balanced data distribution, eliminating hot spots
performance and capacity  Stacking expansion, enabling a single node to be
 Linear growth, predictable and obtainable added within one minute

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 18

 Integration Scalability

Easy to Start, Easy to Manage

 One set of management software:

Unified management of storage, analysis,
and archiving
 Two types of system resources:
Physical and logical resources
 Three types of management
terminals: PCs, mobile phones, and
pads
 Four types of alarm notification methods:
Email, audio, SMS, and indicators

Get started in one minute.

Familiarize in three minutes.
HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 19
OceanStor 9000 Specifications
Performance

OceanStor Industry
Insight node vs.
9000 No. 2

No. 1 5 million 1.1 million

4x+
Massive data performance OPS OPS
retrieval scenarios
Performance node No. 1
400 GB/s 2x 100 GB/s
throughput

No. 1
P25 P36 60 PB 1.2x 50 PB
capacity
Capacity node
P12
OPS-intensive No. 1
288 nodes 2x 144 nodes
application scenarios scalability
C36 C72
Large-capacity
application scenarios No. 1 cache 55 TB 2x 28 TB

Capacity

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 20

Contents

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 21

Typical Application Scenarios of OceanStor 9000

Satellite Mapping Meteorological Science Energy Exploration Aeronautics and

Astronautics

Gene Sequencing Scientific Research and Video Surveillance Media and

Education Entertainment

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 22

OceanStor 9000 Assists Astronomical Mapping with
High-Performance File Storage
HUAWEI TECHNOLOGIES CO., LTD.
Customer Requirements
 Large-scale astronomical star map simulation and
computing requires a read/write rate of over 100,000 OPS
 Massive star map analysis and research requires GB-
level storage bandwidth
 According to the business plan, capacity must be expanded
to 22 PB within five years
 Eight nodes are configured in phase 1, with a
performance of 160,000 OPS
 High-speed interconnection techniques achieve a total
bandwidth of 5 GB/s and support for a maximum of 200
GB/s
 Distributed architecture supports a maximum of 288
nodes and 60PB
Huawei Confidential 23
OceanStor 9000 Helps CCB with Enterprise Content
Management
Customer Requirements
 Storage of files, such as bill images and reports, requiring file
sharing service
 CCB has 38 branches nationwide; over 600 TB data is
generated each year; data must be stored for 20 years; online
dynamic expansion and smooth performance expansion are
required to avoid data loss caused by system upgrade

Solution
 All-in-one architecture, providing data sharing and
efficient access
 Flexible expansion of nodes, performance, and capacity in line
with the business growth, providing a maximum of 288
nodes and a 60 PB single file system
 Data protection implemented based on different data
security levels

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 24

OceanStor 9000 Solution for SZMG
Customer Requirements
 A large virtualized storage resource pool must be
created to implement resource sharing and on-demand allocation,
reducing migration of media data among independent systems
 The total bandwidth must be 13 GB/s and total capacity must be
1,246 TB; additionally, system performance and capacity must be
flexibly expanded

Solution
 Distributed architecture with all-IP
interconnections
 30 nodes (20 nodes for online storage and 10 nodes for archive
storage), with 288-node scalability and a 60 PB single file
system
 High-performance configuration provided for online storage, large-
capacity configuration provided for archive storage, and
Dynamic Storage Tiering (DST) implemented for
different types of data

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 25

OceanStor 9000 used in Saint Petersburg Safe City
Project in Russia
Customer Requirements
 Administrative systems were deployed separately, lacking
unified management and scheduling and causing
underutilized resources
 For the interconnection transition, HD security surveillance,
front-end systems upgrading was required
 Intelligent analysis and warning based on HD images
would be required as planned

Solution
 The HUAWEI Video Cloud Storage Solution provided
storage space for 12,000 channels of cameras
 In the first phase, a 2-PB OceanStor 9000 was deployed
and would be expanded to 15 PB in the future
 The 6+2:1 redundancy algorithm achieved an optimum
balance between disk utilization and reliability

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 26

OceanStor 9000: Designed for Big Data to Address
Storage Challenges Faced by CIOs
5,030,264 OPS
No. 1 performance
OceanStor 9000 288 nodes
No. 1 scalability

400 GB/s 60 PB
No. 1 throughput Convergent, Scalable, Simple No. 1 capacity

and Efficient
HD image storage Number of concurrent HD Seismic data processing 0.45 million bill
Operation time for genetic
capacity increased by 10 production workstations efficiency improved by images archived per
analysis reduced by 80%
times increased by 2 times 50% second
Finance
Satellite Public Media and Energy and
mapping utilities entertainment exploration Securities

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 27

 Thank you
www.huawei.com

Copyright © 2015 Huawei Technologies Co.,Ltd. All Rights Reserved.

The information contained in this document is for reference purposes only and is subject to change or
withdrawal, according to specific customer requirements and conditions.
All logos and images displayed in this document are the sole property of their respective copyright holders. No endorsement, partnership, or affiliation is suggested or implied.