SAP NetWeaver BI

Unicode Compliance

Product Management SAP NetWeaver BI

Version 7.0
December, 2008
 Agenda

1. Unicode in General
2. Excursus: MDMP
3. Unicode support of SAP NetWeaver BI
4. Interfaces to SAP Systems
5. Interfaces to non-SAP Systems

© SAP 2007 / Page 2

Unicode Essentials

What is Unicode?
Character encoding schema for (nearly) all characters used world wide

Each character has a unique number („Unicode code point“)

Notation U+nnnn (where nnnn are hexadecimal digits)
See http://www.unicode.org for complete code charts
Representation of Unicode Characters

UTF-16 – universal transformation format, 16 bit encoding

Fixed length, 1 character = 16bit. Surrogates using 2*16 bit
Platform dependent byte order
16 bit alignment restriction (needs even addresses)

UTF-8 – Unicode transformation format, 8 bit encoding

Variable length, 1 character = 1...4 bytes
Platform independent
No alignment restriction

Character Unicode UTF-16 UTF-16 UTF-8

code point big endian little endian
a U+0061 00 61 61 00 61
ä U+00E4 00 E4 E4 00 C3 A4
U+03B1 03 B1 B1 03 CE B1

U+3479 34 79 79 34 E3 91 B9
Transparent Unicode Enabling of SAP
Business Suite

Character expansion model

Separate Unicode and non-Unicode versions of SAP ERP

1 character = 1 byte
ABAP Non- (types C, N, D, T, STRING)
ABAP
source Unicode Non-Unicode kernel
source
R/3 Non-Unicode database

1 character = 2 bytes (UTF16),

(types C, N, D, T, STRING)
Unicode
Unicode kernel
ERP Unicode database

No explicit Unicode data type in ABAP

Single ABAP source for Unicode and non-Unicode systems
Automatic conversion of character data for communication between Unicode and non-
Unicode systems
Implications

You are not interested in Unicode?

No customer is forced to convert to Unicode
Possibly minor changes for not Unicode enabled programs
(more restrictive syntax check as of WebAS 6.10 on)

You are interested in Unicode?

Major part of ABAP coding is ready for Unicode without any

changes
Minor part of ABAP coding, i.e. some customer-specific ABAP coding,
has to be adapted to comply with Unicode restrictions
SAP will deliver powerful tools to convert your existing system
 Resource Requirements for Unicode-compliant
BI
What are the additional resource requirements after a
Unicode conversion?
Text fields are usually bigger in a Unicode environment than in a Non-Unicode environment. Enhanced
functionality requires additional resources, which strongly depend on the given customer scenario. In the
following, we give a rough estimation what a Unicode conversion could mean for the resources.

CPU’s
We expect additional SAP NetWeaver BI requirements of roughly 30% more CPU power - like
the SAP ERP requirements.

Main Memory
We expect additional SAP NetWeaver BI requirements of roughly 50% more memory - like the
SAP ERP requirements.

Disk Storage
Disk Storage depends strongly on the underlying DBMS and the given data model/volume. For
a significant share of InfoCube data (only numeric keys!), there might not be a significant
increase of the DB size.
For SAP ERP on ORACLE, tests have resulted in roughly 35% additional disk space. For SAP
BI, we expect – depending on the scenario – less additional disk space.
Note that after a conversion, the disk size may even decrease because of DB reorganisation.
The Unicode Upgrade Project

Preparation

Conversion

Post -Conversion
Post-Conversion

Set up the Unicode To be done during system

upgrade project downtime Set up the system
for Unicode
Unload /reload process for
Analyze data to small databases
minimize downtime Start up the Unicode
Minimum downtime tool for system
large databases
(Incremental Migration
IMIG)
 Agenda

1. Unicode in General
2. Excursus: MDMP
3. Unicode support of SAP NetWeaver BI
4. Interfaces to SAP Systems
5. Interfaces to non-SAP Systems

© SAP 2007 / Page 9

MDMP – Definition

What does MDMP mean?

In the past, SAP supported multiple languages by MDMP
MDMP = Multiple Display – Multiple Processing
Mixed code page technology depending on logon language
Several 100 satisfied customers
MDMP was an SAP workaround for accessing multiple code pages
Defined users where accessing the solution using SAPgui frontend
Only little integration into the world wide web
MDMP should not be used anymore for new installations
MDMP is not supported for SAP NetWeaver BI
MDMP – Connection

How can I connect SAP BI to an SAP MDMP source system?

Language-dependent source tables

Most text tables include the language key. In an SAP system, the code page can be derived
from the language key. This enables the correct conversion of the texts from the source
system to SAP NetWeaver BI.

Language-independent source table

If a text table does not include the language key (e.g. customer texts), these texts have to be
converted “manually”. The code page (or language) has to be defined for EACH record in
these language-independent tables either by appending a new column into the table or by
defining the extraction job in the corresponding codepage.
If there is no language information given, the source tables will be converted based on the
logon language of the user.
MDMP – Project Based Solution

Why is there no generic solution for the connection

of MDMP source systems?
All language-independent source tables must be checked and for each record, the code
page which it was created in must be determined
In order to derive this code page information, attribute values, organizational
assignments or heuristics may help, but there is no generic approach
Examples:
in order to determine the code page of a customer name, the country where she/he
lives in may be used usually not correct, if e.g. a Japanese customer lives in
Russia
Alternatively, the assignment to a sales organization may help
For more information see How-To Guide in SDN
Design Principles for Source Systems

How can I ensure a smooth Unicode conversion?

SAP NetWeaver BI receives the following information from source systems
Date and timestamp information
Key figures
Characteristics by key
Texts

How should I design the data in the source system?

Date and timestamp information as well as key figures are just figures; the conversion is not
critical
Characteristic keys (e.g. material number, customer number, brand key, country key) must be
designed as 7-bit-US-ASCII characters (common characters). Don’t use other characters as
keys or attributes.
As a benefit, the permitted character setting in the BW customizing need not be maintained. See SAP
note 173241 for more information.
Note that maintaining the permitted characters makes you code page-dependent. There might be
problems with the conversion to Unicode (e.g. characters like ‘µ’ are a lower case letter in Unicode – and
thus not allowed)
Texts are ONLY loaded into the text tables of the InfoObjects; transaction data and master data
attributes must NOT contain texts. Texts are usually language-dependent and must be designed
in the source system with an associated language key (even if e.g. the SAP ERP data model is
not providing this).
 Agenda

1. Unicode in General
2. Excursus: MDMP
3. Unicode support of SAP NetWeaver BI
4. Interfaces to SAP Systems
5. Interfaces to non-SAP Systems

© SAP 2007 / Page 14

Unicode-Compliant SAP NetWeaver BI

Unicode-compliance means:
SAP NetWeaver BI can interpret and display Unicode characters
User interface will be shown in local language
Business data can use all languages in parallel

SAP NetWeaver BI can extract data from source systems with specific code
pages (Non-Unicode or Unicode)

SAP NetWeaver BI can extract data from SAP source system running mixed
code pages (MDMP)

Interfaces to 3rd party systems support correct code page conversion

Unicode Platform Availability

Which platforms support Unicode?

Database Management Systems
Available DB management systems
MS SQL-Server
DB2/UDB V8
maxDB 7.5+
ORACLE 9.2
DB2/AS400 V5 R2
Delayed DB management systems
DB2/zSeries, available as of 2004
Informix: no Unicode support planned

Operating Systems restrictions

Reliant Unix is not Unicode-enabled
OS/390 available as of 2004
Unicode in SAP NetWeaver BI 7.0 and later

SAP NetWeaver BI 7.0 is fully Unicode compliant

SAP NetWeaver BI Server
Fully Unicode compliant
Note: when loading from a double-byte (i.e. Non-Unicode) SAP ERP source system
(release <= 4.6D) into Unicode SAP NetWeaver BI, so-called container misalignments
may occur (see SAP notes 673533 and 510882 for more information).

SAP ERP
NW BI
<= 4.6D
Non-Unicode Project Solution
Unicode SAP NetWeaver BI

BEx Web Analyzer

Fully Unicode compliant (including right-to-left languages)
BEx Analyzer (.NET tools)
Fully Unicode compliant
Exception: Right-to-left languages are not supported by all controls
Supported Scenarios for SAP BW 3.5

Unicode supprt for SAP BW 3.5/SAP NetWeaver ‘04

SAP NetWeaver BI Server (backend)
Load of data in different code pages
Display of all data in SAP Gui environment
Unicode Web front end
Display of all data if front end is configured in Unicode code page
Restrictions for MS Windows-based client tools (BEx Excel front end, Web
Application Designer, Query Designer) – see also SAP note 588480
Object keys must be in US-7-bit-ASCII to ensure access to keys in all logon languages / code pages
The restrictions only apply to texts, transactional data is not affected.
The client is configured in one Non-Unicode code page. Note that also the Windows default code page must
be switched to this Non-Unicode code page.
Only the data in the logon code page are displayed correctly: I.e. a Russian employee can see all Russian
texts, but not the German ones (‘#’ characters will be displayed for special German characters), whereas the
German employee can see the German texts (but not the Russian ones)
English texts (US-7-bit-ASCII) are displayed correctly
A user must not change and save queries, work books, query elements or web templates with Non-US-7-bit-
ASCII (English) texts that have been originally created in a different code page. For example, a query
containing Russian text elements should only be changed in Russian code page.
Front end Restrictions SAP BW 3.5 in Detail
–1–

Restrictions for MS Windows-based client tools –

Example: BEx Excel Frontend
Logon Language Russian
German characters corrupted

Logon Language German

Russian characters corrupted

No restrictions in Web Frontend

Correct characters from both code pages – independent from logon language
Front end Restrictions SAP BW 3.5 in Detail
–2–
BEx Analyzer Excel front end
Note that the corrupted text is a pure display matter; the data in the database is always
correct!

Query Designer / Web Application Designer

In order to avoid corrupted text written back to the database, organizational means must be
installed. There are basically two options
Queries, query elements, work books, web templates must only contain English texts (US-
7-bit-ASCII)
Queries, query elements, work books, web templates containing texts (also hidden ones in
e.g. restricted key figures or variables) in a specific language / code page must not be
changed by users in a different code page

RTL (Right-to-Left) Languages

The Windows frontend components (BEx Analyzer, Query Designer, Web Application
Designer) do not support RTL languages (like Hebrew, Arabian). However BEx Web does
support them.

Keys in US-7-bit-ASCII
Keys (of any InfoObject) must be in US-7-bit-ASCII to enable users in every code page to
enter the key in a search criterion; if Russian characters were allowed in keys, a German user
could never enter these and, hence, never select these
Supplementation

Business Scenario
You run your SAP NetWeaver BI in different languages, but the availability of translations for
all languages cannot be guaranteed. Hence, you want to define a standard language. This
standard language is used to generate all missing translations.
Example: you want to use English material texts in case the Japanese ones are not available.

Supplementation
The text tables can be supplemented with a standard language.

Usage
Start in transaction SMLT and check by double-clicking on the (target) language (e.g.
Japanese), if the supplementation language has been maintained (e.g. English).
Then use the menu path Language Special Actions Supplementation (Expert) in order to
select all affected tables (usually all language-dependent /BIC/T* and /BI0/T*-tables – those
tables are cross-client class A tables)
If you want to run supplementation periodically, you can use report
RSTLAN_SUPPLEMENT_PERIODIC (parameter: preceding supplementation run) and include
it into your process chain.
Check SAP note 111750 for supplementation of German language
Unicode Installation and Conversion

How can I make SAP NetWeaver BI Unicode-compliant?

Delivery
You can choose between Non-Unicode and Unicode installation
Note: Unicode installation requires more hardware resources (depending on DB platform)

Installation modes
New Install
Conversion of an existing SAP NetWeaver BI
Before the conversion, upgrade your SAP NetWeaver BI to SAP BW 3.5 (or later release)
R3LOAD converts an existing SAP BW 3.5 (or later release) automatically by exporting the
DB, realigning the DB and importing the DB again
Note that the Unicode Conversion is a pilot project as part of a BW System Copy (see note
543715)
Customer-developed programs (variable exits, virtual characteristics/key figures,
transformation rules, table interface etc.) must be in line with the Unicode rules
The duration of a conversion depends on the size of the existing database
 Agenda

1. Unicode in General
2. Excursus: MDMP
3. Unicode support of SAP NetWeaver BI
4. Interfaces to SAP Systems
5. Interfaces to non-SAP Systems

© SAP 2007 / Page 23

Interfaces to SAP Source Systems

Which SAP Source System Types exist ?

Single code page systems
One single code page can include several languages (e.g., Latin-1 contains German, French,
Spanish, Italian etc.)
A Unicode source system is also a single code page system

Mixed code page systems

SAP source systems with several code pages (usually: MDMP)

How does the conversion work?

RFC-Calls are enhanced by source and target code page parameters; they transform
characters „on the fly“ to the correct code page
The RFC call derives the source code page information from the language key
Note: English texts (US-7-bit-ASCII) is converted correctly in ALL code pages
For mixed code page systems, language information on RECORD level defines the conversion
PROJECT based solution
Connection Scenarios – 1 –

Which systems can I connect to ?

Overview on the supported connections
SAP SAP BW 3.x SAP BW 2.x
NetWeaver Non-Unicode Non-
BI Unicode Unicode

SAP source Ok ok, but some Project! Some

Unicode information information
might be lost might be lost
not not
recommended recommended
SAP source Ok (from ok ok
non- double-byte
Unicode sources: on
project (*) see
basis) (*) slide 17
SAP source Project! ok, but some ok, but some
MDMP information information
might be lost might be lost
not not
recommended recommended
Connection Scenarios – 2 –

What are the prerequisites for a proper connection ?

Unicode SAP NetWeaver BI systems and Unicode / Non-Unicode SAP
source systems
SAP BI-Service API 3.0B SP3 resp. PI_BASIS 2002_1_620 SP3 is required to connect the
Unicode source system with SAP BI

e.g. R/3 BI
Unicode / Non-Unicode Unicode BI
Service API 3.0B SP3 or PI_BASIS
2002_1_620 SP3

See slide 17 for restrictions for

double-byte languages from
SAP ERP <= 4.6D. No issues
as of release 4.7.
Connection Scenarios – 3 –

Non-Unicode SAP BW 3.x systems and Unicode SAP source systems

During conversion some information might be lost. The following restrictions apply
Characters might be corrupted as the target system does not know all Unicode characters
In multi-byte code pages (Asian languages), suffix characters could get cut off during the
extraction, as the bigger Unicode containers must be mapped to the target non-Unicode
containers

e.g. R/3 BI
Restrictions apply
Unicode Non-Unicode SAP BW 3.x
Non-Unicode SAP BW 2.x systems and Unicode SAP source systems
SAP BW 2.x as non-Unicode BW system is not generally released to extract data from Unicode
source systems; in special scenarios, this connection might be possible on a project basis

e.g. R/3 BI
Project Solution
Unicode Non-Unicode SAP BW 2.x
Connection Scenarios – 4 –

What about other interfaces to SAP source systems ?

The same connection scenarios apply for transient interfaces to SAP
source systems
Report-Report-Interface
RemoteCubes
Virtual InfoProviders to SAP source systems
Technical Settings for Unicode

Be sure to set up your system according to the following

recommendations
Languages
You can only extract languages to SAP NetWeaver BI, which are defined in the basis parameter
ZCSA/INSTALLED_LANGUAGES (language vector)
 Agenda

1. Unicode in General
2. Excursus: MDMP
3. Unicode support of SAP NetWeaver BI
4. Interfaces to SAP Systems
5. Interfaces to non-SAP Systems

© SAP 2007 / Page 30

Unicode-compliance of Interfaces

Which source system code pages are supported for Non-SAP

sources?
For Non-SAP sources, it is difficult to determine the source code page.
In most cases, only English texts (7-bit-US-ASCII) or Unicode texts are converted correctly
Examples
Flat File
DB Connect (Same restrictions apply like for all MultiConnect connections – independent
from Unicode)
UD Connect
Virtual InfoProviders to Non-SAP sources
Unicode-compliance of BAPIs and 3rd Party
Tools
What about Unicode with respect to BW -BAPI’s and 3rd Party
Tools?
Loading data into BW via Staging-BAPI (and 3rd party ETL-tools)
This interface is released for the following sources: (see SAP note 765543)
Single-byte code pages
Unicode UTF-16
All other code pages, but only for transfer structure of size 250 characters and smaller
This interface is not released for the following sources:
Multi-byte code pages with variable byte length (I.e. Asian code pages and UTF-8) AND
Transfer structure is larger than 250 characters

Displaying data in a 3rd party front end via OLAP BAPI, OLE DB for
OLAP or XMLA
These interfaces are currently released as a pilots for Unicode

ETL or frontend partners

Ask our partners for Unicode-compliance of their products
 Thank you!

© SAP 2007 / Page 33