Tech Exchange:

What’s new in SAS Data Management Advanced 9.4 M5

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 Agenda

What’s new in SAS Data Management Advanced 9.4 M5?

• SAS Data Integration Studio 4.903
• Other enhancements
• SAS/ACCESS engines
• SAS In-Database Technologies
• Hadoop @ SAS
• Useful resources

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 Business drivers

• This release represents ongoing efforts at making it

easier for our customers to continue to use SAS 9.4
technologies while adopting SAS Viya technologies

• Certain aspects of this release give us additional support for Cloud and
Hadoop-based data management activities, letting us meet our customers’
demands for better handling of non-traditional use cases

• SAS will continue to support the investments that our customers have made
in SAS 9.4-based products and offerings

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 Technical drivers
• This release delivers new features that enhance
technical integration between SAS 9.4 and SAS Viya:
• The SAS Platform
The SAS Platform

SAS 9.4 SAS Viya

• Hadoop continues to be an important data storage and runtime

environment for SAS
• This release makes key enhancements to stay up to speed with the latest
developments in Hadoop

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
New Features
SAS Data Integration Studio
• New CAS Table Loader transformation replaces the CAS Transfer transformation.
• New Table Maintenance transformation
• K-functions are supported in code generation.
• Hadoop code generation is enhanced.
• Hive transformation enhancement.
• New parameter used by the Command Line Batch Deployment Tool.
• New option generates JCL-friendly code on z/OS systems when using user-written code.
• New macro variables for generated transformations.
• The Cloud Analytic Services Transfer transformation was replaced by the Cloud Analytic
Services Table Loader transformation. Existing jobs that use the Cloud Analytic Services
Transfer transformation should continue to work.

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
CAS Table Loader transformation replaces the
CAS Transfer transformation
• Lets you continue to run SAS processes in
production on SAS 9.4 while taking advantage of
new capabilities on SAS Viya

• Provides a robust data integration environment

for provisioning data to report builders and data
scientists who can use SAS Data Preparation for
“last mile” data manipulation

• Supports orchestration of complex data

integration processes that span the entire SAS
Platform
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
What’s new? SAS Data Integration Studio 4.903
CAS Table Loader transformation

• Uses PROC CASUTIL with the CAS LIBNAME

• Transformation can load in serial or parallel

• It can also load from

• CAS libraries
• SAS libraries
• Workspace server accessible files

… without using SAS/CONNECT.

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
CAS Table Loader transformation
LOAD CASDATA File-based and Connector based – parallel too if available

• loads a file or table into CAS that is known to CAS through

a CAS library defined in the SAS Viya environment
• Just as with a SAS library, you can make database
connections and path-based connections such as
HDFS, HIVE, Oracle, but everything is defined on the
SAS Viya - CAS server itself (This is the default load
option)

LOAD DATA SAS/ACCESS engines

• loads a table from a SAS 9.4M5 library into CAS as a table

Excel, delimited, SAS Data in Hadoop, SAS Dataset, SAS
LOAD FILE Format Catalog, SAS JMP Data File, SPSS, Stata
• loads a file remote to SAS Viya (but local to the SAS 9.4M5
Workspace Server) into CAS as a table
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 SAS Data Integration Studio 4.903
CAS Table Loader transformation

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS Data Integration Studio 4.903
CAS Table Loader transformation

• In the 9.4 M5 metadata

environment the logged-in
user gets passed to CAS
for authentication
Note the format of the User ID – this matches
with the domain user accessible by Viya

• For a non-metadata
environment – regular SAS
session we must use an .authinfo file on CAS controller
.authinfo file on the CAS
controller and a _authinfo
file on the client machine Corresponding _authinfo file on the client

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS Data Integration Studio 4.903 CAS Table Loader transformation
Load CASDATA

SAS Data SAS Data SAS Analytics

1 SAS Visual Analytics
Integration Studio Preparation Offerings

SAS Viya

SAS 9.4M5 CAS (In-Memory)

Persist Option

CAS CAS
Libraries Libraries
Tables
Files
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 SAS Data Integration Studio 4.903 CAS Table Loader transformation
LOAD DATA

SAS Data SAS Data SAS Analytics

2 SAS Visual Analytics
Integration Studio Preparation Offerings

SAS Viya

SAS 9.4M5 CAS (In-Memory)

Persist Option

SAS CAS
Libraries Libraries

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS Data Integration Studio 4.903 CAS Table Loader transformation
LOAD FILE

SAS Data SAS Data SAS Analytics

3 SAS Visual Analytics
Integration Studio Preparation Offerings

SAS Viya

SAS 9.4M5 CAS (In-Memory)

Persist Option

CAS
Files
Libraries

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 Demo
Demo environment
and
using the CAS Table Loader transformation

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
Table Maintenance transformation

Problem
• You are a DBA or data manager who needs to create or
enhance a table before using it in a job. You do not want
to write a macro, but want to create the table using SAS
code instead of using an Oracle client
• Note: You can have multiple statements for a single
table and use multiple tables in the same job
Solution
• Add the Table Maintenance transformation to the job
and connect it to the target table
• Note: the table does not have to exist
• Most of the effort involves building statements
with user-written code
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
What’s new? SAS Data Integration Studio 4.903
Table Maintenance transformation

• Functionality to aid in table create, alter, drop,

partition, and truncate scenarios

• Allows users to enter code that performs actions

on a table

• Supports submitting pass-through SQL code to

Oracle or other database tables.

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS Data Integration Studio 4.903
Table Maintenance transformation

• You can use the Statements tab of

the transformation to view and
define SQL statements

• Generate pass-through code is

selected by default
• If this check box is selected,
the code from this tab is
placed in generated code and
executed as pass-through
code
• The check box is disabled if
the table type does not
support pass-through
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 SAS Data Integration Studio 4.903
Table Maintenance transformation

• You can use the Statements tab of

the transformation to view and
define SQL statements

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

SAS Data Integration Studio 4.903 Information that an Oracle
Table Maintenance transformation DBA will expect to see

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
K-function support

• K-functions are a set of string functions

• Users can now specify the use of K-functions for

• code generation in the Loop
• SCD Type 2 transformations

• Resolves unexpected behavior when dealing with

multi-byte data (e.g. DBCS)
• handle national characters in all languages

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
Hadoop code generation is enhanced

• Removed the Hadoop Monitoring tool

(formerly used to monitor Hadoop clusters)
• (PROC HADOOP, LIBNAME, FILENAME, INFILE, FILE,
and so on) now support connection to Hadoop
through the SAS_HADOOP_CONFIG_PATH
environment variable setting

• Hive transformation enhancement

• A new Delete the target table
check box added

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
Command Line Batch Deployment

• A new parameter, "disableperfstats", was added for

use by the Command Line Batch Deployment Tool

• When set to true, it disables performance

statistics (default)
• When set to false, it allows performance
statistics

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
JCL-friendly Code on z/OS Systems

• When using user-written code, the Generate line

lengths less than 80 characters option on the Code
Generation tab specifies whether the length of
generated SAS code should be less than 80 characters

• This enables the generation of JCL-friendly code on

z/OS systems

• Note: If a Column Label (description) is longer than

80 characters, then this issue might affect any
transformation

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

What’s new? SAS Data Integration Studio 4.903
New Macro Variables for Generated Transformations

• Macro variables were added for each input column

and the source table or tables for each mapped
column

• These and other macro variables are generated for

the mappings when you select the Generate column
mapping macros check box on the Inputs/Outputs tab
on a generated transformation’s Property window

• If the check box is selected when you create a new

job, then all of the macro variables are added and are
visible on the job’s Code tab
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 Demo
Table Maintenance transformation

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 Other updates
SAS Metadata Bridges, SAS QKB, SAS/ACCESS, SAS in-Database Technologies

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges
Additions to existing bridges
• SAS Metadata Bridge for General Industry Standards
• SAS Metadata Bridge for Big Data
• SAS Metadata Bridge for Tableau

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges
• Added to SAS Metadata Bridge for General Industry Standards
- Actian Vector (Vectorwise) Database (via JDBC)
- Apache Avro Database Files (JSON)
- Apache CouchDb Database (JSON)
- HP Vertica Database (via JDBC)
- MarkLogic NoSQL Database (for JSON only)
- Microsoft Azure SQL Database (via JDBC)
- Microsoft SQL Server Database SQL DDL - Beta Bridge
- MongoDB Database (JSON)
- Oracle Database SQL DDL
- Oracle MySQL Database (via JDBC)
- Teradata Database SQL DDL

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges
• Added to SAS Metadata Bridge for Big Data
- Apache Atlas
- Apache Hadoop HBase Database (Java API)
- Hortonworks Atlas
- Cloudera Navigator - Beta Bridge

• Added to SAS Metadata Bridge for Tableau

- Tableau Server Data Extract

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 Demo
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metadata Bridges

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metabridge Relationship Loader – accessing Apache Atlas

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
SAS Metabridge Relationship Loader – accessing Apache Atlas
cd "\opt\SASInside\SASHome\SASMetadataBridges\4.3\tools"
sas-metabridge-relationship-loader -host sasbap.demo.sas.com -port 80 -user
sasadm@saspw -password Orion123 -bridgeIdentifier ApacheAtlas -bridgeOptions
d:\demo\atlas\Apacheatlas_options_server1.xml -loadRelationships -log
d:\demo\atlas\ApacheAtlas_server1_RelLoad.log
pause

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
Result: SAS Lineage Viewer – showing Apache Atlas metadata

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
QKB Contact Information 28
All Locales
• Country locale guess and match definitions
• Country (ISO 2 Char), Country (ISO 3 Char), and Country (ISO Number)
standardization definitions
• Date (DMY), Date (MDY), and Date (YMD) match, parse, and standardization
definitions
• E-mail identification analysis, match, parse, and standardization definitions
• Phone locale guess definition
All English Locales
• Country, Country (Region), and Country (Sub-Region) standardization definitions

Reference: What’s New in SAS® Quality Knowledge Base for Contact Information 28
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 SAS 9.4M5
QKB Contact Information 28
• English, United States
• Address standardization definition
• Various Locales
• Where they exist, Contact Info extraction and identification definitions are being
renamed to Field Content
• Standardization and match definitions handling Date and Date/Time
information have been updated so that a 2-digit year is assumed to fall between
1950 and 2049

Reference: What’s New in SAS® Quality Knowledge Base for Contact Information 28
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 SAS 9.4M5
QKB Contact Information 28
• English, Philippines
• Address match and standardization definitions
• City match and standardization definitions
• City - State/Province - Postal Code match, parse, and standardization definitions
• Division standardization definition
• Organization match, parse, and standardization definitions
• Phone match, parse, and standardization definitions
• Postal Code match and standardization definitions
• State/Province match and standardization definitions
• Voter ID parse and standardization definitions

Reference: What’s New in SAS® Quality Knowledge Base for Contact Information 28
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 SAS 9.4M5
SAS Data Loader for Hadoop (Hotfix)
SAS Data Loader for Hadoop (hot fix)
• SAS Viya Support
• Load data to/from SAS Viya - CAS (non-parallel) using the Copy Data directive

• Enterprise Functionality
• Multiple Workspace Server support
- can select the SAS Workspace Server that will run the directives created in the application
• Set code options needed to generate High Availability

• Hot fix roll-up & critical customer reported defect fixes

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

SAS/ACCESS Enhancements

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

First a Vocabulary Lesson

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

Variant – something that is not materially different
than another.

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

Variant – something that is not materially different
than another.

Cloud Variant – Cloud version of a currently

supported database.

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

Variant – something that is not materially different
than another.

Cloud Variant – Cloud version of a currently

supported database.

Database Variant – Database that is compatible with

a currently supported database.

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

New Support Cloud and Database Variants

Microsoft SQL Server Cloud and

• Azure SQL Database Database
• SAS/ACCESS to Microsoft SQL Server Variant

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

New Support Cloud and Database Variants

Microsoft SQL Server

• Azure SQL Database
• SAS/ACCESS to Microsoft SQL Server Cloud and
MySQL Database
• Amazon Aurora (MySQL variant) Variant
• SAS/ACCESS to MySQL
• Amazon RDS MySQL
Cloud
• SAS/ACCESS to MySQL
Variant
• Amazon RDS MariaDB
• SAS/ACCESS to MySQL

Cloud and
Database
Variant

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

New Support Cloud and Database Variants (Con’t)

Oracle
• Amazon RDS Oracle
• SAS/ACCESS to Oracle
• Oracle Cloud Platform
• SAS/ACCESS to Oracle
• SAS Scoring Accelerator for Oracle

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

New Support Cloud and Database Variants (Con’t)

Teradata
• Teradata Intellicloud
• SAS/ACCESS to Teradata
• SAS Scoring Accelerator for Teradata
• SAS Code Accelerator for Teradata
• Teradata Database on Amazon Web Services
• Ditto
• Teradata Database on Microsoft Azure
• Ditto
• Teradata Database on VMWare
• Ditto

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 New Support Cloud and Database Variants (Con’t)

Teradata
• Teradata Intellicloud
• SAS/ACCESS to Teradata
• SAS Scoring
11 Additional SAS/ACCESS Accelerator for Teradata
Platforms
• SAS Code Accelerator for Teradata
• Teradata Database on Amazon Web Services
5 Additional Scoring Accelerator
• Ditto Platforms
• Teradata Database on Microsoft Azure
• Ditto
• Teradata Database on VMWare
4 Additional Code Accelerator Platforms
• Ditto

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 New Support Cloud and Database Variants (Con’t)
Coming soon (with the usual Product Management caveats):
Hadoop Variants Teradata
• Amazon EMR (not the MapR version)Intellicloud
• Teradata
• SAS/ACCESS Interface to•Hadoop
SAS/ACCESS to Teradata
• SAS Scoring Accelerator for Teradata
Microsoft SQL Server • SAS Code Accelerator for Teradata
• Amazon RDS Microsoft SQL Server Database on Amazon Web Services
• Teradata
• Ditto
MySQL • Teradata Database on Microsoft Azure
• MariaDB (on-premises) • Ditto
• MemSQL (on-premises and cloud) Database on VMWare
• Teradata
• Ditto
PostgreSQL …
• Amazon Aurora (PostgreSQL version)
• Amazon RDS PostgreSQL (cloud)

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
New Features
SAS/ACCESS
• SAS/ACCESS to Oracle modernization
• Support LOG ERRORs clause INSERT/UPDATE/DELETE

SAS In-Database Technologies

• DSTRANS enhancements for SAS Code Accelerator use cases
• PROC DSTRANS helps by translating a subset of DATA step code into DS2 code

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

 SAS 9.4M5
New Features
Hadoop @ SAS
• Extend Ranger and Sentry+RecordService support to EP-based products
• Extended support for Hive High Availability
• SPDE on HDFS
• Additional WHERE clause pushdown
• In-cluster sorting

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

SAS Viya
The SAS platform
SAS 9
Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .
 Self Service data
SAS Data preparation
Preparation
SAS data management jobs integrate
data, create single view of subject, Empower users with self
apply data quality, and deliver data to service tools at the “last
users mile”.
SAS Data
Integration

CAS

SAS clients
SAS code/SAS programs to deliver data
to users

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

Useful Resources

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

http://support.sas.com/documentation/onlinedoc/etls/index.html

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

For more information Documentation
System Requirements--SAS Data Integration Studio 4.903
SAS Data Integration Studio 4.903: Installation and Configuration Guide
SAS Data Integration Studio 4.903: User’s Guide
SAS 9.4 In-Database Products: Administrator's Guide, Eighth Edition
SAS 9.4 In-Database Products: User’s Guide, Seventh Edition
SAS 9.4 DS2 Language Reference, Sixth Edition
DataFlux Expression Engine Language: Syntax Reference
SAS/Access 9.4 for Relational Databases: Reference, Ninth Edition
SAS Quality Knowledge Base for Contact Information 28: Help
What’s New in SAS® Quality Knowledge Base for Contact Information 28
http://support.sas.com/documentation/onlinedoc/qkb/28/qkbciwn.pdf

Documentation is available at:

http://support.sas.com/documentation/onlinedoc/entdis/index.html

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .

Thank you !

sas.com

Cop yrig h t © SA S In st it u t e Inc . A ll rig h t s rese rved .