Getting connected with your DATA: Using SAS/CONNECT and SAS/ACCESS to work with data housed in a remote environment

Kevin Delaney, New York State Office of Mental Health, Albany, New York

Abstract This paper will provide an overview of SAS/CONNECT and SAS/ACCESS software for accessing and manipulating databases located in a remote environment. Using the example of an Oracle database on a remotely located Unix server, the author will demonstrate many of the main features of SAS/Connect and SAS/Access. SAS/Connect topics to be covered include: connecting to the remote server, submitting SAS code remotely, and moving data back and forth between the server and client. SAS/Access topics to be covered include: interfacing with the data using both the libname statement with SAS/Access specific options and running a query against the data using the SQL pass through facility. Issues of efficiency and practicality will also be discussed. Introduction At this company, many of our large data sets are housed in an ORACLE relational database, on a Unix server. In order to access these data from our local area network, and work with them in SAS, we had to become familiar with the SAS/CONNECT and SAS/ACCESS software packages. Like most SAS software, within these packages there are many different ways to reach the same goal. Once you become familiar with several of these methods, the only challenge is to figure out which method is most appropriate for a given circumstance. This paper will attempt to walk through several of the more common utilities of SAS/CONNECT and SAS/ACCESS software, and hopefully clarify which methods are most efficient and most practical. Throughout this paper I will stick with what I know, I will use examples involving a SAS/CONNECT session with a Unix server, and the SAS/ACCESS interface with ORACLE relational databases. For those of you who know more about other operating systems, or other database management systems, I hope you will find that my examples are adaptable to your host/entity of choice.

SAS/CONNECT : The introduction to the SAS/CONNECT User's GuideTM tells us that SAS/CONNECT is a "SAS-toSAS client/server toolkit." What exactly does this mean? SAS/CONNECT software can be used to connect to a SAS session running on a remote server, to transfer data between environments, and to process data on the remote server. I will attempt to address the multitude of methods that SAS/CONNECT provides for accomplishing these tasks within this paper. SAS/CONNECT also supports SCL commands and SAS/AF applications that allow for remote messaging, linking of objects on different platforms, and running of scheduled applications for routine updates from one server to another, but I will not cover these topics here. Getting Connected There are several methods within SAS/CONNECT that can be used to actually connect to a remote environment. In the Windowing environment you can use the SIGNON window to connect to the remote host. Select RUN from the toolbar, then SIGNON

Figure 1: Select Run and SIGNON from the Display Manager Pull down menus. This gets you the following SIGNON menu:

For my example I am using a script call tcpunix2.scr, to connect to the remote host Unixdata, using the TCP/IP access method. These are the only three lines that need to be filled in, as the NOTE on the bottom of the window states, leaving a field blank will default to the current setting. The only other item you might want to change is whether or not remotely submitted commands execute synchronously, but we will discuss this more fully in a minute. If you prefer a more programmatic approach when signing on to the remote host, the syntax is equally easy to grasp. In SAS Version 8, you need only associate the fileref RLINK with your script file and then issue the SIGNON command. For my example: filename rlink 'tcpunix2.scr'; signon unixdata; Passing SAS statements to the remote host Now that we are signed onto SAS "up" on Unix, lets send some SAS commands through and see how it works. To send SAS statements to a remote host you need only bracket your normal SAS code with an RSUBMIT; - ENDRSUBMIT; block. For instance: rsubmit; libname myunix '/home/myunixdir'; endrsubmit; Creates the LIBRARY MYUNIX within the session of SAS executing on Unix, and then returns the log from this remote session to your local SAS Log. (If we had done something that produced output, the output would also be directed down to the local output window.) Remote Library Services SAS/CONNECT also offers the ability to create a local library that refers to files on the remote session using the REMOTE engine. This is useful if you wish to use the Explorer window to look at the SAS data sets housed in your remote directories. The syntax to create a LOCAL libref to the same directory as our MYUNIX LIBRARY "up there" would be: libname mylocux '/home/myunixdir' server=unixdata;

Figure 2: SAS/CONNECT SIGNON window SAS/CONNECT ships with a number of script files that establish the connection between SAS on the local host and SAS on the remote host. These are specific to the remote host, but can be modified from their standard form. You can also write your own script file, instructions for doing so are included in the SAS/CONNECT User's Guide, Version 8TM. By default these script files are stored in the !SASROOT\CONNECT\SASLINK folder in Windows. SAS will also look for script files in your SASUSER folder. An example of the default TCPUNIX.scr that ships with SAS is attached to this paper, as well as my modified TCPUNIX2.scr, see if you can recognize the changes. As you might guess you can have a lot of fun with the script files, if you are so inclined. SAS/CONNECT supports several different access methods that are operating system dependant. All of my examples will involve the TCP/IP access method for communication between Unix and Windows. I will not say a whole lot about access methods other than to mention that you need to use one that may be used by both the local and remote hosts. For a more in depth discussion see: Communications Access methods for SAS/CONNECT and SAS/SHARE software, Version 8TM. With this information, and the name of the remote host onto which you would like to connect, you can then sign on to your remote SAS session using the SAS/CONNECT SIGNON window pictured in Figure 2. You would place the name of the script file you would like to use in the first line, the remote session's name in the second and your communications access method in the third.

Once you have set up this remote libref you can then manipulate data on the remote host without wrapping it in an RSUBMIT; - ENDRSUBMIT block. For example: proc contents data=mylocux.set1 out=mylocux.set1contents; run; If you happen to know the directory you have been assigned on the remote host this works well, but what about viewing the work directory? You can use the SASHELP.VMEMBER data set view on your remote host to set up a local libref to your remote WORK library: rsubmit; data findwork; x=1; run; data find2(keep=path); set sashelp.vmember; if Upcase(memname)='FINDWORK'; run; proc download data=find2 out=finduxwork; run; endrsubmit; data _null_; set finduxwork; call symput("workdir",trim(path)); run; %put &workdir ; *to make sure it worked; libname unixwork "&workdir" server=uxdata2; Notice we are looking for the Unix WORK library so we need to SET SASHELP.VMEMBER from Unix, by using an RSUBMIT with our data set. For those of you who have not used the VMEMBER data set view in the past, it contains the attributes of all the data sets currently referenced in your SAS session. By creating a dataset in the WORK library and then selecting the variable path for that data set, we obtain the full path of our current WORK library. This example also adds a new SAS/CONNECT procedure. PROC DOWNLOAD, and its partner in crime PROC UPLOAD, are SAS/CONNECT procedures that perform data transfer. The syntax for the procedures really is as easy as it looks. For PROC DOWNLOAD DATA= data set name refers to the data on the remote host which you wish to

push to your local SAS session. OUT=data set name is the name of the data set that will reside in the local session. In this case the procedure copies the data set FIND2 from Unix down into the data set FINDUXWORK on our local SAS session. This data set is then used to create the MACRO variable WORKDIR, and a remote library ref to WORKDIR is established. This seems like a lot of work, but it actually executes in tenths or hundredths of seconds, and then allows you to use the local EXPLORER WINDOW to look at data sets on the remote server, rename them interactively, and even move them to other referenced libraries on either host. You can use the remote library reference as you would any other library reference, so you can SET data on the remote host, and use it to create a local data set, you can use PROC PRINT to print data from the remote host, and well, you get the point. However, this is often not the most efficient way to use the SAS/CONNECT product. For example, lets look at the following code: HEAT # 1 data work.test; set unixwork.smallset; run; rsubmit; VS. proc download data=work.smallset out=work.test2; run; endrsubmit; Heat #2 data unixwork.test; set localref.smallset; run; VS. rsubmit; proc upload data=localref.smallset out=work.test2; run; endrsubmit;

Heat # 3 proc format library=work cntlout=unixwork.fmts; run; rsubmit; proc format library=work cntlin=work.fmts; run; endrsubmit; VS. proc format library=work cntlout=work.fmts; run; rsubmit; proc upload data=work.fmts out=work.fmts2; run; proc format cntlin=fmts2; run; endrsubmit; I am not sure where the word HEAT comes from, but definition 10a in my dictionary does state " One round of many in a sporting competition, such as a race." This example pits remote library services against PROC DOWNLOAD/UPLOAD in a little contest to see who is faster. With relatively small numbers of observations, and particularly with small numbers of variables, these two methods come pretty close. However, PROC DOWNLOAD/UPLOAD definitely wins both HEAT # 1 and HEAT # 2. The advantage to using this procedure over the Remote library option grows wider as you add more variables and observations to the data sets you are moving between hosts. Of course if you are cleaning up for the night and interactively moving data from your Unix work directory to a permanent library it might be easier to click and drag in the EXPLORER WINDOW, but for long programs that need to be duplicable and or completely automated, PROC DOWNLOAD/UPLOAD seems to make more sense. HEAT # 3 is much closer, because there is an extra step needed to use PROC UPLOAD to move the data. Also, unless you have a HUGE FORMAT CATALOG, I don't know that the FMTS data set will ever be big enough to see a real difference in efficiency. What would be neat (this is directed to those SAS people who make this stuff happen) is if options fmtsearch = (work.formats unixwork.formats library.formats); actually worked. Unfortunately as it stands now if you try to assign formats located in the unixwork library or any other remote library using the OPTIONS FMTSEARCH=() option and a remote library reference, you won't get an error, but when you try to assign a format from a remote FORMAT catalog to a local session variable it won't work. This is because "You cannot open a catalog through a server because access to catalogs is not supported when the user machine and server machine have different data representations." (If you want to see this "NOTE" yourself double click on the FORMATS catalog as it appears in the UNIXWORK folder of the EXPLORER WINDOW.) Are we having fun yet? The best attributes of SAS/CONNECT software are still ahead of us. Not only can SAS/CONNECT talk back and forth with a remote host, but it can also do so asynchronously. To this point we have not made use of the WAIT=NO option in any of our RSUBMIT statements. This option tells SAS to send the SAS statements in the RSUBMIT; - ENDRSUBMIT; block through to the host server, but to immediately return control to the local SAS session. We haven't used this option thus far because we haven't needed it; all of the code we have submitted executed and returned results faster than we could blink. This would not be true if we were trying to pull records out of a database with a couple million records, or to perform an SQL query that combines ten tables from a relational database. In my mind the best reason for using SAS/CONNECT is to be able to send large, memory intensive tasks such as these to another server, and let the processing take place on the remote host, allowing you to be free to do other things locally. This is especially true if you store your data remotely so as not to bog down your local server. We will look more closely at the uses of the SAS/CONNECT WAIT=NO and other statements that work with it as we turn our attention to another important piece of SAS software.

SAS/ACCESS software If your data is stored in a format other than a SAS data set on the remote server you are CONNECTed to, how do we ACCESS it?? In effect SAS/ACCESS software provides a SAS-toNONSAS database management software connection in the same way that SAS/CONNECT is a SAS-to-SAS connection. SAS/ACCESS allows you to read in and modify data housed in a NONSAS data storage package, and then write that modified data back out to the database. From the data analysts prospective, I don't have a need to write data back out to the database, in fact, in my job; I don't have the privilege of doing so. My focus will therefore be on the various ways to 'access' data stored in a relational database, using SAS/ACCESS, rather than on the way to write these data back out (PROC DBLOAD). Again, the examples in this paper discuss accessing ORACLE tables on a Unix server, if you are using a different DBMS, see the SAS/ACCESS User's Guide specific to your product for modifications that you might need to run these examples on your system. SAS/ACCESS software provides three main methods for accessing a relational database, The ACCESS Procedure, a DBMS specific LIBNAME statement, or the SQL Pass-through facility. I will compare and contrast the three. The ACCESS Procedure This procedure is the most code intensive method of accessing a DBMS (Those of you deathly afraid of SQL will note that I didn't say 'of using DBMS data'), although none of the code is particular difficult to grasp. The ACCESS procedure for relational databases consists of two distinct components, the ACCESS descriptor, and the VIEW descriptor. The ACCESS descriptor is a set of statements that tells SAS how to access a DBMS table. For example: proc access dbms=oracle; create work.mytest.access; user=kpd; orapw=mypassword; table=category_service; path='prda';

assign=yes; rename catsrv_code=CATCODE catsrv_label=Service; list all; This is the access descriptor for an ORACLE table called CATEGORY_SERVICE within the ORACLE instance 'prda'. The access descriptor contains the information SAS/ACCESS will need to read this table when it is called upon to do so, including my userid (USER=) and password (ORAPW=). ASSIGN=YES tells SAS that all attributes of data sets created from this ACCESS descriptor must conform to what is described here. For example, I have renamed the ORACLE field catsrv_code to be CATCODE. Any SAS data sets created using this descriptor will contain the variable CATCODE, and I will not be able to rename them in the VIEW descriptor. In addition to RENAME you can also use such familiar SAS options as FORMAT and DROP within the ACCESS descriptor. A VIEW descriptor uses the information contained in its reference ACCESS descriptor to access the database, then CREATE VIEW to "take a picture" of the data. When you create a view, you actually set up a query of the data, which can later be called by any SAS procedure or DATA step. You can also create a SAS data set from the ACCESS procedure, but it must occur after the initialization of a VIEW description. In other words while we would like: rsubmit; proc access dbms=oracle; create work.mytest.access; user=kpd; orapw=noturpassword; table=category_service; path='prda'; assign=yes; rename catsrv_code=CATCODE catsrv_label=Service; list all; create work.myview.view out=outputdataset; select catsrv_code catsrv_label; subset where catsrv_code ='96'; run; We instead need to use a second PROC ACCESS statement to create the data set: rsubmit;

proc access dbms=oracle; create work.mytest.access; user=coevkpd; orapw=urnosey; table=category_service; path='prda'; assign=yes; rename catsrv_code=CATCODE catsrv_label=Service; list all; create work.myview.view; select catsrv_code catsrv_label; subset where catsrv_code ='96'; run; proc access viewdesc=work.myview out=oratable1; run; endrsubmit; Notice that I submitted this code to my SAS session running remotely. This is, even in the case of a data set with only two variables and one observation, the most efficient way of using PROC ACCESS. There are two reasons for this, first the Unix server is far less bogged down with everyday traffic than my Windows server. Even if I had a copy of this ORACLE database available locally, SAS could probably read it faster "up there." Second, since I don't actually have a copy of the data to access locally, it is much faster to access and manipulate it up where it lives than to pull the data through my network connection to Unix (which is what would happen if I submitted the code without the RSUBMIT). The LIBNAME statement The next option available to me is to reference the ORACLE instance where my data is stored using a LIBNAME statement. libname dwh1 oracle user=kpd password=stopasking path='dwh1' schema=cpeom; libname dwh1 oracle dbprompt=yes schema=cpeom; rsubmit; libname dwh2 oracle user=kpd password=iwonttell path='dwh1' schema=cpeom; endrsubmit;

The first piece of code represents a local LIBRARY reference to the remotely stored ORACLE data. The second demonstrates the DBPROMT= option discussed below. The main reason I can think of to set up the local LIBREF is the same as the reason we used the SERVER= option earlier. It provides a way to look at and move the smaller data tables interactively. The third example shows the preferred method, remotely submitting the library reference to create the ORACLE library as close to the data as possible. Like remote submitting PROC ACCESS in the previous section, we are trying to avoid pulling data through the network until absolutely necessary, i.e., when we have a small enough subset of our data to use PROC DOWNLOAD or REMOTE LIBRARY SERVICES. In case you are wondering the SERVER= option presented in the SAS/CONNECT portion of the paper applies to the REMOTE library engine, while ORACLE in your LIBREF here calls the ORACLE library engine, so we can't combine the two to get a local copy of a remote ORACLE library. Nice thought though. The LIBNAME statement with options for the SAS/ACCESS to ORACLE engine has two distinct advantages over PROC ACCESS. First, by referencing the ORACLE instance (an instance is ORACLE's way of saying LIBRARY) you set up a reference to an entire group of tables at once, rather than having to create a descriptor for each table. Second, by using the DBPROMPT= option you can tell SAS to prompt you for your username, password and path rather than leaving them laying out in open code. (Note: this obviously will not work in BATCH SAS code, nor will it work for a remote library reference, since you won't have access to the resulting prompt locally.) SQL Pass-Through Facility For those of you familiar with SQL, the code for PROC ACCESS probably looked familiar. That is because SQL queries underlie most of what SAS does with SAS/ACCESS for relational databases. (SAS/ACCESS software for other types of database management systems that do not use SQL to operate on the data stored within them works differently.) If you do not use SQL, don't know how to use SQL, and have no interest in learning SQL, then the SQL Pass-through facility is not for you. You can do pretty much everything you want to do with your

DBMS data using PROC ACCESS or the LIBNAME statement, and never have to write any "real SQL code." But if you are going to be working with data with large numbers of observations, or many (50, 100, 250, etc.) related tables, you might want to start playing with SQL. Here is an example of what looks to be a complicated SQL Query (its really not that bad, but I am not teaching SQL today so you will have to take my word for it) that combines information from three different tables in a relational database with over 1 million total records. It produces a count of the total number of clients served per year by county.: rsubmit wait=no; proc sql; connect to oracle (user=kpd orapw=mylipsrsealed path='pwh1' schema=snp); create table querytable as select * from connection to oracle (select dates.year, counties.ctyofres, count(distinct services.recipient) as tot_served from snp.dates dates, snp.services services, snp.counties counties where dates.datekey=services.datekey and counties.countykey=services.countykey group by dates.year counties.cntyofres); disconnect from oracle; quit; endrsubmit; *rdisplay unixdata; /*Pick one of us not both*/ *rget unixdata; There are several key points. First to toot SQL's horn a little, notice that it did not require sorting the database to perform by group processing, that it produced a frequency count for me, and that it also essentially produced a report dataset of Total clients served by county and year. Second, what you may not have noticed is probably the most exciting part of this SQL code, the CONNECT TO ORACLE and SELECT * (SQL for

ALL) FROM CONNECTION TO ORACLE statements. These statements are used to leave SAS entirely and run this query from within the ORACLE database itself. SAS then is passed the results of this ORACLE SQL query, which it uses to make the data set QUERYTABLE. This is by far the most efficient way of running a query against a database this large and complicated. It lets ORACLE do the work it was designed to do, and then lets SAS do the rest. This could have been submitted on Unix using a LIBRARY reference for ORACLE such as the DWH2 from my LIBNAME example, but this would have been slower than the query that uses the SQL-Pass Through facility. The query could also have been run using the local LIBREF DWH1, but this would have been by far the slowest option (in the case of queries against HUGE data sets the slowest by HOURS). Also, since this was submitted remotely with the WAIT=NO you can run other SAS procedures locally while this is running on your remote SAS session. The last two lines of code bring us back to SAS/CONNECT. The RDISPLAY and RGET commands are used with the WAIT=NO option to go up to the remote server at a later point in time and pull down the SAS LOG and output printed to the LISTING OUTPUT destination. RGET puts these results into your local LOG and OUTPUT windows respectively, while RDISPLAY opens two more windows to display this output separately. Of these two, I prefer RGET. The reason for this preference being that you can use RGET with PROC PRINTTO to save a local copy of the remote SAS session's LOG and OUTPUT, separate from your local SAS session log. proc printto log='remote.log' print='remote.lst' new;

run; rget unixdata; proc printto;run; I haven't figured out a good way to do this with RDISPLAY output, other than to interactively copy the LOG or OUTPUT and then paste it into some other text file for later. Conclusion This paper was intended to present just some of the many ways to use SAS/CONNECT and SAS/ACCESS software, and, within the ways presented, to describe their pros and cons.

Hopefully the suggestions CONNECTed with you, and they will serve to make these two valuable packages more ACCESSible to you. References SAS Institute Inc. (1999), Communication Access Methods for SAS/CONNECT and SAS/SHARE software, Version 8, Cary, NC: SAS Institute Inc. SAS Institute Inc. (1999), SAS/CONNECT User's Guide, Version 8, Cary, NC: SAS Institute Inc. SAS Institute Inc. (1999), SAS OnlineDoc, Version 8, Cary, NC: SAS Institute Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. Contact Information Please send questions, comments and suggestions to: Kevin Delaney NYS Office of Mental Health 44 Holland Ave Albany, NY 12229 (518) 473-7868 coevkpd@omh.state.ny.us