How to create and use Macro ?

A macro is a Teradata extension to ANSI SQL that contains pre written SQL statements. Macros are
used to run a repeatable set of tasks.The details of macro can be found in data dictionary (DD) . Macros
are database objects and thus they belong to a specified user or database. A macro can be executed by
Queryman. , BTEQ, by another macro.


How to create a Macro
Create a macro to generate a DOB list for department 321:

CREATE MACRO DOB_Details AS
(SELECT first_name ,last_name ,DOB
FROM TERADATA.employees
WHERE dept_numbr =321
ORDER BY DOB asc;);

EXECUTE a Macro
To execute a macro, call it along with the exec command.
EXEC DOB_Details;

last_name first_name DOB
Ram Kumar 75/02/22
Laxman Sinha 79/04/06

DROP a Macro
To drop a macro, use following command .
DROP MACRO DOB_Details;

REPLACE a Macro
If we need to modify an existing macro , instead of dropping and re-creating it
We can use replace macro command as follows

REPLACE MACRO DOB_Details AS
(SELECT first_name,last_name ,DOB
FROM TERADATA.employees
WHERE dept_numbr = 321
ORDER BY DOB, first_name;);

Parameterized Macros

Parametrized macros allow usage of variables . we can pass values to these variables. Advantage of
using parametrized macros is , Values can be passed to these variables at run-time.

Example
CREATE MACRO dept_list (dept INTEGER) AS
(
SELECT last_name
FROM TERADATA.employees
WHERE dept_numbr = :dept; );

To Execute the macro
EXEC dept_list (321);

Macros may have more than one parameter. Each name and its associated type are separated by a
comma from the next name and its associated type. The order is important. The first value in the EXEC of
the macro will be associated with the first value in the parameter list. The second value in the EXEC is
associated with the second value in the parameter list, and so on.

Example
CREATE MACRO emp_verify (dept_numbr INTEGER ,salary DEC(18,0))
AS (
SELECT emp_numbr
from TERADATA.employees
WHERE dept_numbr = :dept
AND salary< :sal;) ;

To Execute this macro
EXEC emp_check (301, 50000);

Key points to note about Macros:
Macros are a Teradata extension to SQL.
Macros can only be executed with the EXEC privilege.
Macros can provide column level security.
NOTE: A user must have EXEC privileges to execute the macros. It doesn't matter if
he has privileges for the underlying tables or views that the macro uses.


Introduction to teradata utilities : BTEQ
BTEQ is a Teradata native query tool for DBA and programmers. BTEQ (Basic
TEradata Query) is a command-driven utility used to 1) access and manipulate
data, and 2) format reports for both print and screen output.




DEFINITION

BTEQ, short for Basic TEradata Query,
is a general-purpose command-driven utility used to access and manipulate data
on the Teradata Database, and format reports for both print and screen output. [1]



OVERVIEW

As part of the Teradata Tools and Utilities (TTU), BTEQ is a
Teradata native query tool for DBA and programmers a real Teradata workhorse,
just like SQLPlus for the Oracle Database. It enables users on a workstation to
easily access one or more Teradata Database systems for ad hoc queries, report
generation, data movement (suitable for small volumes) and database
administration.

All database requests in BTEQ are expressed in Teradata
Structured Query Language (Teradata SQL). You can use Teradata SQL statements
in BTEQ to:

* Define
data create and modify data structures;
* Select
data query a database;
* Manipulate
data insert, delete, and update data;
* Control
data define databases and users, establish access rights, and secure
data;
* Create
Teradata SQL macros store and execute sequences of Teradata SQL
statements as a single operation.


BTEQ supports Teradata-specific SQL functions for doing
complex analytical querying and data mining, such as:

* RANK -
(Rankings);
* QUANTILE
- (Quantiles);
* CSUM -
(Cumulation);
* MAVG -
(Moving Averages);
* MSUM -
(Moving Sums);
* MDIFF
- (Moving Differences);
* MLINREG
- (Moving Linear Regression);
* ROLLUP
- (One Dimension of Group);
* CUBE -
(All Dimensions of Group);
* GROUPING
SETS - (Restrict Group);
* GROUPING
- (Distinguish NULL rows).


Noticeably, BTEQ supports the conditional logic (i.e.,
"IF..THEN..."). It is useful for batch mode export / import
processing.



OPERATING FEATURES

This section is based on Teradata documentation for the
current release.[1]



BTEQ Sessions

In a BTEQ session, you can access a Teradata Database easily
and do the following:

* enter
Teradata SQL statements to view, add, modify, and delete data;
* enter
BTEQ commands;
* enter
operating system commands;
* create
and use Teradata stored procedures.






Operating Modes

BTEQ operates in two modes: interactive mode and batch mode.
In interactive mode, you start a BTEQ session by entering BTEQ at
the system prompt on your terminal or workstation, and submit commands to the
database as needed. In batch mode, you prepare BTEQ scripts or macros, and then
submit them to BTEQ from a scheduler for processing. A BTEQ script is a set of
SQL statements and BTEQ commands saved in a file with the extension
".bteq"; however, it does not matter what file extension is used. The
BTEQ script can be run using the following command (in UNIX or Windows):

bteq < infle > outfile

Here infile is the BTEQ script, and outfile
is the output or log file.



BTEQ COMMANDS

This section is based on Teradata documentation[1],
and for the detailed usage, please refer to Reference 1.



BTEQ Command Summary

BTEQ commands can be categorized into four functional
groups, as described below:

* Session
control Session control commands begin and end BTEQ sessions, and control
session characteristics;
* File
control specify input and output formats and identify information
sources and destinations;
* Sequence
control control the sequence in which other BTEQ commands and Teradata
SQL statements will be executed within scripts and macros;
* Format
control control the format of screen and printer output.


[edit]

1. Commands for Session Control


COMMAND NAME

FUNCTION

ABORT

abort any active requests and transactions without
exiting
BTEQ.

COMPILE

create or replace a Teradata stored procedure.

DECIMALDIGITS

override the precision specified by a CLI System
Parameter
Block (SPB) max_decimal_returned entry,

or if that entry does not exist,
to indicate what the precision should be for
decimal
values associated with subsequently issued SQL
requests for non-fieldmode
responses.

DEFAULTS

Resets BTEQ command options to the values that
were set
when BTEQ was first invoked.

EXIT

end the current sessions and exit BTEQ.

HALT EXECUTION

abort any active requests and transactions and exit
BTEQ;
also called "HX".

LOGOFF

end the current sessions without exiting BTEQ.

LOGON

start a BTEQ session.

LOGONPROMPT

bypass the warnings related to conventional
LOGON command
use.

QUIT

end the current sessions and exit BTEQ.

SESSION CHARSET

specify the name of a character set for the current
session.

SESSION RESPBUFLEN

override the buffer length specified in
resp_buf_len.

SESSION SQLFLAG

specify the disposition of warnings issued in
response to
violations of ANSI-compliant syntax.

SESSION TRANSACTION

specify whether transaction boundaries are
determined by
Teradata SQL semantics or ANSI semantics.

SESSION TWORESPBUFS

specify whether CLI double-buffering is used.

SESSIONS specify the number of sessions to use with the next
LOGON
command.

SHOW CONTROLS

display the current configuration of the BTEQ
control
command options.

SHOW VERSIONS

display the BTEQ version number, module
revision numbers,
and linking date.

TDP

specify the Teradata server for subsequent logons
during
the current session.

[edit]

2. Commands for File Control


COMMAND NAME

FUNCTION

=

Repeats the previous Teradata SQL request a
specified
number of times.

AUTOKEYRETRIEVE

enables users to specify whether the values of any
fields
associated with Identity Data are returned in
response to a SQL Insert
operation.

CMS

executes a VM CMS command from within the
BTEQ
environment.

ERROROUT

Routes the standard error stream and the standard
output
stream to two files or devices for channel-attached
systems, or to one file
or device for network-attached client systems.

EXPORT

Specifies the name and format of an export file that
BTEQ
will use to store database information returned by a
subsequent SQL SELECT
statement.

EXPORTEJECT

Enables suppression of the additional Page
Advance ASA
Carriage Control Character at the top of the
EXPORT file in REPORT mode for
MVS/VM BTEQ.

HALT EXECUTION

aborts any active requests and transactions and exit
BTEQ;
also called "HX".

FORMAT

Enables all of the page-oriented formatting
commands, or
disables them and centers the response from SQL
SELECT statements, using the
value of the WIDTH command option to determine
the space available.

IMPORT

Opens a channel- or network-attached system file,
of the
specified format, to provide data for USING
modifiers of subsequent SQL
statements.

INDICDATA

Specifies the mode of information returned from
the
Teradata Database in response to SQL SELECT
statements.

INDICDATA and / or
LARGEDATAMODE

specify the response mode, either Field mode,
Indicator
mode, Record mode, or Multipart Indicator Mode,
for data selected from the
Teradata Database.

LARGEDATAMODE

Enables use of Teradata Databases Multipart
Indicator
response mode for inline mode retrieval of Large
Object (LOB) data. BTEQ
limits the record size for exported files to
approximately 64K (65473 for
workstation builds and 64260 for mainframe
builds).
If more than 64K is required, SET
LARGEDATAMODE allows
hex-dump style output (similar to RecordMode
directed to standard output).

OS

executes an MS-DOS, PC-DOS, or UNIX
command from within
the BTEQ environment.

QUIET

Limits BTEQ output to errors and request
processing
statistics. BTEQ displays the results in a format
that is suitable for
performance testing.

RECORDMODE

Returns data from SQL SELECT statements in
client-oriented
data representations rather than character format.

REPEAT

submits the next request a specified number of
times.

RUN

executes Teradata SQL requests and BTEQ
commands from a
specified run file.

TSO

executes an MVS TSO command from within the
BTEQ environment.

[edit]

3. Commands for Sequence Control

Use the following commands to control the sequence in which
BTEQ executes commands:

ABORT
ERRORLEVEL
EXIT
GOTO
HANG
IF...
THEN...
LABEL
MAXERROR
QUIT
REMARK
REPEAT
=

For the commands not listed below, refer to the tables
above.


COMMAND NAME

FUNCTION

ERRORLEVEL

Assigns severity levels to errors.

GOTO

Skips over all intervening BTEQ commands and
SQL
statements until a specified label is encountered,
then resumes processing in
sequence.

HANG

Pauses BTEQ processing for a specified period of
time.

IF... THEN...

Tests the validity of the condition stated in the IF
clause.

LABEL

Identifies the point at which BTEQ resumes
processing, as
specified in a previous GOTO command.

MAXERROR

Designates a maximum error severity level beyond
which
BTEQ terminates job processing.

[edit]

4. Format Control Commands

Use the following BTEQ commands to specify the way BTEQ
presents information for screenoriented and printer/printer-file oriented
output:

DEFAULTS
ECHOREQ
EXPORT
FOLDLINE
FOOTING
FORMAT
HEADING
IMPORT
INDICDATA
NULL
OMIT
PAGEBREAK
PAGELENGTH
QUIET
RECORDMODE
RETCANCEL
RETLIMIT
RETRY
RTITLE
SEPARATOR
SHOW
CONTROLS
SIDETITLES
SKIPDOUBLE
SKIPLINE
SUPPRESS
TITLEDASHES
UNDERLINE
WIDTH

For the commands not listed below, refer to the tables
above.


COMMAND NAME

FUNCTION

ECHOREQ

Enables the echo required function that returns a
copy of
each Teradata SQL request and BTEQ command to
the standard output stream.

FOLDLINE

Splits (fold) each line of a report into two or more
lines.

FOOTING

Specifies a footer to appear at the bottom of every
page
of a report.

HEADING

Specifies a header to appear at the top of every
page of a
report.

NULL

Specifies a character or character string to
represent
null field values returned from the Teradata
Database.

OMIT

Excludes specified columns returned from SQL
SELECT
statements.

PAGEBREAK

Ejects a page whenever the value for one or more
specified
columns changes.

PAGELENGTH

specify the page length of printed reports, in lines
per
page.

RETCANCEL

cancel a request when the value specified by the
RETLIMIT
command ROWS option is exceeded.

RETLIMIT

Specifies the maximum number of rows and/or
columns
displayed or written in response to a Teradata SQL
request.

RETRY

resubmit requests that fail under certain error
conditions.

RTITLE

Specifies a header to appear at the top of every
page of a
report.

SEPARATOR

Specifies a character string or width (in blank
characters) to separate columns of a report.

SIDETITLES

Position summary titles to the left of the summary
lines
in a report.

SKIPDOUBLE

insert two blank lines in a report whenever the
value of a
specified column changes.

SKIPLINE

Inserts a blank line in a report whenever the value
of a
specified column changes.

SUPPRESS

Replaces all consecutively repeated values with all-
blank
character strings.

TITLEDASHES

Display a row of dash characters before each report
line
summarized by a WITH clause.

UNDERLINE

Displays a row of dash characters whenever the
value of a
specified column changes.

WIDTH

Specifies the width of screen displays and printed
reports, in characters per line.

Macros are nothing but pre-written SQL (select, insert, update) which is stored in DBC.

Macros are Teradata Extension.

To execute macro we need EXEC privilege over the macro. Its very imp to note that if a
user has access on macro, then it does not matter whether or not he has access over the
underlying tables or views that macro uses. .i.e. if you dont have access over a table, but
if there is macro that reads from this table and you have access over this macro then you
can still read from that table.

Macro's can be executed from any of viable SQL front end:

SQL assistant(QueryMan)
BTEQ
Preprocessor
CLI
Another MACRO( note that macro can be called from another macro)

MACRO's can be parameterized which allow values to be passed at run time.
Following the MACRO name we write the parameter list. In the parameter list we write each
parameter followed by its data type. When we use this parameter in the macro body we precede
it with a colon.

Example 1: The Below macro will calculate the current perm space used by the table name we
pass as parameter. It takes two parameters, database name and table name as inputs.

create macro MY_SUPPORT_MACROS.CAL_SPACE
( DATABASENAME VARCHAR(30),TABLENAME VARCHAR(30)) --> Parameter List
AS
(select sum(currentperm) as currentperm_in_kb from DBC.tablesize where
DATABASENAME =:DATABASENAME --> using parameter with :(colon)
AND
TABLENAME = :TABLENAME; --> Note this semicolon. Inner query should have its own
semicolon.
);

Executing a macro:

EXEC MY_SUPPORT_MACROS.CAL_SPACE('REFERENCE','MY_TEST_TABLE'); --> Passing
Parameters to macro
Note that when we execute a macro we dont use the word 'MACRO'.

Example 2:

CREATE MACRO MY_SUPPORT_MACROS.SEARCH_TABLE
(tablename VARCHAR(40)) AS(
select * from DBC.tables where tablename like :tablename || '%' ;)
;

EXEC SUPPORT_MACROS.SEARCH_TABLE('TCCR');

The order of parameters is important .The first value in the EXEC of the macro will be
associated with the first value in the parameter list. The second value in the EXEC is
associated with the second value in the parameter list, and so on.

Passing parameters by explicitly naming them.

We can explicitly name the parameters while passing them to MACRO. When this option is used
we can pass parameters in any sequence.

In case we have given a default value for any of the parameter while defining the macro, we can
simply omit that parameter.

Example:

EXEC
MY_SUPPORT_MACROS.CAL_SPACE(TABLENAME='MY_TEST_TABLE',DATABASENAME='REFER
ENCE');

Note that the sequence of the parameters passed is different in the above example.


Defaulting a parameter value :

Example:

create macro MY_SUPPORT_MACROS.CAL_SPACE
( DATABASENAME VARCHAR(30) DEFAULT 'MY_TEST_DB' --> if we dont provide any
value for database name it will pick up 'MY_TEST_DB' as the default database name
,TABLENAME VARCHAR(30))
AS
(select sum(currentperm) as currentperm_in_kb from DBC.tablesize where
DATABASENAME =:DATABASENAME
AND
TABLENAME = :TABLENAME;
);


Passing NULL values to a MACRO:

To pass NULL value we can use the keyword 'NULL'
OR
Use positional comma's to implicitly pass null.

EXEC MY_TEST_MACRO('value1',NULL);

OR

EXEC MY_TEST_MARCO2('value1',,'value3');


Performance Effect of using MACRO:

When using parameterized macro's, we can execute the same macro by changing the parameter
multiple times. Each time the body of the query does not change. Only the value being passed
changes.

This allows to reuse the same cached steps. Thus it helps improve performance.

Implementing BTEQ restart Logic using OS command
The logic involved maintaining a file (say bteq_restart_step.txt) which will have the name
of the step from where the BTEQ has to be restarted.

Every Time a step completes the label name for the next step is written into the file.
So next time to submit the BTEQ it skips the step that were already executed and starts
running from the next one.

At the end of the BTEQ we delete that file.

Note that for writing and deleting the file we are using the BTEQ OS command.



.logon ${PTD_TDP}/${PTD_DB_USER},${PTD_DB_PASSWD};
.run file bteq_restart_step.txt;

.label sql1;
DELETE FROM ${PQA_GSC_DCM_DB}.table1; ---- sq11
.if errorcode <> 0 then .goto EXIT_ERROR;
.if errorcode = 0 then .os echo '.goto sql2' > bteq_restart_step.txt;

.label sql2;
DELETE FROM ${PQA_GSC_DCM_DB}.table2; ---- sql2
.if errorcode <> 0 then .goto EXIT_ERROR;
.if errorcode = 0 then .os echo '.goto sql3' > bteq_restart_step.txt;

.label sql3;
DELETE FROM ${PQA_GSC_STG_DB}.table3; ---- sql3
.if errorcode <> 0 then .goto EXIT_ERROR;
.if errorcode = 0 then .os echo '.goto sql4' > bteq_restart_step.txt;

.label sql4;
DELETE FROM ${PQA_GSC_STG_DB}.table4; ---- sql4
.if errorcode <> 0 then .goto EXIT_ERROR;
.if errorcode = 0 then .os echo '.goto sql5' > bteq_restart_step.txt;

.label sql5;
DELETE FROM ${PQA_GSC_STG_DB}.table5; ---- sql5

.os rm -f bteq_restart_step.txt;
.quit 0;

.label EXIT_ERROR;
.quit 99;