Introduction to AutoSys

AutoSys is an automated job control system for scheduling, monitoring and reporting. An AutoSys job is any single command, executable, script, or NT batch file. AutoSys job definition contains a variety of qualifying attributes, including the conditions specifying when and where a job should be run.

There are the two methods to create job definitions: Using the AutoSys Graphical User Interface (GUI). Using the AutoSys Job Information Language (JIL) through a command-line interface.

TCS GE Confidential
TCS GE Confidential

Autosys Job File Details

JOB Definition Name Starting Conditions Alarms Restart Conditions Current status Start time End time Exit code

DEPENDS ON

TIME

Current State

COMMAND Command to execute File to source Machine to run on Standard output files Standard input file

FILE WATCHER

CONTAINER

File Name Minimum File size Watch Interval

Box

Run Command on Machine

Watch for File

TCS GE Confidential
TCS GE Confidential

Box jobs are containers that hold other jobs. A box job can be used to organize and control process flow. The box itself performs no actions, although it can trigger other jobs to run. An important feature of this type of job is that boxes can be put inside of other boxes. Default Box Job Behavior: Some important rules to remember about boxes are Jobs run only once per box execution. Jobs in a box will start only if the box itself is running. As long as any job in a box is running, the box remains in RUNNING state; the box cannot complete until all jobs have run. By default, a box will return a status of SUCCESS only when all the jobs in the box have run and the status of all the jobs is "success. By default, a box will return a status of FAILURE only when all jobs in the box have run and the status of one or more of the jobs is "failure." Unless otherwise specified, a box will run indefinitely until it reaches a status of SUCCESS or FAILURE. Changing the state of a box to INACTIVE changes the state of all the jobs in the box to INACTIVE.

TCS GE Confidential
TCS GE Confidential

Defining Autosys Jobs AutoSys Graphical User Interface:

The AutoSys GUI allows you to interactively set the attributes that describe when, where, and how a job should run.

Job Information Language:

AutoSys JIL is a specification language, with its own syntax, that is used to describe when, where, and how a job should run. Job Information Language JIL) is a scripting language which provides a way to specify how AutoSys jobs should behave. JIL scripts contain one or more JIL sub-commands and one or more attribute statements; these elements constitute a job definition.
TCS GE Confidential
TCS GE Confidential

Various states in autosys:

ACTIVATED: The top-level box that this job is in is now in the RUNNING state, but the job itself has not started yet. STARTING: The event processor has initiated the start job procedure with the Remote Agent. RUNNING: The job is running. If the job is a box job, this value simply means that the jobs within the box may be started (other conditions permitting). If it is a command or file watcher job, the value means that the process is actually running on the remote machine. SUCCESS: The job exited with an exit code equal to or less than the maximum exit code for success. By default, only the exit code 0 is interpreted as success. If the job is a box job, this value means that all the jobs within the box have finished with the status SUCCESS (the default), or the Exit Condition for Box Success evaluated to true FAILURE: The job exited with an exit code greater than the maximum exit code for success. By default, any number greater than zero is interpreted as failure. AutoSys issues an alarm if a job fails TERMINATED: The job terminated while in the RUNNING state. A job can be terminated if a user sends a KILLJOB event or if it was defined to terminate if the box it is in failed. If the job itself fails, it has a FAILURE status, not a TERMINATED status. A job may also be terminated if it has exceeded the maximum run time (term_run_time attribute, if one was specified for the job), or if it was killed from the command line through a UNIX kill command. AutoSys issues an alarm if a job is terminated. INACTIVE: The job has not yet been processed. Either the job has never been run, or its status was intentionally altered to turn off its previous completion status

TCS GE Confidential
TCS GE Confidential

RESTART: The job was unable to start due to hardware or application problems, and has been scheduled to restart. QUE_WAIT: The job can logically run (that is, all the starting conditions have been met), but there are not enough machine resources available.

ON_HOLD: This job is on hold and will not be run until it receives the JOB_OFF_HOLD event. ON_ICE: This job is removed from all conditions and logic, but is still defined to AutoSys. Operationally, this condition is like deactivating the job. It will remain on ice until it receives the JOB_OFF_ICE event.

TCS GE Confidential
TCS GE Confidential

Status Abbreviations: The following table lists the abbreviations used in the ST (status) column of the autorep report, and gives the status for each abbreviation.

AC - ACTIVATED FA - FAILURE IN - INACTIVE OH - ON_HOLD OI - ON_ICE QU - QUE_WAIT RE - RESTART RU - RUNNING ST - STARTING SU - SUCCESS TE - TERMINATED

TCS GE Confidential
TCS GE Confidential

Note:
The difference between "on hold" and "on ice" is that when an "on hold" job is taken off hold, if its starting conditions are already satisfied, it will be scheduled to run, and it will run. On the other hand, if an "on ice" job is taken "off ice," it will not start, even if its starting conditions are already satisfied. This job will not run until its starting conditions reoccur. The other major distinction is that jobs downstream from the job that is "on ice" will run as though the job succeeded. Whereas, all dependent jobs do not run when a job is on "on hold"nothing downstream from this job will run.

Some basic Autosys Commands: This will display a list of jobs with complete details with box/jobname, last/latest run date & time, status, exit code, etc. autorep j <job_name> example: autorep j gpag0201f_orders_load_daily_inf

TCS GE Confidential
TCS GE Confidential

Viewing JIL code for any Autosys job: autorep j <job_name> -q example: autorep j gpag0201f_orders_load_daily_inf q To check the dependent jobs: job_depends -c -j <job_name> example: job_depends -c -j gpag0201f_orders_load_daily_inf To get all the jobs which are running at the time when this command is fired: autorep -j gp%inf| egrep "RU|FA To obtain the information of previous runs autorep j <job_name> -r -1 example: autorep -j gpag0201f_orders_load_daily_inf -r -1

TCS GE Confidential
TCS GE Confidential

SAMPLE JIL FILE

Sample JIL file saved as *.jil extension

TCS GE Confidential
TCS GE Confidential

STEPS TO RAISE A CASE FOR AUTOSYS RELATED ISSUE

The following steps is abt how to raise a request on kintana for your jobs(New/change/delete/One-Time)!! 1) Go to http://insidegis.ge.com 2) Login with your SSO Id / password 3) On the left hand side under 'Change Management', there is a link 'Modify a Service/Asset'. Clicking on this link refreshes and brings up a new page. 4) The first field is 'Service' which is a drop down field. Choose 'Server' as applicable to you. 5) Enter you contact information/Desired Start Time/Desired Completion Time. 6) Enter the 'Change Type' . Most of the cases it will be 'Standard'. 7) In 'Affected Scope', you need to enter the server system name(gcoasys01). 8) In Change Reasons, enter the type of request. a) Production Scheduling Request - New (For a new Job) b) Production Scheduling Request - Change (For changing a job parameter) c) Production Scheduling Request - Special One Time Process (For force starting/putting a job ON/OFF ICE/HOLD) d) Production Scheduling Request - Remove (For deleting a job) 9) You can also enter any free text in the 'Change Description' area. 10) Click on Submit and note down the ticket number.

TCS GE Confidential
TCS GE Confidential

Monitoring Autosys Jobs

Option I GUI Link For Autosys http://gcoasys02.corporate.ge.com/autosys/login Job Monitoring & Reports: GUI to Monitor the Jobs Reporting Through GUI Dashboard for Functional Users: In Progress

Option II Autosys Command Prompt Go to Remote Login Server > Start > Unicenter Autosys Cmd Prompt > Enter the command autorep j autosys jobname

TCS GE Confidential
TCS GE Confidential

FAQs

What if I don't want to run the ETL cycle for the given day? How do I delay a box? How do I delay the job within a box? How do I cancel a box? How do I cancel a job within a box? How do I run a failed/errored out job or box on the same day? How do re run any job/box on the same day?

TCS GE Confidential
TCS GE Confidential

How to contact Autosys team?

There are several ways to reach the Autosys team: 1. Here in this community you could enter a case and direct it to any expert that lists Autosys in their expertise but only for general questions. We would prefer that you use the procedure documented in Request for Service sub-category under the Process category in this community. See document there titled "How to Use RFC for Production Scheduling requests" to follow the directions to have a request ticket entered in our system. The document addresses Autosys job/schedule requests as either New, Change or Remove that go to the schedulers or "One Time" requests that go to the production monitors on duty. 2. If you have a problem, a question about Autosys or need to make a request, you follow a different path. Still using http://insidegis.ge.com/ you can select "Ask A Question" (to ask a question or make a request), select "Modify a Service/Asset" under "Change Management" (to request a remote agent install or license key) or select "Incident Management" to enter an issue. There's a new document in this community's Process section called "How to ask a question, report a problem or submit a non-schedule change request" that has all the details. See the attachment to this FAQ for some further details on submitting a request or reporting a problem. GIS Help Desk is also available via *222-6666. The InsideGIS portal lists the various ways to reach the help desk. Via e-mail there's a distribution list @CORP GIS Autosys Support that can be used to reach the entire team. However, once a request has been sent and you want to contact the Autosys schedulers to forward documents, etc., you can use the distribution list @CORP GIS Autosys Schedulers to reach them. For installs and license keys you can use the DL @CORP GIS Autosys Installers. 4) For Sametime access just take any of the DL's listed above, remove the "@" and that is a group name you can add to your list TCS GE Confidential
TCS GE Confidential

3.

 What is Autosys?

Limitations of BO DI Scheduler
Distinctive Features

Offerings by GIS
GIS Customers

TCS GE Confidential
TCS GE Confidential

 What is Autosys?

CA-Autosys is a batch monitoring and scheduling tool for the Open Systems environment.
It uses several supplemental products to provide a complete cross platform scheduling and monitoring solution for GIS.

TCS GE Confidential
TCS GE Confidential

Limitations of BO DI Scheduler:
Jobs can only be scheduled based on Day/Time Jobs can not be scheduled on the Success or Failure of Particular Job Lots of manual intervention is needed to complete the Data Warehouse Load Continuous Monitoring efforts required Longer Job Cycle Time due to Manual Intervention

TCS GE Confidential
TCS GE Confidential

 Distinctive Features of Autosys:

Increased Performance: Improved performance using multithreaded processing to enable more jobs to run in parallel.
No Manual Intervention: Due to scheduling capability based on Time/day, file arrival and job dependency, the entire loading process can be automated. Unparalleled Reliability and Scalability: Scales tens of thousands of jobs easily over hundreds of multiple platform machine per night, while maintaining its hallmark reliability.

TCS GE Confidential
TCS GE Confidential

 Distinctive Features of Autosys:

Enterprise-wide cross Platform Job Management: Simplifies cross platform job management through direct integration communication to Unicenter Job management Solutions

TCS GE Confidential
TCS GE Confidential

 Offerings of GIS for Autosys

High availability: Provides High Availability of Autosys Scheduling Server with fell over Servers (Cincinnati and Alpharetta) 24/7 Support Team: Provides 24 * 7 support team for monitoring of Autosys Jobs Web Based User interface: Provides single point of control and administration of jobs and reduces IT learning and operational costs No Additional Cost to GE Security: GIS offers the Autosys services with no additional cost of Hardware/Software/Services Job Scheduling Flexibility: Job scheduling and Monitoring flexibility is offered by GIS to the Application team. The application team may or may not choose the option for Scheduling and Monitoring the Jobs.

TCS GE Confidential
TCS GE Confidential

Pre-requisite for Autosys Scheduling:

Oracle SQL*Net V2.3 and Oracle TCP/IP Adapter are required Installer must have access to the server The TNS entry to be made for two Autosys Oracle databases Autosys Agent to be Installed on Server Getting Started: http://data.supportcentral.ge.com/upload/16285/doc_172780.doc

TCS GE Confidential
TCS GE Confidential

Types Of Autosys Jobs:

Command Job: This is what is commonly referred to as a job. The command can be a shell script, an executable program, a file transfer, etc. When the job runs, there is the execution of a specific command on the client machine. Box Job: The box job or box is a container of other jobs. It can be used to organize and control process flow. The box itself performs no actions, although it can trigger other jobs to run. File Watcher Job: A file watcher job starts a process that monitors for the existence and size of a specific file. Once that file reaches the specified minimum size, and is no longer growing in size for a specified time, the file watcher job completes successfully, indicating that the file has arrived.

TCS GE Confidential
TCS GE Confidential


1. 2. 3. 4. 5.

Process to Schedule Jobs through Autosys:

Identify the Jobs to be Schedule Decide upon the Dependency Logical grouping of the Jobs (Box) Creating JIL Files Raise a Request to GIS team to Schedule them OR

Get Access to the GUI tool to schedule them

TCS GE Confidential
TCS GE Confidential

Sensing Production Jil Files

TCS GE Confidential
TCS GE Confidential

Autosys Job Monitoring from DI server

Log in to Autosys Command Prompt Start->All Programs->Computer Associates ->Unicenter->Unicenter Autosys ->Unicenter Autosys Command Prompt

TCS GE Confidential
TCS GE Confidential

GUI Link For Autosys http://gcoasys02.corporate.ge.com/autosys/login Job Monitoring & Reports: GUI to Monitor the Jobs Reporting Through GUI Dashboard for Functional Users: In Progress

TCS GE Confidential
TCS GE Confidential

Gui Monitoring of Jobs

Various Views ( Prod/QA/Dev) with jobs under them

TCS GE Confidential
TCS GE Confidential

Various Boxes giving Status of Jobs in them ( Green Success, Yellow Ice, Red- Failure ) Boxes are used to bring together similar jobs

TCS GE Confidential
TCS GE Confidential

Jobs in Boxes can be seen by double clicking on Box which gives job details

TCS GE Confidential
TCS GE Confidential

Link to Create Ticket for Gui Account http://gis.kintana.corporate.ge.com/itg/web/knta/crt/RequestDetail.jsp?REQUEST_ID=5070271

TCS GE Confidential
TCS GE Confidential

To run Informatica job from Autosys we have two shell scripts 1) Normal job not having flat file load /inforprdsec/data/Scripts/exe_workflows.sh SEC_SC_PSI w_Sec_SC_PSI_Intransit_STG /inforprdsec/data/Scripts/exe_workflows.sh : -- The shell script that is called. Take two params Param 1 -- SEC_SC_PSI Folder Name Param 2 -- w_Sec_SC_PSI_Intransit_STG --Workflow Name 1) Job execution having flat file. /inforprdsec/data/Scripts/exe_workflow_flat_file.sh ILX_TMP_EMEA_OVERDUE_DEMAND.ipf SEC_EMEA_Miscellaneous w_SEC_EMEA_Miscellaneous_Tmp_Emea_Overdue_Demand /inforprdsec/data/Scripts/exe_workflow_flat_file.sh ---------- Shell script used Takes 3 param Param 1 --- ILX_TMP_EMEA_OVERDUE_DEMAND.ipf -Parameter File of the Job Param 2 -SEC_EMEA_Miscellaneous --Folder Param 3 -- w_SEC_EMEA_Miscellaneous_Tmp_Emea_Overdue_Demand -Work flow name What happens in the shell script on High level is Based on the parameter file name this select is run select FIELD02_VALUE_TXT ||'~'|| FTP_DIRECTORY_NAM ||'~'|| FTP_ARCHIVE_DIRECTORY_NAM || '~' || EMAIL_MISSING_FILES_NAM ||'~'|| FTP_FILE_EXTENSION_NAM ||'~'|| FIELD03_VALUE_TXT from tb_etl_ctl_parameters where PARAMETERFILE_NAME = '$PARAMETER_FILE_NAME';

TCS GE Confidential
TCS GE Confidential

Then this section FILE_PAT_TMP=`awk -F~ '{print $1}' $PARAM_TMP` Pattern Of the File SOURCE_DIR=`awk -F~ '{print $2}' $PARAM_TMP` Source Direcotry TARGET_DIR=`awk -F~ '{print $3}' $PARAM_TMP` FTP_ARCHIVE_DIRECTORY_NAM Target Directory MAIL_LIST=`awk -F~ '{print $4}' $PARAM_TMP` EMAIL_MISSING_FILES_NAM --- Mailing List FILE_EXTN=`awk -F~ '{print $5}' $PARAM_TMP` FTP_FILE_EXTENSION_NAM -- File extension FTP_FLAG=`awk -F~ '{print $6}' $PARAM_TMP` flag decides if FTP from DI is req or not If FTP from DI is req then following is run

--- FIELD02_VALUE_TXT ----- FTP_DIRECTORY_NAM ----------- FIELD03_VALUE_TXT FTP

${SEC_SCRIPTS}/ftp_get_flat_files.sh ${PARAMETER_FILE_NAME} "${FILE_PAT}" Shell Script Name Parameter File Name File Pattern This shell script basically opens FTP session Pulls file and does Archive of the file in DI server. After the FTP part is complete below is done # Loop through the files in the directory for CURR_FILE in `ls ${FILE_PAT} 2>/dev/null` --- Loops thru the file list to be loaded do FCNT=`expr $FCNT + 1` --- Takes care Count

TCS GE Confidential
TCS GE Confidential

------------- Below part updates the Parameter Table with File name and file count ${SQLPLUS} -s $DBUSER/$DBPASS@$DBNAME <<EOF > ${ORA_OUT} ---- All these $dbuser things come from Env_Var_Config.sh WHENEVER SQLERROR EXIT 1; WHENEVER OSERROR EXIT 1; update tb_etl_ctl_parameters set FIELD04_VALUE_TXT = '$CURR_FILE' , FIELD05_VALUE_TXT = '$FCNT' where PARAMETERFILE_NAME = '$PARAMETER_FILE_NAME'; commit; EXIT; EOF And then runs the Informatica Workflow # Call the workflow to execute the job echo "Invoking the workflow ${INFORMATICA_FOLDER}.${WORKFLOW_NAME}" >> ${LOG_FILE} $SEC_SCRIPTS/exe_workflows.sh $INFORMATICA_FOLDER $WORKFLOW_NAME >> ${LOG_FILE}

TCS GE Confidential
TCS GE Confidential