Documente Academic
Documente Profesional
Documente Cultură
Creation of psets.
Capturing job statistics details in the EME when using generic graphs.
02/12/15
02/12/15
02/12/15
02/12/15
02/12/15
02/12/15
02/12/15
Each separate input values set you create in this step represents a separate instance
of the graph. To enable the Job Tracking of the generic graph, for different such value
sets, simply check these .pset files with different input value sets into the EME data
store.
This graph instance represented by the .pset file is analyzed and saved in the EME
data store as a graph object. For .pset file to be analyzed set analysis_level
parameters in each parameter set to expand. This was mandatory in Abinitio V-13.
NOTE: Abinitio V-14 automatically expands the psets when they are checked in.
02/12/15
Distinct values of logical EME datasets are passed from different psets to the same
generic graph. This is done to achieve data lineage. When psets are checked in they
are expanded and dependency analysis takes place. Different instances of the
generic graph will show up in EME with unique values of logical datasets.
02/12/15
As above different data lineage are achieved in two instances of the same graph in EME.
02/12/15
Physical dataset names overwrites the logical EME dataset names passed
from psets. Physical dataset names are set and then passed while
executing the graph from within the wrapper via pset.
For e.g. exporting physical datasets
02/12/15
02/12/15
Capturing job statistics details in the EME when using generic graphs
AB_AIR_JOB_GRAPH
Specifies the graph/application being run so that it may be linked to the job object.
-
When a generic graph is called the job statistics are stored in the EME under the name of
the generic graph. This causes confusion and discrepancies when tracking stats in EME
because a generic graph may be used in multiple projects. The objective is to store job
statistics under the pset name so that they can be correlated with the logical use of the
generic graph.
- This parameter needs to be set in the calling script/program to have a generic graph reposit
tracking to the .graph (pset version) of the graph.
-
If the graph is generic then you should set AB_AIR_JOB_GRAPH because you want the
job to be associated with pset instance of the graph which does the specific task according
to values passed through pset.
02/12/15
Benefits
Job statistics will be reposited with the logical use of the graph
The statistics will be accurately reported by the appropriate job group or project
Performance improvement in graph execution time.
02/12/15
/opt/abinitio/abinitio-V2-15-5-0/doc/EME_Developer_Guide.pdf
/opt/abinitio/abinitio-V2-15-5-0/doc/EME_Reference.pdf
02/12/15
THANK YOU
02/12/15