ExecuteProfilerJob

Jobflow Component

Short Description
Ports
ExecuteProfilerJob Attributes
Details
Best Practices
See also

Short Description

ExecuteProfilerJob allows running of Profiler Jobs with user-specified settings and provides execution results to output ports.

[Note]Note

To be able to use this component, you need a license with Profiler and Jobflow. Also, the component requires your project is executed on CloverETL Server.

Component Same input metadata Sorted inputs Inputs Outputs Each to all outputs Java CTL Auto-propagated metadata
ExecuteProfilerJob
no
no
0-10-2
yes
no
yes
yes

Icon

Ports

Please refer to ExecuteGraph Ports.

ExecuteProfilerJob Attributes

Please refer to ExecuteGraph Attributes for the description of attributes. Compared to ExecuteGraph, the ExecuteProfilerJob component is missing the attributes Execution group and Skip checkConfig.

Also, the Input mapping and Output mapping attributes offer slightly different configuration, specific to Profiler Jobs.

Details

Input mapping
Output mapping

This component works similarly to ExecuteGraph. See ExecuteGraph component documentation. See the section ExecuteProfilerJob Attributes for the list of main differences between these two components.

Input mapping

The Input mapping attribute allows to override the settings of the component based on the data from the incoming token. Moreover, job parameters of the executed profiler job can be changed in the input mapping.

Input mapping is a regular CTL transformation which is executed before each profiler job execution. Input token, if any, is the only input for this mapping and outputs of the transformation are up to two records: RunConf and Parameters.

  • RunConf record represents execution settings. If a field of the record is not populated by this mapping, a default value from respective attribute of the component is used instead.

    Field NameTypeDescription
    jobURLstring Overrides component attribute Profiler Job URL.
    sourcestring Overrides the profiled data source. In case a file or an XLS spreadsheet is profiled, this will change which file is profiled. In case of a DB table job, this will override the table from which the data is obtained.
    charsetstring Overrides the input encoding (charset) of the profiled data for file and XLS profiler jobs.
    executionTypestring Overrides the component attribute Execution type.
    timeoutlong Overrides the component attribute Timeout.
    clusterNodeIdstring Overrides the component attribute Cluster node ID.
    daemonboolean Overrides the component attribute Execute profiler job as daemon.

  • Parameters record represents all external profiler job parameters of the triggered profiler job.

[Note]Note

The Parameters record is available in the transform dialog only if the component attribute Profiler Job URL links to an existing profiler job which is used as a template for extraction of the parameters structure. Only parameters from this profiler job can be populated by input mapping, no matter which profiler job will be actually executed in runtime.

Output mapping

Output mapping is regular CTL transformation which is used to populate token passed to the first output port. The mapping is executed for successful profiler job executions. Up to four input data records are available for this mapping. If output mapping is empty, fields of RunStatus record are mapped to output by name.

  • The input RunConf token based on which the profiler job was executed is not available for component usage without input connector. This is very helpful for passing through some fields from input token to output token. This record has Port 0 displayed in the Type column.

  • RunStatus record provides information about profiler job execution.

    Field NameTypeDescription
    runIdlongUnique identifier of the profiler job run.
    orignalJobURLstringPath to executed profiler job.
    startTimedateTime of the job execution.
    endTimedateTime of job finish, or null for asynchronous execution.
    durationlongJob execution time in milliseconds.
    statusstringFinal job execution status (FINISHED_OK | ERROR | ABORTED | TIMEOUT | RUNNING for asynchronous execution).
    errExceptionstringCause an exception for failed jobs only.
    errMessagestringError message for failed jobs only.

  • RunInfo provides additional information about the job execution, specific to profiler jobs.

    Field NameTypeDescription
    inputRecordCountlongNumber of profiled records.
    rejectedRecordCountlongNumber of records rejected from profiling, e.g. due to parse errors.

  • RunResults record provides profiling results - output values of metrics enabled on profiled fields. The results of profiling will be available only in the case of Synchronous execution and only if the current user has sandbox privileges to read the profiling results.

    Metrics with structured results return values as Multivalue Fields. This includes charts, format count metrics, etc.

Best Practices

[Tip]Tip

If you drag a *.cpj file and drop it into the jobflow pane, you will add the ExecuteProfilerJob component.

See also

ExecuteGraph
ExecuteJobflow
ExecuteMapReduce
Common Properties of Components
Specific Attribute Types
Common Properties of Job Control
Job control Comparison