Defining Transformations
For basic information about transformations see Chapter 31, Transformations.
Here we will explain how you should create transformations that change the data flowing through some components.
For brief table overview of transformations see Transformations Overview.
Below we can learn the following:
What components allow transformations.
What language can be used to write transformations.
Whether definition can be internal or external.
What the return values of transformations are.
What can be done when error occurs.
The Transform editor and how to work with it.
What interfaces are common for many of the transformation-allowing components.
Components Allowing Transformation
The transformations can be defined in the following components:
DataGenerator, Reformat, and Rollup
These components require a transformation.
You can define the transformation in Java or Clover transformation language.
In these components, different data records can be sent out through different output ports using return values of the transformation.
In order to send different records to different output ports, you must both create some mapping of the record to the corresponding output port and return the corresponding integer value.
Partition, or ClusterPartitioner
In the Partition, or ClusterPartitioner component, transformation is optional. It is required only if neither the Ranges nor the Partition key attributes are defined.
You can define the transformation in Java or Clover transformation language.
In Partition, different data records can be sent out through different output ports using return values of the transformation.
In ClusterPartitioner, different data records can also be sent out to different Cluster nodes (through virtual output ports) using return values of the transformation.
In order to send different records to different output ports or Cluster nodes, you must return corresponding integer value. But no mapping need to be written in this component since all of the records are sent out automatically.
DataIntersection, Denormalizer, Normalizer, Pivot, ApproximativeJoin, ExtHashJoin, ExtMergeJoin, LookupJoin, DBJoin, and RelationalJoin
These components require a transformation.
You can define the transformation in Java or Clover transformation language.
In Pivot, transformation can be defined setting one of the Key or Group size attributes. Writing it in Java or CTL is still possible.
MultiLevelReader and JavaExecute
These components require a transformation.
You can only write it in Java.
JMSReader and JMSWriter
In these components, transformation is optional.
If any is defined, it must be written in Java.
Java or CTL
Transformations can be written in Java or Clover transformation language (CTL):
Java can be used in all components.
Transformations executed in Java are faster than those written in CTL. Transformation can always be written in Java.
CTL cannot be used in JMSReader, JMSWriter, JavaExecute, and MultiLevelReader.
Nevertheless, CTL is very simple scripting language that can be used in most of the transforming components. Even people who do not know Java are able to use CTL. CTL does not require any Java knowledge.
Internal or External Definition
Each transformation can be defined as internal or external:
Internal transformation:
An attribute like Transform, Denormalize, etc. must be defined.
In such a case, the piece of code is written directly in the graph and can be seen in it.
External transformation:
One of the following two kinds of attributes may be defined:
Transform URL, Denormalize URL, etc., for both Java and CTL
The code is written in an external file. Also charset of such external file can be specified (Transform source charset, Denormalize source charset, etc.).
For transformations written in Java, folder with transformation source code need to be specified as source for Java compiler so that the transformation may be executed successfully.
Transform class, Denormalize class, etc.
It is a compiled Java class.
The class must be in classpath so that the transformation may be executed successfully.
Here we provide a brief overview:
Transform, Denormalize, etc.
To define a transformation in the graph itself, you must use the Transform editor (or the Edit value dialog in case of JMSReader, JMSWriter and JavaExecute components). In them you can define a transformation located and visible in the graph itself. The languages which can be used for writing transformation have been mentioned above (Java or CTL).
For more detailed information about the editor or the dialog see Transform Editor or Edit Value Dialog.
Transform URL, Denormalize URL, etc.
You can also use a transformation defined in some source file outside the graph. To locate the transformation source file, use the URL File Dialog. Each of the mentioned components can use this transformation definition. This file must contain the definition of the transformation written in either Java or CTL. In this case, transformation is located outside the graph.
For more detailed information see URL File Dialog.
Transform class, Denormalize class, etc.
In all transforming components, you can use some compiled transformation class. To do that, use the Open Type wizard. In this case, transformation is located outside the graph.
See Open Type Dialog for more detailed information.
More details about how you should define the transformations can be found in the sections concerning corresponding components. Both transformation functions (required and optional) of CTL templates and Java interfaces are described there.
Here we present a brief table with an overview of transformation-allowing components:
Table 37.1. Transformations Overview
Legend
1): If this is yes, each data record is
always sent out through all connected output ports.
2): If this is yes, each data record can be
sent out through the connected output port whose number is returned by
the transformation. See Return Values of Transformations for more
information.