Chapter 19. Engine Configuration

CloverETL internal settings (defaults) are stored in the defaultProperties file located in the CloverETL engine. This source file contains various parameters that are loaded at run-time and used during transformation execution. We do not recommend changing values in this file.

In Designer, the path to the file is plugins/com.cloveretl.gui/lib/lib/cloveretl.engine.jar. In Server Core, the path to the file is WEB-INF/lib/cloveretl.engine.jar.

If you need to change the default setting, create a local file with only those properties you need to override and place the file in the project directory. To instruct CloverETL to retrieve the properties from this local file, go to WindowPreferencesCloverETLETL Runtime and either define the path to the file in the CloverETL Engine Properties field or put the following parameter in the VM parameters field:

-Dclover.engine.config.file=/full/path/to/file.properties

Content of defaultProperties file

Here we present some of the properties and their values as they are presented in the defaultProperties file:

  • Record.RECORD_LIMIT_SIZE = 33554432

    Limits the maximum size of a record. Theoretically, the limit is tens of MBs, but you should keep it as low as possible for an easier error detection. For more details on memory demands, see Edge Memory Allocation.

  • Record.RECORD_INITIAL_SIZE = 65536

    Sets the initial amount of memory allocated to each record. The memory can grow dynamically up to Record.RECORD_LIMIT_SIZE, depending on how memory-greedy an edge is. See Edge Memory Allocation.

  • Record.FIELD_LIMIT_SIZE = 33554432

    Limits the maximum size of one field within a record. For more details on memory demands, see Edge Memory Allocation.

  • Record.FIELD_INITIAL_SIZE = 65536

    Sets the initial amount of memory allocated to each field within a record. The memory can grow dynamically up to Record.FIELD_LIMIT_SIZE, depending on how memory-greedy an edge is. See Edge Memory Allocation.

  • Record.DEFAULT_COMPRESSION_LEVEL = 5

    This sets the compression level for compressed data fields (cbyte).

  • DEFAULT_INTERNAL_IO_BUFFER_SIZE = 32768

    Determines the internal buffer size the components allocate for I/O operations. Increasing this value affects performance negligibly.

  • USE_DIRECT_MEMORY = false

    The clover engine can use direct memory for data records manipulation. For example, underlying memory of CloverBuffer (container for serialized data records) uses direct memory (if the usage is enabled). This attribute is by default false.

    Using direct memory can slightly improve performance in some cases. However, direct memory is out of control of a java virtual machine, as the direct memory is allocated outside of the Java heap space in direct memory. If OutOfMemory exception occurs and usage of direct memory is enabled, try to turn it off.

    In CloverETL 4.9.0M2, the default value was changed from true to false.

  • DEFAULT_DATE_FORMAT = yyyy-MM-dd

  • DEFAULT_TIME_FORMAT = HH:mm:ss

  • DEFAULT_DATETIME_FORMAT = yyyy-MM-dd HH:mm:ss

  • DEFAULT_REGEXP_TRUE_STRING = true|T|TRUE|YES|Y|t|1|yes|y

  • DEFAULT_REGEXP_FALSE_STRING = false|F|FALSE|NO|N|f|0|no|n

  • DataParser.DEFAULT_CHARSET_DECODER = UTF-8

  • DataFormatter.DEFAULT_CHARSET_ENCODER = UTF-8

  • Lookup.LOOKUP_INITIAL_CAPACITY = 512

    The initial capacity of a lookup table when created without specifying the size.

  • DataFieldMetadata.DECIMAL_LENGTH = 12

    Determines the default maximum precision of decimal data field metadata. Precision is the number of digits in a number, e.g. the number 123.45 has a precision of 5.

  • DataFieldMetadata.DECIMAL_SCALE = 2

    Determines the default scale of decimal data field metadata. Scale is the number of digits to the right of the decimal point in a number, e.g. the number 123.45 has a scale of 2.

  • Record.MAX_RECORD_SIZE = 33554432

    [Note]Note

    This is a deprecated property. Nowadays, you should use Record.RECORD_LIMIT_SIZE.

    Limits the maximum size of a record. Theoretically, the limit is tens of MBs, but you should keep it as low as possible for an easier error detection.

[Important]Important

You can define locale that should be used as the default one.

The setting is the following:

# DEFAULT_LOCALE = en.US

By default, system locale is used by CloverETL. If you uncomment this row you can set the DEFAULT_LOCALE property to any locale supported by CloverETL, see the List of all Locale

Similarly, the default time zone can be overridden by uncommenting the following entry:

# DEFAULT_TIME_ZONE = 'java:America/Chicago';'joda:America/Chicago'

For more information about time zones, see the Time Zone section.

[Note]Compatibility

In 4.4.0-M2, the default encoding was changed from ISO-8859-1 to UTF-8. Therefore, DataParser.DEFAULT_CHARSET_DECODER and DataFormatter.DEFAULT_CHARSET_ENCODER were set to UTF-8.