Selecting Input Records

Some readers allow you to limit the number of records that should be read. You can set up

The last two constraints can be defined only in readers that allow reading more files at the same time. In these Readers, you can define the records that should be read for each input file separately and for all of the input files in total.

Per Reader Configuration

Maximum number of records to be read is defined by the Max number of record attribute. The number of records to be skipped is defined by the Number of skipped records attribute. The records are skipped and/or read continuously throughout all input files. The records are skipped and/or read independently of the values of per source file attributes.

Per Source File Configuration

In some components, you can also specify how many records should be skipped and/or read from each input file. To do this, set up the following two attributes: Number of skipped records per source and/or Max number of records per source.

Combination of per File and per Reader Configuration

If you set up both: per file and per reader limits; firstly, the per file limits are applied. Secondly, the per reader limits are applied.

For example, there are two files:

1 | Alice
2 | Bob
3 | Carolina
4 | Daniel
5 | Eve
6 | Filip
7 | Gina
8 | Henry
9 | Isac
10| Jane

And the reader has the following configuration:

AttributeValue
Number of skipped records2
Max number of record5
Number of skipped records per source1
Max number of records per source3

The file are read in the following way:

  1. From each file, the first record is skipped and the next three records are read.

  2. The first two records of the records read in the previous step are skipped. The following records (at most four) are sent to the output.

The record read by the reader are

4 | Daniel
7 | Gina
8 | Henry
9 | Isac

The example shows that you can read even less records than is the number specified in the Max number of records attribute.

The total number of records that are skipped equals to Number of skipped records per source multiplied by the number of source files plus Number of skipped records.

And total number of records that are read equals to Max number of records per source multiplied by the number of source files plus Max number of records.

The Readers that allow limiting the records for both individual input file and all input files in total are:

The following two Readers allow you to limit the total number of records by using the Number of skipped mappings and/or Max number of mappings attributes. What is called mapping here, is a subtree which should be mapped and sent out through the output ports.

The following Readers allow limiting the numbers in a different way:

The following Readers do not allow limiting the number of records that should be read (they read them all):