DeDuplication+Filter Details

Element Details: DeDuplication+Filter

Available in: Logi Info

Source code name: DeDuplicationFilter

This filter can be used to remove duplicate records from the datalayer. The value of the ColumnNames attribute will be used to determine unique records. The filter will keep the first record it encounters.

If the ExceptionDataID attribute has a value, the removed records can be used in additional processing. If the ExceptionDataID does not have a value, then the records are simply deleted from the returned datalayer.

ATTRIBUTES

Click attribute Name to drill down for more information.

Name

Possible Values

Description

ColumnNames

UserDefined

(Required) A list of column names, separated by commas. For example: OrderID, CustomerID, EmployeeID, OrderDate

ExceptionDataID

UserDefined

Defines an ETL ID value that can be used to address the exception data that was removed by the filter. The value of this attribute may be referenced by a DataLayer.ETL, LoadDbTable, SaveData, or any other element that supports the EtlElementID attribute.

UserDefined

(Required) The ID attribute is a pervasive attribute that uniquely identifies an element within a definition file. The ID needs to be a unique value within the definition.

PARENT ELEMENTS

Click element to drill down for more information.

CHILD ELEMENTS

Click element to drill down for more information.