CDP Pipeline: Overview of Pre-Defined Filters in Destinations

Introduction

This feature helps you find and remove duplicate records in your Destination. This functionality is available in pipeline.

Follow These Steps to Remove Duplicate Records from Your Pipeline:

Step 1: In the navigation panel, click Pipeline.

Step 2: A list of pipelines appears. Click the pipeline you wish to open.

Step 3: Click Destinations.

Step 4: A list of Destinations will appear, and on the right side of each Destination, you'll find a settings option. Click Settings.

Step 5: When you click on Settings, the Destination Schema panel will open. Click Filters.

Step 6: Click the Edit button to modify the Filters.

Step 7: Check the Enable Filter box.

Step 8: When you check the box, the Function tab will open.

Under the Type dropdown, select Pre Defined Fn.

Predefined Functions: Built-in system functions with standard logic that require no customization.

Step 8.1: In the Select Function Type dropdown, select Event Deduplication.

Step 8.2: Additional Arguments will open. In the Event Namespace, select the value from the corresponding Value dropdown, or you can enter a value manually.

In the Deduplication ID, select the appropriate value from the corresponding Value dropdown, or you can enter a value manually.

Step 9: Click Custom Fn

Custom Functions: User-defined functions that allow customization based on specific requirements.

Step 9.1: Function Arguments: To configure function arguments, follow these steps:

Step: 9.2: Enter a Name in the Name field to define the function argument.

Step: 9.3: Click on the Data Source Field tab. A list of available data source fields appears on the right side. Select the appropriate Data Source Field from the list.

Step 9.4: Code

In this tab, you can write custom code for the function. After writing the code, click the Test Function button to execute and validate the function.

Step 10: Click Save.