# CDP Pipeline: Overview of Pre-Defined Filters in Destinations ### Introduction This feature helps you find and remove duplicate records in your **Destination**. This functionality is available in pipeline. ### Follow These Steps to Remove Duplicate Records from Your Pipeline: **Step 1:** In the navigation panel, click **Pipeline**.

**Step 2:** A list of pipelines appears. Click the pipeline you wish to open.

**Step 3:** Click **Destinations**.

**Step 4:** Click on **Add Destination**.

**Step 5:** The Destinations panel opens on the right side. Click **Select Destination.**

**Step 6:** The Select Destination panel opens. Use the search bar to find a destination, select one from the list, or create a new destination by clicking Create Destination.

**Step 7:** Click on Filter.

**Step 8:** Check the **Enable Filter** box.

**Step 8:** When you check the box, the Function tab will open. Under the Type dropdown, select **Pre Defined Fn**. **Predefined Functions:** Built-in system functions with standard logic that require no customization.

**Step 8.1:** In the **Select Function Typ**e dropdown, select **Event Deduplication**.

**Step 8.2:** **Additional Arguments** will open. In the **Event Namespace**, select the value from the corresponding **Value** dropdown, or you can enter a value manually. In the **Deduplication ID**, select the appropriate value from the corresponding **Value** dropdown, or you can enter a value manually.

**Step 9:** Click **Custom Fn** **Custom Functions:** User-defined functions that allow customization based on specific requirements. **Step 9.1**: Function Arguments: To configure function arguments, follow these steps: **Step: 9.2: Enter a Name** in the **Name** field to define the function argument.

**Step: 9.3:** Click on the **Data Source Field** tab. A list of available data source fields appears on the right side. Select the appropriate **Data Source Field** from the list.

**Step 9.4**: Code In this tab, you can write custom code for the function. After writing the code, click the **Test Function** button to execute and validate the function.

**Step 10:** Click **Save**. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.ingestlabs.com/media-data-platform/pipelines/cdp-pipeline-overview-of-pre-defined-filters-in-destinations.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.