> For the complete documentation index, see [llms.txt](https://docs.ingestlabs.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ingestlabs.com/media-data-platform/pipelines/cdp-pipeline-overview-of-pre-defined-filters-in-destinations.md).
# CDP Pipeline: Overview of Pre-Defined Filters in Destinations
### Introduction
This feature helps you find and remove duplicate records in your **Destination**. This functionality is available in pipeline.
### Follow These Steps to Remove Duplicate Records from Your Pipeline:
**Step 1:** In the navigation panel, click **Pipeline**.
**Step 2:** A list of pipelines appears. Click the pipeline you wish to open.
**Step 3:** Click **Destinations**.
**Step 4:** Click on **Add Destination**.
**Step 5:** The Destinations panel opens on the right side. Click **Select Destination.**
**Step 6:** The Select Destination panel opens. Use the search bar to find a destination, select one from the list, or create a new destination by clicking Create Destination.
**Step 7:** Click on Filter.
**Step 8:** Check the **Enable Filter** box.
**Step 8:** When you check the box, the Function tab will open.
Under the Type dropdown, select **Pre Defined Fn**.
**Predefined Functions:** Built-in system functions with standard logic that require no customization.
**Step 8.1:** In the **Select Function Typ**e dropdown, select **Event Deduplication**.
**Step 8.2:** **Additional Arguments** will open. In the **Event Namespace**, select the value from the corresponding **Value** dropdown, or you can enter a value manually.
In the **Deduplication ID**, select the appropriate value from the corresponding **Value** dropdown, or you can enter a value manually.
**Step 9:** Click **Custom Fn**
**Custom Functions:** User-defined functions that allow customization based on specific requirements.
**Step 9.1**: Function Arguments: To configure function arguments, follow these steps:
**Step: 9.2: Enter a Name** in the **Name** field to define the function argument.
**Step: 9.3:** Click on the **Data Source Field** tab. A list of available data source fields appears on the right side. Select the appropriate **Data Source Field** from the list.
**Step 9.4**: Code
In this tab, you can write custom code for the function. After writing the code, click the **Test Function** button to execute and validate the function.
**Step 10:** Click **Save**.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.ingestlabs.com/media-data-platform/pipelines/cdp-pipeline-overview-of-pre-defined-filters-in-destinations.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.