Ensuring Reliable Data Transfers: Retry Configuration

Introduction

In data workflows, failures can occur due to network interruptions, server downtimes, or credential mismatches. To address such failures, a retry mechanism is essential—if something fails, the system must automatically retry to ensure successful data delivery.

Configuring Retry Parameters

Step 1: In the navigation panel, click Pipeline.

Step 2: A list of pipelines appears. Click the pipeline you wish to open.

Step 3: Click Destinations.

Step 4: under the Type column, you can see different types of databases. These indicate the data storage systems where data from the server will be pushed to the destination.

If one of the destinations fails, for example, while transferring data to two destinations, the system will automatically retry the operation for the failed destination until the transfer is successful.

Step 5: Click the Settings button to configure the retry mechanism for destinations.

Step 6: Click Retry Configuration to access the retry settings.

Step 7: Click the Edit button to modify the retry parameters.

Step 8: Check the Enable checkbox.

Step 9: Under Retry After, you will see a dropdown with options such as Seconds, Minutes, and Hours. You can enter a number in the Retry After field to specify the retry interval. Then, choose whether you want the system to retry after a specific number of seconds, minutes, or hours.

For example, if you enter 2 in the Retry After field and select Hours, the system will retry the operation every 2 hours until it succeeds.

Step 10: Under Maximum Attempts, you can specify the maximum number of retry attempts. The system will retry the operation until the specified number of attempts is reached, after which it will stop retrying, even if the operation has not succeeded.

Step 11: Click the Save button to save your retry configuration settings.