Search code examples
zapier

How to apply deduplication for an array returned in Zapier Code output

I have a Zapier Code block that does fetch for JSON array and the preprocess this data. I cannot use Zapier Webhook with polling, because I need to process the data a bit.

Zapier Webhook offers deduplication feature, by having id parameter associated with the items returned in an array from the URL endpoint. How can I achieve the same for Zapier Code? Currently, my zap is trying to process and trigger on the same data twice. This leads to the error that Zapier tries to send out the same tweet twice, every time the Code is triggered.

Here is mock data returned by my Code:

output = [{id: 1, name: "foo"}, {id: 2, name: "bar"}]

Currently, without deduplication, I am getting this email and having my zap disabled:

Your Zap named XXX was just stopped. This happened because our systems detected this Zap posted a duplicate tweet, which is against Twitter's Terms Of Service.

Solution

  • You can use storage by Zapier to achieve this. the ideal flow will be :

    1. Trigger
    2. Storage by Zapier [Get Value (use storage key = lastItemId) ]
    3. Code By Zapier (Filter array return only those record that has id greater than the lastItemId)
    4. Storage By Zapier (Set Value) : update lastItemId with the last item processed by Code By Zapier step

    You can also use StoreClient in place of the Storage By zapier, but always update a existing key lastItemId and compare id of the record with ```lastItemId`` and at the end update StoreCLient key (lastItemId)