Webhooks are delivered at-least-once, not exactly-once. Tolinku retries failed deliveries (3 retries at 1 minute, 5 minutes, and 30 minutes). If your receiver timed out but actually processed the event, the retry delivers the same event again. If your handler isn't idempotent, you'll double-count analytics, send duplicate emails, or create duplicate CRM records.
This guide covers how to make your webhook handlers idempotent so that processing the same event multiple times produces the same result as processing it once. For the retry logic details, see the webhook retry logic guide. For payload structure, see the webhook payload formats guide.
The webhooks page with create form, webhook list, and delivery log.
Why Duplicates Happen
There are several scenarios that lead to duplicate webhook deliveries:
- Timeout retries. Your receiver processed the event but took longer than 10 seconds to respond. Tolinku times out and retries.
- Network issues. Your 200 response was lost in transit. From Tolinku's perspective, the delivery failed.
- Receiver restarts. Your server restarted between receiving the request and responding. Tolinku retries.
- Infrastructure retries. Load balancers, proxies, or API gateways may retry requests on their own.
You cannot prevent duplicates. You can only handle them.
Generating a Deduplication Key
Tolinku's webhook payload doesn't include a delivery ID in the body. The payload contains event, timestamp, and data. To deduplicate, generate a deterministic hash from the payload content:
import crypto from 'crypto';
function getEventHash(rawBody: Buffer): string {
return crypto
.createHash('sha256')
.update(rawBody)
.digest('hex');
}
This hash is deterministic: the same payload always produces the same hash. Retries send the exact same payload, so they produce the same hash.
Use the raw body (the Buffer from express.raw()), not a re-serialized version. JSON serialization doesn't guarantee key ordering, so JSON.stringify(JSON.parse(body)) might produce a different string than the original body.
Pattern 1: Database Deduplication
The most reliable approach. Use a database table with a unique constraint on the event hash.
PostgreSQL
CREATE TABLE webhook_events (
event_hash VARCHAR(64) PRIMARY KEY,
event_type VARCHAR(50) NOT NULL,
event_timestamp TIMESTAMPTZ NOT NULL,
data JSONB NOT NULL,
processed_at TIMESTAMPTZ DEFAULT NOW()
);
async function processWebhook(rawBody: Buffer) {
const hash = getEventHash(rawBody);
const event = JSON.parse(rawBody.toString());
try {
await db.query(
`INSERT INTO webhook_events (event_hash, event_type, event_timestamp, data)
VALUES ($1, $2, $3, $4)
ON CONFLICT (event_hash) DO NOTHING`,
[hash, event.event, event.timestamp, JSON.stringify(event.data)]
);
// Check if the insert actually happened
const result = await db.query(
'SELECT processed_at FROM webhook_events WHERE event_hash = $1',
[hash]
);
// If processed_at is very recent (within 1 second), this is the first insert
const processedAt = new Date(result.rows[0].processed_at);
if (Date.now() - processedAt.getTime() < 1000) {
// First time seeing this event; process it
await handleEvent(event);
}
} catch (err) {
console.error('Webhook processing error:', err);
}
}
The ON CONFLICT DO NOTHING clause means the insert silently succeeds the first time and does nothing on duplicates. The database enforces uniqueness, so even if two instances of your receiver process the same event simultaneously, only one record is created.
SQLite
CREATE TABLE webhook_events (
event_hash TEXT PRIMARY KEY,
event_type TEXT NOT NULL,
processed_at TEXT DEFAULT (datetime('now'))
);
const stmt = db.prepare(
'INSERT OR IGNORE INTO webhook_events (event_hash, event_type) VALUES (?, ?)'
);
function processWebhook(rawBody: Buffer, event: any): boolean {
const hash = getEventHash(rawBody);
const result = stmt.run(hash, event.event);
return result.changes > 0; // true if this is a new event
}
Pattern 2: Redis Deduplication
Faster than a database lookup and works well for high-volume webhooks. Use Redis SET with NX (only set if not exists) and an expiry.
import Redis from 'ioredis';
const redis = new Redis(process.env.REDIS_URL!);
const DEDUP_TTL = 3600; // 1 hour
async function isNewEvent(rawBody: Buffer): Promise<boolean> {
const hash = getEventHash(rawBody);
const key = `webhook:dedup:${hash}`;
// SET NX returns 'OK' if the key was set (new event), null if it already existed
const result = await redis.set(key, '1', 'EX', DEDUP_TTL, 'NX');
return result === 'OK';
}
// Usage
app.post('/webhooks/tolinku', async (req, res) => {
// Verify signature...
res.status(200).send('OK');
if (await isNewEvent(req.body)) {
const event = JSON.parse(req.body.toString());
await handleEvent(event);
}
});
The TTL should be longer than the maximum retry window. Tolinku retries at 1 minute, 5 minutes, and 30 minutes, so the last retry arrives at most ~36 minutes after the first delivery. A 1-hour TTL covers this with margin.
Tradeoff: If Redis is down, you lose deduplication. For critical paths (financial transactions, email sends), use database deduplication as the primary mechanism and Redis as a fast-path optimization.
Pattern 3: In-Memory Deduplication
For low-volume integrations or ephemeral environments (Lambda functions), an in-memory set works if you accept some limitations.
const seen = new Set<string>();
const MAX_SIZE = 10_000;
function isNewEvent(rawBody: Buffer): boolean {
const hash = getEventHash(rawBody).substring(0, 16); // Shorter hash for memory
if (seen.has(hash)) return false;
seen.add(hash);
// Prevent unbounded memory growth
if (seen.size > MAX_SIZE) {
const iterator = seen.values();
for (let i = 0; i < MAX_SIZE / 2; i++) {
seen.delete(iterator.next().value!);
}
}
return true;
}
Limitations: Memory is lost on restart, so retries after a restart will be processed as new events. In a multi-instance deployment, each instance has its own set, so the same event delivered to different instances will be processed twice.
Use this only when the consequences of occasional duplicates are minor (e.g., logging to a data warehouse that handles dedup downstream).
Idempotent Operations
Some operations are naturally idempotent and don't need explicit deduplication:
Database upserts. If your handler creates or updates a record by a unique key, the operation is idempotent by nature:
-- Running this twice produces the same result
INSERT INTO installations (link_token, platform, installed_at)
VALUES ('summer-sale', 'ios', '2026-05-24T10:00:00Z')
ON CONFLICT (link_token) DO UPDATE SET installed_at = EXCLUDED.installed_at;
Counter increments are NOT idempotent. UPDATE campaigns SET click_count = click_count + 1 will double-count on duplicate deliveries. Either deduplicate before incrementing, or track individual events and compute counts from them.
Email sends are NOT idempotent. Sending the same email twice is a visible user experience problem. Always deduplicate before triggering emails.
CRM record creation may or may not be idempotent. It depends on whether the CRM API supports upsert. HubSpot and Salesforce do; some others don't.
Combining Deduplication with Processing
Here's a complete pattern that uses Redis for fast dedup and a database for durable storage:
app.post('/webhooks/tolinku', async (req, res) => {
// Verify signature
const signature = req.headers['x-webhook-signature'] as string;
const expected = crypto
.createHmac('sha256', process.env.WEBHOOK_SECRET!)
.update(req.body)
.digest('hex');
if (signature !== expected) {
return res.status(401).send('Invalid signature');
}
res.status(200).send('OK');
const hash = getEventHash(req.body);
// Fast path: check Redis
const isNew = await redis.set(`webhook:dedup:${hash}`, '1', 'EX', 3600, 'NX');
if (isNew !== 'OK') {
console.log(`Duplicate event ${hash.substring(0, 8)}, skipping`);
return;
}
const event = JSON.parse(req.body.toString());
// Durable path: insert into database (with ON CONFLICT as safety net)
await db.query(
`INSERT INTO webhook_events (event_hash, event_type, event_timestamp, data)
VALUES ($1, $2, $3, $4)
ON CONFLICT (event_hash) DO NOTHING`,
[hash, event.event, event.timestamp, JSON.stringify(event.data)]
);
// Process the event (side effects)
await handleEvent(event);
});
Redis catches most duplicates quickly (sub-millisecond). The database ON CONFLICT is a safety net for the rare case where Redis was unavailable when the first delivery arrived.
Testing Idempotency
Send the same event to your receiver twice and verify the outcome:
import { describe, it, expect } from 'vitest';
describe('idempotent webhook processing', () => {
it('should process an event only once', async () => {
const payload = JSON.stringify({
event: 'install.tracked',
timestamp: '2026-05-24T10:00:00.000Z',
data: { prefix: 'go', token: 'test', platform: 'ios', campaign: 'test' },
});
// Send the same event twice
const res1 = await sendWebhook(payload);
const res2 = await sendWebhook(payload);
// Both should return 200
expect(res1.status).toBe(200);
expect(res2.status).toBe(200);
// But only one record should exist
const count = await db.query(
"SELECT COUNT(*) FROM webhook_events WHERE event_type = 'install.tracked'"
);
expect(parseInt(count.rows[0].count)).toBe(1);
});
});
For more testing patterns, see the webhook testing tools guide.
Cleanup
Deduplication records accumulate over time. Set a retention policy:
- Redis: The TTL handles this automatically (keys expire after 1 hour)
- Database: Run a daily cleanup job to delete old records
-- Delete dedup records older than 7 days
DELETE FROM webhook_events WHERE processed_at < NOW() - INTERVAL '7 days';
Keep records long enough to cover any reasonable retry window plus a safety margin. 7 days is conservative; 24 hours is usually sufficient since Tolinku's retry window is under 1 hour.
For the complete webhook integration overview, see the webhooks and integrations guide.
Get deep linking tips in your inbox
One email per week. No spam.
