Webhooks

Receive signed HTTP event notifications when your jobs finish — subscription models, request format, signature verification, and delivery semantics

Every asynchronous AnakinScraper product can POST a signed JSON event to your server the moment work finishes — no polling loop required. One request format, one signature scheme, one event taxonomy across URL scraping, crawling, mapping, agentic search, Wire, website monitoring, and AI Visibility.

Webhooks complement polling — the poll endpoints keep working exactly as before. Use webhooks when you'd rather be told than ask.

Results are delivered by reference, not by value. Every terminal event carries data.result_url — an authenticated GET endpoint for the complete result, fetched with your API key. Small results are also inlined as data.result as a convenience, but only while the whole envelope stays under 256 KB; past that the inline copy is dropped (resultOmitted: "size"), and some products (AI Visibility) never inline full results at all. Build your receiver around result_url — the inline copy is an optimization, not a contract. See Event Types & Payloads for the fetch pattern per product.

Two ways to subscribe

1. Per-request webhook_url

Pass an optional webhook_url in the body of any async submit endpoint and we notify that URL when that job settles:

Per-request deliveries are signed with your account's default signing secret — fetch it with GET /v1/webhooks/signing-secret, rotate it with POST /v1/webhooks/signing-secret/rotate.

2. Registered endpoints

Account-level destinations that receive every matching event across all products, with optional event filters — point one receiver at everything, or split events across services. Each endpoint has its own whsec_… secret; an empty filter subscribes to all events, and you can register up to 10 endpoints per account. Manage them in the dashboard (Account → Webhooks) or via the management API.

Sync endpoints emit events too — POST /v1/search and POST /v1/url-scraper/scrape return results inline, but still emit job.completed / job.failed events to registered endpoints, so one receiver can observe every job uniformly. (webhook_url itself is not accepted on POST /v1/search.)

Request format

Every delivery is an HTTP POST to your URL with these headers:

The body is the event envelope:

{
  "id": "evt_a1b2c3…",
  "type": "job.completed",
  "createdAt": "2026-07-13T10:00:00Z",
  "data": { }
}

The one exception is monitor.change, which keeps its original flat payload for backward compatibility — see Event Types.

Per-request timeout is 10 seconds — acknowledge quickly (see Best practices).

Verify the signature

Your webhook URL is public: anyone who learns it could POST forged events. So every delivery is HMAC-SHA256 signed with a secret only you and we hold — the secret never travels in the request, only the signature does. Verification happens on your side: recompute the signature from the received body and your copy of the secret, then compare. Match → genuinely from us, untampered → process it. No match → forged or wrong secret → reject it (return 401, ignore the body). Verifying is optional but strongly recommended for any endpoint that acts on the data.

Which secret signs a delivery:

The signature is computed over the exact raw bytes of the body, so verify against the raw body, never a re-serialized object (JSON key order or whitespace would differ and the check would fail).

import express from "express"
import crypto from "crypto"

const app = express()
app.use("/anakin-webhook", express.raw({ type: "application/json" })) // raw bytes, not parsed

app.post("/anakin-webhook", (req, res) => {
  const secret = process.env.ANAKIN_WEBHOOK_SECRET // whsec_… (endpoint secret or default signing secret)
  const got = req.header("X-Anakin-Signature") || ""
  const expected = "sha256=" + crypto.createHmac("sha256", secret).update(req.body).digest("hex")

  const ok =
    got.length === expected.length &&
    crypto.timingSafeEqual(Buffer.from(got), Buffer.from(expected)) // constant-time compare
  if (!ok) return res.status(401).send("bad signature")

  const event = JSON.parse(req.body.toString())
  // de-dupe on req.header("X-Anakin-Delivery-Id"), then process asynchronously
  res.status(200).send("ok")
})

import hmac, hashlib, os
from flask import Flask, request, abort

app = Flask(__name__)

@app.post("/anakin-webhook")
def webhook():
    secret = os.environ["ANAKIN_WEBHOOK_SECRET"].encode()
    raw = request.get_data()                        # raw bytes — do NOT use request.json here
    expected = "sha256=" + hmac.new(secret, raw, hashlib.sha256).hexdigest()
    got = request.headers.get("X-Anakin-Signature", "")
    if not hmac.compare_digest(expected, got):      # constant-time compare
        abort(401)
    event = request.get_json()
    # de-dupe on request.headers.get("X-Anakin-Delivery-Id"), then process
    return "", 200

Delivery, retries & idempotency

• Success: your endpoint returns 2xx → recorded as success.

• Permanent failure (no retry): any 4xx other than 408 / 429 (e.g. 400, 401, 404) → we assume the endpoint is misconfigured and stop → recorded as exhausted.

• Retried with backoff: a connection error / timeout, any 5xx, a 408, or a 429 → retried up to 6 attempts (≈8.5 h total), then marked exhausted:

  After attempt	Wait before next
  1	1 min
  2	5 min
  3	30 min
  4	2 h
  5	6 h

• Each attempt times out after 10 seconds and follows at most 3 redirects.

• At-least-once — de-duplicate. A delivery can arrive more than once (e.g. you processed it but replied slowly, so we retried). De-dupe on X-Anakin-Delivery-Id, which is stable across retries.

• Delivery log & resend. Every attempt is recorded in your account-wide delivery log, retained for 30 days. failed / exhausted deliveries can be resent — byte-identical body and signature — from the dashboard or via POST /v1/webhooks/deliveries/{id}/resend.

Callback URLs

• HTTPS is strongly recommended; plain http is allowed.
• Maximum length is 2048 characters.
• Private and internal addresses are rejected (e.g. localhost, private IP ranges).

Best practices

• Verify the signature with a constant-time compare, over the raw body.
• Respond 2xx fast (within 10s). Acknowledge first, do heavy work asynchronously — slow endpoints get retried and look like failures.
• Be idempotent — key processing on X-Anakin-Delivery-Id.
• Return the right status — 2xx = accepted; 5xx / 408 / 429 to ask for a retry; any other 4xx tells us to stop permanently, so use it only for genuinely bad requests.
• Don't rely on inline results — data.result is inlined only when small, and AI Visibility events never inline full answers; data.result_url is always present. Fetch it whenever resultOmitted is set (or whenever there's no inline result).
• Keep secrets secret — treat them like passwords; rotate the default secret if it leaks.
• Don't assume ordering — deliveries can overlap under retries; use createdAt / X-Anakin-Timestamp if you need sequence.

In this section

• Event Types & Payloads — the envelope, the full taxonomy, and per-event data fields with examples
• Managing Endpoints — the management API: registered endpoints, the delivery log, resend, and signing secrets
• Website Monitor Webhooks — the monitor-specific guide (flat monitor.change payload, snapshot fetching)