GET Site Runs & Pages

Monitor a whole site — crawl on a schedule and track pages added, removed, and changed

A site monitor (scope: "site") crawls the whole site on each interval, compares the set of pages against the previous run, and reports pages added, removed, and changed (with a per‑page content diff). It reuses the same schedule, alerting (webhook + email), and AI meaningful‑change filtering as page monitors — the unit of work is a crawl instead of a single fetch.

Create one by setting scope: "site" and the crawl config on POST /v1/monitors:

{
  "url": "https://example.com",
  "scope": "site",
  "maxPages": 25,
  "maxDepth": 2,
  "excludePatterns": ["/cart", "/checkout"],
  "intervalMinutes": 720,
  "alertWebhookUrl": "https://your-app.com/hooks/anakin"
}

How the page set stays stable. Discovered URLs are sorted before the maxPages cap, so a stable site yields the same pages each run. To pin the set exactly, hand‑pick pages (the dashboard's "Inspect" flow stores them as exact URLs in includePatterns), or remove pages you don't care about (below) so their crawl budget goes to others.

Credits & limits

• 1 credit per page crawled per run (a browser‑rendered crawl costs more, same as the URL scraper).
• maxPages is capped at 50 per run.
• Site monitors have their own active‑monitor caps per plan (lower than page monitors, since each run crawls many pages).

The change payload

For a site monitor, each entry from GET /v1/monitors/:id/changes has a diffJson of type "site":

{
  "type": "site",
  "runId": "…",
  "counts": { "added": 2, "removed": 1, "changed": 4, "pages": 25 },
  "added":   [{ "url": "https://example.com/new" }],
  "removed": [{ "url": "https://example.com/gone" }],
  "changed": [{ "url": "https://example.com/pricing", "summary": "Price rose from $19 to $24." }]
}

summary on a changed entry is present only when AI mode is on. The same digest is delivered to your webhook/email as one alert per run.

List runs

Each site‑monitor check is a run. Returns the runs newest‑first with their set‑diff counts.

{
  "runs": [
    {
      "id": "…",
      "jobId": "…",
      "status": "completed",
      "startedAt": "2026-07-06T12:00:00Z",
      "finishedAt": "2026-07-06T12:01:30Z",
      "pageCount": 25,
      "addedCount": 2,
      "removedCount": 1,
      "changedCount": 4
    }
  ]
}

Tracked pages

The pages currently tracked (from the latest run) and the URLs you've removed from tracking.

{ "tracked": ["https://example.com", "https://example.com/pricing"], "excluded": ["https://example.com/blog"] }

Stop tracking a page. It's excluded from every future crawl (freeing budget for other pages) and never reported as removed. Returns { "success": true, "excludedUrls": [...] }.

Resume tracking a previously‑removed page. Returns { "success": true, "excludedUrls": [...] }.