# Anakin.io Documentation > Anakin.io is a web scraping and web data API from Anakin Inc (YC S21) built for AI agents: scrape, crawl, search, and pre-built Wire endpoints for 940+ sites. No API key is needed to start — the keyless "Zero Touch" tier lets an agent scrape a URL or run a read-only Wire action in a single unauthenticated HTTP call, then upgrade in place by adding a key. This file concatenates every Anakin.io documentation page as plain Markdown for LLMs. It contains 139 pages. Canonical docs: https://anakin.io/docs/documentation ## How to read this file - Pages are separated by a line containing only `---`, and each page begins with an `# (<url>)` heading. Page bodies contain `---` horizontal rules of their own, so split on the `# <Title> (<url>)` headings, not on `---`. - Sections appear in this order: api-reference, documentation, sdks, integrations, releases. - Every page is also available on its own as raw Markdown at `https://anakin.io/llms.mdx/docs/{slug}`. The Contents index below lists each URL. - A shorter, curated map of the whole site is at https://anakin.io/llms.txt. ## How to tell if this file was truncated This file is large. If your client caps how much it reads, you will get a prefix of it and no error. The last line of a complete copy is exactly: # End of anakin.io llms-full.txt If that line is missing, your copy is incomplete. Use the Contents index below — which is emitted before any page body, so it survives truncation — and fetch the pages you need individually from their `https://anakin.io/llms.mdx/docs/{slug}` URLs. ## Contents ### api-reference - API Reference — https://anakin.io/docs/api-reference (Markdown: https://anakin.io/llms.mdx/docs/api-reference) - Agentic Search — https://anakin.io/docs/api-reference/agentic-search (Markdown: https://anakin.io/llms.mdx/docs/api-reference/agentic-search) - AI Visibility — https://anakin.io/docs/api-reference/ai-visibility (Markdown: https://anakin.io/llms.mdx/docs/api-reference/ai-visibility) - Browser API — https://anakin.io/docs/api-reference/browser-api (Markdown: https://anakin.io/llms.mdx/docs/api-reference/browser-api) - Browser Sessions — https://anakin.io/docs/api-reference/browser-sessions (Markdown: https://anakin.io/llms.mdx/docs/api-reference/browser-sessions) - Crawl — https://anakin.io/docs/api-reference/crawl (Markdown: https://anakin.io/llms.mdx/docs/api-reference/crawl) - Error Responses — https://anakin.io/docs/api-reference/error-responses (Markdown: https://anakin.io/llms.mdx/docs/api-reference/error-responses) - Map (URL Discovery) — https://anakin.io/docs/api-reference/map (Markdown: https://anakin.io/llms.mdx/docs/api-reference/map) - Website Monitoring — https://anakin.io/docs/api-reference/monitoring (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring) - Polling Jobs — https://anakin.io/docs/api-reference/polling-jobs (Markdown: https://anakin.io/llms.mdx/docs/api-reference/polling-jobs) - Search API — https://anakin.io/docs/api-reference/search (Markdown: https://anakin.io/llms.mdx/docs/api-reference/search) - Supported Countries & Territories — https://anakin.io/docs/api-reference/supported-countries (Markdown: https://anakin.io/llms.mdx/docs/api-reference/supported-countries) - URL Scraper — https://anakin.io/docs/api-reference/url-scraper (Markdown: https://anakin.io/llms.mdx/docs/api-reference/url-scraper) - Webhooks — https://anakin.io/docs/api-reference/webhooks (Markdown: https://anakin.io/llms.mdx/docs/api-reference/webhooks) - Wire — https://anakin.io/docs/api-reference/wire (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire) - GET Get Results — https://anakin.io/docs/api-reference/agentic-search/get-search-result (Markdown: https://anakin.io/llms.mdx/docs/api-reference/agentic-search/get-search-result) - POST Research Query — https://anakin.io/docs/api-reference/agentic-search/submit-search (Markdown: https://anakin.io/llms.mdx/docs/api-reference/agentic-search/submit-search) - GET Search — https://anakin.io/docs/api-reference/ai-visibility/get-search (Markdown: https://anakin.io/llms.mdx/docs/api-reference/ai-visibility/get-search) - GET List Searches — https://anakin.io/docs/api-reference/ai-visibility/list-searches (Markdown: https://anakin.io/llms.mdx/docs/api-reference/ai-visibility/list-searches) - GET List Sources — https://anakin.io/docs/api-reference/ai-visibility/list-sources (Markdown: https://anakin.io/llms.mdx/docs/api-reference/ai-visibility/list-sources) - POST Retry Source — https://anakin.io/docs/api-reference/ai-visibility/retry-source (Markdown: https://anakin.io/llms.mdx/docs/api-reference/ai-visibility/retry-source) - POST Submit Search — https://anakin.io/docs/api-reference/ai-visibility/submit-search (Markdown: https://anakin.io/llms.mdx/docs/api-reference/ai-visibility/submit-search) - WS Geo-Targeting — https://anakin.io/docs/api-reference/browser-api/geo-targeting (Markdown: https://anakin.io/llms.mdx/docs/api-reference/browser-api/geo-targeting) - Limits & Billing — https://anakin.io/docs/api-reference/browser-api/limits (Markdown: https://anakin.io/llms.mdx/docs/api-reference/browser-api/limits) - WS Recording — https://anakin.io/docs/api-reference/browser-api/recording (Markdown: https://anakin.io/llms.mdx/docs/api-reference/browser-api/recording) - WS Saved Sessions — https://anakin.io/docs/api-reference/browser-api/saved-sessions (Markdown: https://anakin.io/llms.mdx/docs/api-reference/browser-api/saved-sessions) - GET Get Crawl Result — https://anakin.io/docs/api-reference/crawl/get-crawl-result (Markdown: https://anakin.io/llms.mdx/docs/api-reference/crawl/get-crawl-result) - POST Submit Crawl Job — https://anakin.io/docs/api-reference/crawl/submit-crawl-job (Markdown: https://anakin.io/llms.mdx/docs/api-reference/crawl/submit-crawl-job) - GET Get Map Result — https://anakin.io/docs/api-reference/map/get-map-result (Markdown: https://anakin.io/llms.mdx/docs/api-reference/map/get-map-result) - POST Submit Map Job — https://anakin.io/docs/api-reference/map/submit-map-job (Markdown: https://anakin.io/llms.mdx/docs/api-reference/map/submit-map-job) - Alerts, Deliveries & Webhooks — https://anakin.io/docs/api-reference/monitoring/alerts (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/alerts) - POST Pause / Resume / End Date / Run Now — https://anakin.io/docs/api-reference/monitoring/control-monitor (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/control-monitor) - POST Create Monitor — https://anakin.io/docs/api-reference/monitoring/create-monitor (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/create-monitor) - GET List & Get Monitors — https://anakin.io/docs/api-reference/monitoring/list-monitors (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/list-monitors) - GET Site Runs & Pages — https://anakin.io/docs/api-reference/monitoring/site-monitors (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/site-monitors) - GET Snapshots & Changes — https://anakin.io/docs/api-reference/monitoring/snapshots-and-changes (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/snapshots-and-changes) - PUT / DELETE Update & Delete Monitor — https://anakin.io/docs/api-reference/monitoring/update-monitor (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/update-monitor) - Wire Monitors — https://anakin.io/docs/api-reference/monitoring/wire-monitors (Markdown: https://anakin.io/llms.mdx/docs/api-reference/monitoring/wire-monitors) - POST Search — https://anakin.io/docs/api-reference/search/search (Markdown: https://anakin.io/llms.mdx/docs/api-reference/search/search) - POST Scrape URLs — https://anakin.io/docs/api-reference/url-scraper/batch-url-scraping (Markdown: https://anakin.io/llms.mdx/docs/api-reference/url-scraper/batch-url-scraping) - GET Get Results — https://anakin.io/docs/api-reference/url-scraper/get-job-status (Markdown: https://anakin.io/llms.mdx/docs/api-reference/url-scraper/get-job-status) - GET Download Screenshot — https://anakin.io/docs/api-reference/url-scraper/get-screenshot (Markdown: https://anakin.io/llms.mdx/docs/api-reference/url-scraper/get-screenshot) - POST Scrape URL (Inline) — https://anakin.io/docs/api-reference/url-scraper/scrape (Markdown: https://anakin.io/llms.mdx/docs/api-reference/url-scraper/scrape) - POST Scrape URL — https://anakin.io/docs/api-reference/url-scraper/submit-scrape-job (Markdown: https://anakin.io/llms.mdx/docs/api-reference/url-scraper/submit-scrape-job) - Event Types & Payloads — https://anakin.io/docs/api-reference/webhooks/events (Markdown: https://anakin.io/llms.mdx/docs/api-reference/webhooks/events) - Managing Endpoints — https://anakin.io/docs/api-reference/webhooks/managing-endpoints (Markdown: https://anakin.io/llms.mdx/docs/api-reference/webhooks/managing-endpoints) - POST Build Request — https://anakin.io/docs/api-reference/wire/create-build-request (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/create-build-request) - POST Connect Source — https://anakin.io/docs/api-reference/wire/create-identity-source (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/create-identity-source) - DELETE Disconnect Source — https://anakin.io/docs/api-reference/wire/delete-identity-source (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/delete-identity-source) - POST Execute Task — https://anakin.io/docs/api-reference/wire/execute-task (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/execute-task) - GET Catalog Details — https://anakin.io/docs/api-reference/wire/get-catalog (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/get-catalog) - GET Source Details — https://anakin.io/docs/api-reference/wire/get-identity-source (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/get-identity-source) - GET Job Status — https://anakin.io/docs/api-reference/wire/get-job (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/get-job) - GET Download Job File — https://anakin.io/docs/api-reference/wire/get-job-download (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/get-job-download) - GET List Catalogs — https://anakin.io/docs/api-reference/wire/list-catalogs (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/list-catalogs) - GET List Identities — https://anakin.io/docs/api-reference/wire/list-identities (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/list-identities) - GET List Sources — https://anakin.io/docs/api-reference/wire/list-identity-sources (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/list-identity-sources) - GET List Providers — https://anakin.io/docs/api-reference/wire/list-providers (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/list-providers) - GET List Containers — https://anakin.io/docs/api-reference/wire/list-source-containers (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/list-source-containers) - GET List Entries — https://anakin.io/docs/api-reference/wire/list-source-entries (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/list-source-entries) - GET Identities Using a Source — https://anakin.io/docs/api-reference/wire/list-source-identities (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/list-source-identities) - POST Create Identity / Login — https://anakin.io/docs/api-reference/wire/login (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/login) - Provider Setup — https://anakin.io/docs/api-reference/wire/provider-setup (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/provider-setup) - GET Resolve Actions — https://anakin.io/docs/api-reference/wire/search-actions (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/search-actions) - PATCH Rename or Rotate Source — https://anakin.io/docs/api-reference/wire/update-identity-source (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/update-identity-source) - POST Verify Identity / Re-Login — https://anakin.io/docs/api-reference/wire/verify-credential (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/verify-credential) - POST Verify Source — https://anakin.io/docs/api-reference/wire/verify-identity-source (Markdown: https://anakin.io/llms.mdx/docs/api-reference/wire/verify-identity-source) ### documentation - Overview — https://anakin.io/docs/documentation (Markdown: https://anakin.io/llms.mdx/docs/documentation) - AI Visibility — https://anakin.io/docs/documentation/ai-visibility (Markdown: https://anakin.io/llms.mdx/docs/documentation/ai-visibility) - Overview — https://anakin.io/docs/documentation/getting-started (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started) - Pricing & Credits — https://anakin.io/docs/documentation/pricing (Markdown: https://anakin.io/llms.mdx/docs/documentation/pricing) - Rate Limits — https://anakin.io/docs/documentation/rate-limits (Markdown: https://anakin.io/llms.mdx/docs/documentation/rate-limits) - Use Cases — https://anakin.io/docs/documentation/use-cases (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases) - CLI — https://anakin.io/docs/documentation/getting-started/cli (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/cli) - cURL — https://anakin.io/docs/documentation/getting-started/curl (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/curl) - .NET — https://anakin.io/docs/documentation/getting-started/dotnet (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/dotnet) - Elixir — https://anakin.io/docs/documentation/getting-started/elixir (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/elixir) - Go — https://anakin.io/docs/documentation/getting-started/go (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/go) - Java — https://anakin.io/docs/documentation/getting-started/java (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/java) - Node.js — https://anakin.io/docs/documentation/getting-started/node (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/node) - PHP — https://anakin.io/docs/documentation/getting-started/php (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/php) - Python — https://anakin.io/docs/documentation/getting-started/python (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/python) - Ruby — https://anakin.io/docs/documentation/getting-started/ruby (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/ruby) - Rust — https://anakin.io/docs/documentation/getting-started/rust (Markdown: https://anakin.io/llms.mdx/docs/documentation/getting-started/rust) - AI & Agent Data Ingestion — https://anakin.io/docs/documentation/use-cases/ai-agent-data-ingestion (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/ai-agent-data-ingestion) - Browser Automation & Web Workflows — https://anakin.io/docs/documentation/use-cases/browser-automation (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/browser-automation) - Competitive Intelligence & Pricing — https://anakin.io/docs/documentation/use-cases/competitive-intel (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/competitive-intel) - Content Aggregation — https://anakin.io/docs/documentation/use-cases/content-aggregation (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/content-aggregation) - Financial & Corporate Filings — https://anakin.io/docs/documentation/use-cases/financial-filings (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/financial-filings) - Lead Generation — https://anakin.io/docs/documentation/use-cases/lead-generation (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/lead-generation) - Structured Market & Web Research — https://anakin.io/docs/documentation/use-cases/market-research (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/market-research) - ML & Training Data Collection — https://anakin.io/docs/documentation/use-cases/ml-training-data (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/ml-training-data) - Review & Sentiment Data Extraction — https://anakin.io/docs/documentation/use-cases/review-sentiment (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/review-sentiment) - SEO & Search Intelligence — https://anakin.io/docs/documentation/use-cases/seo-marketing (Markdown: https://anakin.io/llms.mdx/docs/documentation/use-cases/seo-marketing) ### sdks - SDKs & CLI — https://anakin.io/docs/sdks (Markdown: https://anakin.io/llms.mdx/docs/sdks) - Anakin CLI — https://anakin.io/docs/sdks/cli (Markdown: https://anakin.io/llms.mdx/docs/sdks/cli) - .NET SDK — https://anakin.io/docs/sdks/dotnet (Markdown: https://anakin.io/llms.mdx/docs/sdks/dotnet) - Elixir SDK — https://anakin.io/docs/sdks/elixir (Markdown: https://anakin.io/llms.mdx/docs/sdks/elixir) - Go SDK — https://anakin.io/docs/sdks/go (Markdown: https://anakin.io/llms.mdx/docs/sdks/go) - Java SDK — https://anakin.io/docs/sdks/java (Markdown: https://anakin.io/llms.mdx/docs/sdks/java) - Node.js SDK — https://anakin.io/docs/sdks/node (Markdown: https://anakin.io/llms.mdx/docs/sdks/node) - PHP SDK — https://anakin.io/docs/sdks/php (Markdown: https://anakin.io/llms.mdx/docs/sdks/php) - Python SDK — https://anakin.io/docs/sdks/python (Markdown: https://anakin.io/llms.mdx/docs/sdks/python) - Releases — https://anakin.io/docs/sdks/releases (Markdown: https://anakin.io/llms.mdx/docs/sdks/releases) - Ruby SDK — https://anakin.io/docs/sdks/ruby (Markdown: https://anakin.io/llms.mdx/docs/sdks/ruby) - Rust SDK — https://anakin.io/docs/sdks/rust (Markdown: https://anakin.io/llms.mdx/docs/sdks/rust) - CLI Commands — https://anakin.io/docs/sdks/cli/commands (Markdown: https://anakin.io/llms.mdx/docs/sdks/cli/commands) - CLI Examples — https://anakin.io/docs/sdks/cli/examples (Markdown: https://anakin.io/llms.mdx/docs/sdks/cli/examples) ### integrations - Integrations — https://anakin.io/docs/integrations (Markdown: https://anakin.io/llms.mdx/docs/integrations) - Claude Connector — https://anakin.io/docs/integrations/ai-agents/claude-connector (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-agents/claude-connector) - MCP Server — https://anakin.io/docs/integrations/ai-agents/mcp-server (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-agents/mcp-server) - Agent Onboarding (SKILL.md) — https://anakin.io/docs/integrations/ai-agents/skill-md (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-agents/skill-md) - Zero Touch — https://anakin.io/docs/integrations/ai-agents/zero-touch (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-agents/zero-touch) - CrewAI — https://anakin.io/docs/integrations/ai-frameworks/crewai (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-frameworks/crewai) - Flowise — https://anakin.io/docs/integrations/ai-frameworks/flowise (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-frameworks/flowise) - Google ADK — https://anakin.io/docs/integrations/ai-frameworks/google-adk (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-frameworks/google-adk) - LangChain — https://anakin.io/docs/integrations/ai-frameworks/langchain (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-frameworks/langchain) - Langflow — https://anakin.io/docs/integrations/ai-frameworks/langflow (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-frameworks/langflow) - LlamaIndex — https://anakin.io/docs/integrations/ai-frameworks/llamaindex (Markdown: https://anakin.io/llms.mdx/docs/integrations/ai-frameworks/llamaindex) - Claude Code — https://anakin.io/docs/integrations/ide-plugins/claude-code (Markdown: https://anakin.io/llms.mdx/docs/integrations/ide-plugins/claude-code) - Cursor — https://anakin.io/docs/integrations/ide-plugins/cursor (Markdown: https://anakin.io/llms.mdx/docs/integrations/ide-plugins/cursor) - MCP Server — https://anakin.io/docs/integrations/ide-plugins/mcp-server (Markdown: https://anakin.io/llms.mdx/docs/integrations/ide-plugins/mcp-server) - OpenClaw — https://anakin.io/docs/integrations/ide-plugins/openclaw (Markdown: https://anakin.io/llms.mdx/docs/integrations/ide-plugins/openclaw) - Website Monitor Webhooks — https://anakin.io/docs/integrations/website-monitoring/webhooks (Markdown: https://anakin.io/llms.mdx/docs/integrations/website-monitoring/webhooks) - Dify — https://anakin.io/docs/integrations/workflow/dify (Markdown: https://anakin.io/llms.mdx/docs/integrations/workflow/dify) - Make — https://anakin.io/docs/integrations/workflow/make (Markdown: https://anakin.io/llms.mdx/docs/integrations/workflow/make) - n8n — https://anakin.io/docs/integrations/workflow/n8n (Markdown: https://anakin.io/llms.mdx/docs/integrations/workflow/n8n) - Zapier — https://anakin.io/docs/integrations/workflow/zapier (Markdown: https://anakin.io/llms.mdx/docs/integrations/workflow/zapier) ### releases - Releases — https://anakin.io/docs/releases (Markdown: https://anakin.io/llms.mdx/docs/releases) ### geo-aeo - Overview — https://anakin.io/docs/geo-aeo (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo) - All engines — https://anakin.io/docs/geo-aeo/engines (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines) - Pricing — https://anakin.io/docs/geo-aeo/pricing (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/pricing) - ChatGPT — https://anakin.io/docs/geo-aeo/engines/chatgpt (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines/chatgpt) - Gemini — https://anakin.io/docs/geo-aeo/engines/gemini (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines/gemini) - Google AI Mode — https://anakin.io/docs/geo-aeo/engines/google-ai-mode (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines/google-ai-mode) - Google AI Overview — https://anakin.io/docs/geo-aeo/engines/google-ai-overview (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines/google-ai-overview) - Naver — https://anakin.io/docs/geo-aeo/engines/naver (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines/naver) - Perplexity — https://anakin.io/docs/geo-aeo/engines/perplexity (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines/perplexity) - Wenxin (ERNIE) — https://anakin.io/docs/geo-aeo/engines/wenxin (Markdown: https://anakin.io/llms.mdx/docs/geo-aeo/engines/wenxin) --- # API Reference (/docs/api-reference) Explore the full AnakinScraper REST API. All endpoints use the base URL `https://api.anakin.io/v1` and require an `X-API-Key` header for authentication. ### Products <CardGrid> <ProductCard title="Wire" description="Pre-built actions for popular websites — scrape and extract structured data" href="/docs/api-reference/wire" /> <ProductCard title="URL Scraper" description="Scrape single or batch URLs with caching and AI extraction" href="/docs/api-reference/url-scraper" /> <ProductCard title="Map (URL Discovery)" description="Discover all URLs on a website via sitemap and link extraction" href="/docs/api-reference/map" /> <ProductCard title="Crawl" description="Scrape multiple pages from a website with pattern filtering" href="/docs/api-reference/crawl" /> <ProductCard title="Search API" description="AI-powered web search with citations (synchronous)" href="/docs/api-reference/search" /> <ProductCard title="Agentic Search" description="Multi-stage automated research pipeline" href="/docs/api-reference/agentic-search" /> <ProductCard title="Browser API" description="Control a stealth browser with Playwright or Puppeteer via CDP WebSocket" href="/docs/api-reference/browser-api" /> <ProductCard title="Browser Sessions" description="Save and reuse login sessions for authenticated scraping" href="/docs/api-reference/browser-sessions" /> <ProductCard title="Webhooks" description="Signed event notifications to your server when jobs finish — no polling" href="/docs/api-reference/webhooks" /> </CardGrid> ### Authentication Every request requires an API key passed via the `X-API-Key` header: ``` X-API-Key: your_api_key ``` Get your API key from the [Dashboard](/dashboard). --- # Agentic Search (/docs/api-reference/agentic-search) Advanced 4-stage AI research pipeline that automatically searches the web, scrapes relevant sources, and synthesizes a comprehensive research report. Jobs are processed asynchronously. ### How It Works 1. **Query refinement** — AI refines your research question for optimal search 2. **Web search** — discovers relevant sources and citations 3. **Citation scraping** — automatically scrapes top citation URLs for full content 4. **Analysis & synthesis** — produces a comprehensive research report ### Endpoints <EndpointCard method="POST" path="/v1/agentic-search" description="Submit Search — start an agentic research job" href="/docs/api-reference/agentic-search/submit-search" /> <EndpointCard method="GET" path="/v1/agentic-search/{id}" description="Get Search Result — poll for status and retrieve research report" href="/docs/api-reference/agentic-search/get-search-result" /> --- # AI Visibility (/docs/api-reference/ai-visibility) AI Visibility fans a single prompt out to multiple AI engines (ChatGPT, Gemini, Google AI Overview) through Wire's catalogue and returns every engine's answer — full response payload, extracted summary, measured latency, and credit cost — for side-by-side comparison. Searches run **asynchronously**: submitting returns a `search_id` immediately, each source's result lands the moment its engine answers, and you poll (or receive a webhook) for completion. ### Endpoints <EndpointCard method="POST" path="/v1/ai-visibility/search" description="Submit Search — fan a prompt out to the selected AI sources" href="/docs/api-reference/ai-visibility/submit-search" /> <EndpointCard method="GET" path="/v1/ai-visibility/search/{id}" description="Get Search — poll status and per-source results" href="/docs/api-reference/ai-visibility/get-search" /> <EndpointCard method="GET" path="/v1/ai-visibility/searches" description="List Searches — your 20 most recent runs" href="/docs/api-reference/ai-visibility/list-searches" /> <EndpointCard method="POST" path="/v1/ai-visibility/search/{id}/retry" description="Retry Source — re-run one failed source" href="/docs/api-reference/ai-visibility/retry-source" /> <EndpointCard method="GET" path="/v1/ai-visibility/sources" description="List Sources — the AI engines available to query" href="/docs/api-reference/ai-visibility/list-sources" /> ### Authentication All endpoints require your API key in the `X-API-Key` header. See [Error Responses](/docs/api-reference/error-responses) for the full list of accepted auth headers. ### Search geography Each search runs from a caller-chosen country (`country` on [Submit Search](/docs/api-reference/ai-visibility/submit-search), 2-letter ISO code, default `us`) — the same geo targeting the URL scraper offers. AI engine answers are region-dependent, and Google AI Overview in particular may not be served at all for some query/region combinations, so results reflect what a user in that country would see. ### Credits Each source that completes bills its Wire action's per-call rate, snapshotted at execution time. The per-source cost is returned as `credits_used` on every result, and failed sources cost nothing. Results are never cached — AI engine answers change constantly, so every submit queries the engines fresh (and bills accordingly). ### Webhooks Instead of polling, subscribe an endpoint to the `ai.search.completed` / `ai.search.failed` events — a signed delivery fires the moment a search settles, carrying per-source statuses, latencies, credits, and bounded answer previews. > **Note — webhook deliveries never contain the full answers.** Engine answers routinely run to hundreds of KB, so each `sources[]` entry inlines only a `summary` preview (≤ 500 chars). The event's `data.result_url` — always present — is the authenticated endpoint for the complete results: > > ```bash > curl "https://api.anakin.io/v1/ai-visibility/search/{searchId}" \ > -H "X-API-Key: your_api_key" > ``` `ai.search.*` events are delivered to **registered endpoints** only (the submit endpoint takes no per-request `webhook_url`). See [Event Types & Payloads](/docs/api-reference/webhooks/events) for the full payload schema and [Webhooks](/docs/api-reference/webhooks) for signatures and endpoint management. --- # Browser API (/docs/api-reference/browser-api) ## What is Browser API? Browser API gives you a **stealth browser in the cloud** that you control with your own code. Connect Playwright, Puppeteer, or any CDP-compatible client to Anakin's anti-detection browser via a single WebSocket URL. Unlike traditional scraping APIs where you submit a URL and get results back, Browser API gives you **full browser control**: navigate pages, click buttons, fill forms, take screenshots, extract data — all through your own automation scripts. --- ## Why use Browser API? - **Anti-detection built in** — Stealth browser with aligned fingerprint, locale, and timezone signals. A UK session reports `Europe/London` + `en-GB`, not `UTC` + `en-US`. - **Playwright and Puppeteer support** — Connect with `connect_over_cdp` (Playwright) or `browser.connect()` (Puppeteer). No code changes beyond the connection URL. - **Residential exits in 20+ countries** — Pass `?country=GB` for a UK residential IP, `?country=JP` for Japan, etc. See [Geo-Targeting](/docs/api-reference/browser-api/geo-targeting). - **Saved sessions** — Reuse a logged-in session across runs. Cookies + localStorage stored on disconnect, replayed on the next connect. See [Saved Sessions](/docs/api-reference/browser-api/saved-sessions). - **Opt-in recording** — Add `?record=true` to capture a WebM video of any session. No extra cost. See [Recording](/docs/api-reference/browser-api/recording). - **Nothing to install or manage** — No browsers, no displays, no containers on your side. Just connect and scrape. - **Same API key** — Uses your existing Anakin API key. No separate auth flow. --- ## Quick Start <Tabs items={["Playwright (Python)", "Playwright (Node.js)", "Puppeteer (Node.js)"]}> <Tab value="Playwright (Python)"> ```python import asyncio from playwright.async_api import async_playwright API_KEY = "your_api_key" async def main(): async with async_playwright() as p: browser = await p.chromium.connect_over_cdp( "wss://api.anakin.io/v1/browser-connect", headers={"X-API-Key": API_KEY}, ) page = browser.contexts[0].pages[0] # Navigate and extract data await page.goto("https://books.toscrape.com", wait_until="domcontentloaded") print("Title:", await page.title()) # Extract structured data books = await page.evaluate(""" Array.from(document.querySelectorAll('article.product_pod')).slice(0, 5).map(el => ({ title: el.querySelector('h3 a')?.title, price: el.querySelector('.price_color')?.textContent, })) """) print("Books:", books) # Take a screenshot await page.screenshot(path="screenshot.png") await browser.close() asyncio.run(main()) ``` </Tab> <Tab value="Playwright (Node.js)"> ```javascript const { chromium } = require('playwright'); const API_KEY = 'your_api_key'; (async () => { const browser = await chromium.connectOverCDP( 'wss://api.anakin.io/v1/browser-connect', { headers: { 'X-API-Key': API_KEY } } ); const page = browser.contexts()[0].pages()[0]; await page.goto('https://books.toscrape.com', { waitUntil: 'domcontentloaded' }); console.log('Title:', await page.title()); const books = await page.evaluate(() => Array.from(document.querySelectorAll('article.product_pod')).slice(0, 5).map(el => ({ title: el.querySelector('h3 a')?.title, price: el.querySelector('.price_color')?.textContent, })) ); console.log('Books:', books); await page.screenshot({ path: 'screenshot.png' }); await browser.close(); })(); ``` </Tab> <Tab value="Puppeteer (Node.js)"> ```javascript const puppeteer = require('puppeteer-core'); const API_KEY = 'your_api_key'; (async () => { const browser = await puppeteer.connect({ browserWSEndpoint: 'wss://api.anakin.io/v1/browser-connect', headers: { 'X-API-Key': API_KEY }, }); const page = (await browser.pages())[0] || await browser.newPage(); await page.goto('https://books.toscrape.com', { waitUntil: 'domcontentloaded', timeout: 30000, }); console.log('Title:', await page.title()); console.log('HTML:', (await page.content()).length, 'chars'); await page.screenshot({ path: 'screenshot.png' }); await browser.disconnect(); })(); ``` </Tab> </Tabs> --- ## Supported Features ### Playwright | Feature | Status | |---------|--------| | `page.goto()` | ✅ | | `page.title()` / `page.content()` | ✅ | | `page.evaluate()` | ✅ | | `page.screenshot()` | ✅ | | `page.locator()` — count, text, attributes | ✅ | | `page.wait_for_selector()` / `page.wait_for_function()` | ✅ | | `page.inner_text()` / `page.text_content()` | ✅ | | `page.set_extra_http_headers()` | ✅ | | `page.on('request')` / `page.on('response')` | ✅ | | `page.on('console')` (all severities) | ✅ | | `page.on('websocket')` with framesent/framereceived | ✅ | | `page.on('pageerror')` (uncaught exceptions) | ✅ | | Mouse and keyboard input | ✅ | | Scroll / pagination | ✅ | | Cross-site navigation | ✅ | | Multiple pages per context (`context.newPage()`) | ✅ | | Stealth (`webdriver=false`) + geo-aligned fingerprint | ✅ | | `page.route()` / `context.route()` — fulfill / abort / continue | ✅ | | `page.unroute()` | ✅ | ### Puppeteer | Feature | Status | |---------|--------| | `page.goto()` | ✅ | | `page.title()` / `page.content()` | ✅ | | `page.evaluate()` | ✅ | | `page.screenshot()` | ✅ | | `page.$$()` — querySelectorAll | ✅ | | `page.$eval()` — scoped evaluate | ✅ | | `page.waitForSelector()` | ✅ | | `page.setExtraHTTPHeaders()` | ✅ | | Mouse and keyboard input | ✅ | | Scroll / pagination | ✅ | | Cross-site navigation | ✅ | | `browser.pages()` / `browser.newPage()` | ✅ | | `page.waitForNavigation()` | ✅ | ### Not supported - **Selenium / ChromeDriver** — only CDP-native clients (Playwright, Puppeteer, chromedp) are supported. Use Playwright or Puppeteer instead. --- ## Scraping Example: Extract Product Data ```python import asyncio from playwright.async_api import async_playwright async def scrape_books(): async with async_playwright() as p: browser = await p.chromium.connect_over_cdp( "wss://api.anakin.io/v1/browser-connect", headers={"X-API-Key": "your_api_key"}, ) page = browser.contexts[0].pages[0] await page.goto("https://books.toscrape.com", wait_until="domcontentloaded") # Wait for products to load await page.wait_for_selector("article.product_pod") # Extract all books on the page books = await page.evaluate(""" Array.from(document.querySelectorAll('article.product_pod')).map(el => ({ title: el.querySelector('h3 a')?.title, price: el.querySelector('.price_color')?.textContent, rating: el.querySelector('.star-rating')?.className.replace('star-rating ', ''), inStock: !!el.querySelector('.instock'), link: el.querySelector('h3 a')?.href, })) """) print(f"Scraped {len(books)} books") for book in books[:3]: print(f" {book['title']} — {book['price']}") await browser.close() asyncio.run(scrape_books()) ``` --- ## Next steps - [Geo-Targeting](/docs/api-reference/browser-api/geo-targeting) — Route through residential proxies in specific countries - [Saved Sessions](/docs/api-reference/browser-api/saved-sessions) — Connect with pre-authenticated cookies + localStorage - [Limits & Billing](/docs/api-reference/browser-api/limits) — Rate limits, idle timeout, billing model --- # Browser Sessions (/docs/api-reference/browser-sessions) > **Tip:** > - Session data is protected using **AES-256-GCM encryption** with complete user isolation. > - The system does not collect, store, or retain passwords, authentication secrets, or credentials at any time. > - Session data is permanently and irreversibly deleted upon user-initiated session removal. ## What are Browser Sessions? Browser sessions allow you to scrape content that requires authentication. Instead of handling complex login flows programmatically, you log in once through a real browser, and we save your session for future API requests. This is useful for scraping: - Account dashboards and order history - Subscription-based content - Social media profiles - Any page that requires a login --- ## How It Works There are two ways to create a session: ### Option A: Interactive (Dashboard) 1. **Create** — From your [dashboard](https://anakin.io/dashboard), click **Create Session** to launch an interactive browser 2. **Log in** — Navigate to the website and log in with your credentials. Complete 2FA or captchas as needed 3. **Save** — Click **Save Session** to encrypt and store your cookies and localStorage ### Option B: Programmatic (Browser API) Save sessions directly from [Browser API](/docs/api-reference/browser-api/saved-sessions) — just add `?save_session=my-name` when connecting. The session auto-saves when you disconnect: ```python browser = await p.chromium.connect_over_cdp( "wss://api.anakin.io/v1/browser-connect?save_session=my-amazon-login&save_url=https://amazon.com", headers={"X-API-Key": "your_api_key"}, ) # ... automate login with Playwright ... await browser.close() # session auto-saved ``` ### Use in API Requests Include the `sessionId` in your scrape requests. The API will use your saved session to access authenticated pages. --- ## Using Sessions with the API Add the `sessionId` parameter to your [URL Scraper](/docs/api-reference/url-scraper/submit-scrape-job) request: ```json { "url": "https://amazon.com/your-orders", "sessionId": "session_abc123xyz", "country": "us" } ``` When using a session, browser-based scraping is automatically enabled since sessions require a full browser environment. <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/url-scraper \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://amazon.com/your-orders", "sessionId": "session_abc123xyz", "country": "us" }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/url-scraper', headers={'X-API-Key': 'your_api_key'}, json={ 'url': 'https://amazon.com/your-orders', 'sessionId': 'session_abc123xyz', 'country': 'us' } ) data = response.json() print(f"Job submitted: {data['jobId']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/url-scraper', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ url: 'https://amazon.com/your-orders', sessionId: 'session_abc123xyz', country: 'us' }) }); const data = await response.json(); console.log(data.jobId); ``` </Tab> </Tabs> --- ## Using Sessions with Browser API You can also load saved sessions into [Browser API](/docs/api-reference/browser-api) for full programmatic control of an authenticated browser. Pass `?session_id` or `?session_name` when connecting: <Tabs items={["Python", "JavaScript"]}> <Tab value="Python"> ```python import asyncio from playwright.async_api import async_playwright async def main(): async with async_playwright() as p: browser = await p.chromium.connect_over_cdp( "wss://api.anakin.io/v1/browser-connect?session_id=session_abc123xyz", headers={"X-API-Key": "your_api_key"}, ) page = browser.contexts[0].pages[0] # Cookies are pre-loaded — navigate directly to authenticated pages await page.goto("https://amazon.com/your-orders") orders = await page.evaluate("document.title") print("Page:", orders) await browser.close() asyncio.run(main()) ``` </Tab> <Tab value="JavaScript"> ```javascript const { chromium } = require('playwright'); (async () => { const browser = await chromium.connectOverCDP( 'wss://api.anakin.io/v1/browser-connect?session_name=my-amazon-login', { headers: { 'X-API-Key': 'your_api_key' } } ); const page = browser.contexts()[0].pages()[0]; await page.goto('https://amazon.com/your-orders'); console.log('Title:', await page.title()); await browser.close(); })(); ``` </Tab> </Tabs> This is useful when: - You need to interact with authenticated pages (click, scroll, fill forms) - The URL Scraper API doesn't give you enough control - You want to combine session auth with custom Playwright/Puppeteer automation See the [Browser API docs](/docs/api-reference/browser-api/saved-sessions) for full details. --- ## Managing Sessions You can manage your sessions from the [dashboard](https://anakin.io/dashboard) or via the API. ### API Reference All session endpoints require an `X-API-Key` header. #### List sessions ```bash GET /v1/sessions ``` Returns all sessions belonging to the authenticated user. Optionally filter by domain: ```bash curl https://api.anakin.io/v1/sessions \ -H "X-API-Key: your_api_key" # Filter by domain curl "https://api.anakin.io/v1/sessions?domain=amazon.com" \ -H "X-API-Key: your_api_key" ``` #### Rename a session ```bash PATCH /v1/sessions/:id ``` ```bash curl -X PATCH https://api.anakin.io/v1/sessions/session_abc123 \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{"name": "new-session-name"}' ``` #### Delete a session ```bash DELETE /v1/sessions/:id ``` Permanently deletes the session and its encrypted storage. This action is irreversible. ```bash curl -X DELETE https://api.anakin.io/v1/sessions/session_abc123 \ -H "X-API-Key: your_api_key" ``` --- # Crawl (/docs/api-reference/crawl) The Crawl API scrapes multiple pages from a website. It first discovers URLs (like [Map](/docs/api-reference/map)), then scrapes each page and returns the content. Use it when you need the actual content from multiple pages, not just the URLs. ### Features - **Multi-page scraping** — crawl up to 100 pages in a single job - **Pattern filtering** — include/exclude URLs by glob patterns - **Browser rendering** — use headless Chrome for JS-heavy sites - **Per-page results** — get HTML and markdown for each crawled page - **Proxy routing** — route through [207 countries](/docs/api-reference/supported-countries) ### Endpoints <EndpointCard method="POST" path="/v1/crawl" description="Submit Crawl Job — crawl multiple pages from a website" href="/docs/api-reference/crawl/submit-crawl-job" /> <EndpointCard method="GET" path="/v1/crawl/{id}" description="Get Crawl Result — retrieve crawled page content" href="/docs/api-reference/crawl/get-crawl-result" /> --- # Error Responses (/docs/api-reference/error-responses) Every AnakinScraper endpoint returns errors in a consistent JSON shape. This page is the canonical reference — what we return, what each code means, when to retry, and how to diagnose the most common failures. For per-endpoint rate limits and the recommended request pacing, see [Rate Limits](/docs/documentation/rate-limits). --- ## Error response format All errors return a JSON body with two fields: ```json { "error": "rate_limit_exceeded", "message": "Too many requests. Please try again later." } ``` | Field | Type | Description | |-------|------|-------------| | `error` | string | A short, machine-readable code. Stable across releases — safe to switch on in code. | | `message` | string | A human-readable explanation. Subject to copy changes — log it but don't parse it. | > **Async jobs are different.** When a polled job ends in `status: "failed"`, the failure shape lives inside the job response (`error` field, free-form string). See [Async job failures](#async-job-failures) below. --- ## HTTP status codes | Status | Meaning | Retry? | |--------|---------|--------| | `200` | Success — synchronous endpoint returned a result, or polled job is complete. | — | | `202` | Accepted — async job was queued, or polled job is still in progress. | — | | `400` | Bad Request — your request body or parameters are invalid. | No — fix the request. | | `401` | Unauthorized — API key missing, malformed, or revoked. | No — check the key. | | `402` | Payment Required — insufficient credits for the operation. | No — top up credits. | | `403` | Forbidden — the resource exists but you don't own it. | No. | | `404` | Not Found — the job ID, session, or scraper doesn't exist. | No. | | `409` | Conflict — resource is locked or already exists (e.g., session in use, duplicate name). | After resolving the conflict. | | `422` | Unprocessable Entity — request was valid but the resource isn't ready (e.g., session not yet saved). | After the prerequisite is met. | | `429` | Too Many Requests — rate limit exceeded. | Yes — see [Rate limit handling](#rate-limit-handling). | | `500` | Internal Server Error — unexpected failure on our side. | Yes — exponential backoff. | | `502` | Bad Gateway — a downstream service is unreachable. | Yes — exponential backoff. | | `503` | Service Unavailable — a required service is temporarily down or unconfigured. | Yes — wait 30–60s. | --- ## Error code catalog The `error` field uses a fixed set of codes. The table below covers every code returned by the public API. ### Validation & input | Code | HTTP | When you'll see it | What to do | |------|------|--------------------|-----------| | `invalid_request` | 400 | Body is not valid JSON, a required field is missing, or a value is out of range (e.g., `depth > 5`, batch with 0 or >10 URLs, prompt >8KB, schema >50KB). | Inspect `message` for the specific field, fix the request, and resubmit. | | `invalid_url` | 400 | A URL in a batch request is malformed. | Fix the URL. The `message` indicates the index. | | `invalid_job_type` | 400 | The `job_type` field on `POST /v1/request` doesn't match a registered handler. | Use a supported value (`url_scraper`, `crawl`, `map`, `agentic_search`, `search`, `web_scraper`). | ### Auth & authorization | Code | HTTP | When you'll see it | What to do | |------|------|--------------------|-----------| | `unauthorized` | 401 | No API key was sent, or the key is malformed, revoked, or belongs to a deleted user. | Send a valid key in `X-API-Key` (or one of the [accepted header variants](#accepted-api-key-headers)). Generate a new key in the dashboard if needed. | | `forbidden` | 403 | The resource (job, session, scraper) exists but belongs to a different user. | Use a job ID from your own account. | ### Credits | Code | HTTP | When you'll see it | What to do | |------|------|--------------------|-----------| | `insufficient_credits` | 402 | Account balance is below the cost of the operation. The `message` includes the cost and your current balance. | Top up credits in [Billing](/buy-credits) or upgrade your plan. | ### Rate limiting | Code | HTTP | When you'll see it | What to do | |------|------|--------------------|-----------| | `rate_limit_exceeded` | 429 | You exceeded the per-endpoint rate limit. | See [Rate limit handling](#rate-limit-handling). | ### Resource state | Code | HTTP | When you'll see it | What to do | |------|------|--------------------|-----------| | `not_found` | 404 | Job ID, session ID, or scraper ID doesn't exist. | Verify the ID. Job IDs are valid for 30 days. | | `session_not_saved` | 422 | You tried to attach a saved browser session before its storage state was uploaded. | Run the manual save flow first (see [Browser Sessions](/docs/api-reference/browser-sessions/manual-sessions)). | | `session_in_use` | 409 | A saved session is already attached to an active automation. | Wait for the other run to finish, or use a different session. | | `duplicate_name` | 409 | A session name is already taken for this user. | Use a unique name. | ### Server-side | Code | HTTP | When you'll see it | What to do | |------|------|--------------------|-----------| | `server_error` | 500 | Unhandled error in our handler — usually a database or internal service issue. | Retry with backoff. If it persists, [contact support](#reporting-unexpected-errors) with the request ID. | | `queue_error` | 500 | Failed to enqueue the job (the job queue is temporarily unavailable). | Retry with backoff. | | `configuration_error` | 500 | A required service-side config is missing for this endpoint. | Retry; if persistent, contact support. | | `internal_error` | 500 | Generic catch-all for unexpected failures. | Retry with backoff. | | `search_error` | 500 | The upstream search provider returned an error. | Retry with backoff; reword the prompt if persistent. | | `service_unavailable` | 503 | A dependent service is temporarily offline. | Retry after 30–60 seconds. | > **Note on format consistency.** A small number of older endpoints currently return errors using slight variations of the format above (e.g., omitting `message`, or a default `{"statusCode": 400, "message": "..."}` shape for validation errors). Treat them as still conforming to the principle: a string `error` field is always present, and the HTTP status is authoritative. ### Accepted API key headers The API accepts the key under any of the following headers (and a few query params for WebSocket endpoints), in priority order: `X-API-Key`, `X-Api-Key`, `Api-Key`, `API-Key`, `X-Access-Key`, `Access-Key`, `apikey`, `api_key`, `Authorization` (with `Bearer `, `API-Key `, `ApiKey ` prefix or raw). For `/v1/browser-connect` (WebSocket): `?api_key=`, `?apikey=`, or `?token=` query parameters also work. --- ## Async job failures For async endpoints (`/v1/url-scraper`, `/v1/agentic-search`, `/v1/map`, `/v1/crawl`, Wire's `/v1/wire/task`), HTTP status `200`/`202` only confirms that polling is working. The actual outcome lives in the `status` field of the job response: | `status` | Meaning | |----------|---------| | `pending` | Queued, not yet picked up by a worker. | | `processing` | A worker is actively running the job. | | `completed` | Finished successfully — results are in the response. | | `failed` | The job ran but could not produce a result. See `error`. | A failed job response looks like this: ```json { "id": "job-abc123", "status": "failed", "error": "Blocked by website (HTTP 403)", "createdAt": "2025-04-30T18:12:04Z", "completedAt": "2025-04-30T18:12:34Z", "durationMs": 30000 } ``` The `error` field is a free-form, human-readable string. Common substrings to switch on if you must: | Substring | Cause | Suggested fix | |-----------|-------|---------------| | `Blocked by website`, `HTTP 403`, `HTTP 429`, `bot detection`, `CAPTCHA` | Anti-bot protection. | Set `useBrowser: true` and/or specify a `country`. Try a [browser session](/docs/api-reference/browser-sessions/manual-sessions) for sites that require login. | | `Connection timeout`, `timeout` | Page didn't finish loading in time. | For SPAs, set `useBrowser: true` and increase wait time. | | `DNS resolution failed`, `no such host` | The domain can't be resolved. | Verify the URL is reachable. | | `TLS`, `SSL` | Certificate validation failure. | Confirm the target uses a trusted certificate. | | `Invalid URL` | Malformed URL passed all the way through. | Pre-validate URLs client-side. | > **Batch jobs.** A batch URL scraper job is `completed` if any child finishes — partial failures don't fail the parent. Iterate `results[]` and check each child's `status` and `error`. --- ## Retry guidance ### When to retry | Status | Retry? | Why | |--------|--------|-----| | `400`, `401`, `402`, `403`, `404`, `409`, `422` | **No** | The request itself is the problem. Retrying will return the same error. | | `429` | **Yes** | Transient — the bucket refills. Read `Retry-After` if present, otherwise back off. | | `500`, `502`, `503` | **Yes** | Transient server-side issue. Cap retries at 3–5 and use exponential backoff with jitter. | | Network errors (no response) | **Yes** | Treat the same as `5xx`. | ### Recommended pattern: exponential backoff with jitter Jitter spreads retries from many clients so a thundering herd doesn't synchronize. Cap the total wait so a stuck worker fails fast instead of looping forever. <Tabs items={["Python", "Node.js"]}> <Tab value="Python"> ```python import random import time import requests RETRYABLE = {429, 500, 502, 503} MAX_ATTEMPTS = 5 BASE_DELAY = 1.0 # seconds MAX_DELAY = 30.0 def request_with_retry(method, url, *, headers=None, json=None): """POST/GET with exponential backoff + jitter on retryable failures.""" for attempt in range(MAX_ATTEMPTS): try: response = requests.request(method, url, headers=headers, json=json, timeout=30) except requests.RequestException: if attempt == MAX_ATTEMPTS - 1: raise time.sleep(_backoff(attempt)) continue if response.status_code not in RETRYABLE: return response # Honor server-sent Retry-After when present retry_after = response.headers.get("Retry-After") delay = float(retry_after) if retry_after else _backoff(attempt) if attempt == MAX_ATTEMPTS - 1: return response # caller decides what to do time.sleep(delay) return response def _backoff(attempt: int) -> float: """Full-jitter exponential backoff, capped at MAX_DELAY.""" cap = min(MAX_DELAY, BASE_DELAY * (2 ** attempt)) return random.uniform(0, cap) # Usage resp = request_with_retry( "POST", "https://api.anakin.io/v1/url-scraper", headers={"X-API-Key": "ak-your-key-here"}, json={"url": "https://example.com"}, ) resp.raise_for_status() print(resp.json()["jobId"]) ``` </Tab> <Tab value="Node.js"> ```javascript const RETRYABLE = new Set([429, 500, 502, 503]); const MAX_ATTEMPTS = 5; const BASE_DELAY = 1000; // ms const MAX_DELAY = 30_000; async function requestWithRetry(url, init = {}) { for (let attempt = 0; attempt < MAX_ATTEMPTS; attempt++) { let response; try { response = await fetch(url, init); } catch (err) { if (attempt === MAX_ATTEMPTS - 1) throw err; await sleep(backoff(attempt)); continue; } if (!RETRYABLE.has(response.status)) return response; // Honor server-sent Retry-After when present const retryAfter = response.headers.get("Retry-After"); const delay = retryAfter ? Number(retryAfter) * 1000 : backoff(attempt); if (attempt === MAX_ATTEMPTS - 1) return response; await sleep(delay); } } function backoff(attempt) { const cap = Math.min(MAX_DELAY, BASE_DELAY * 2 ** attempt); return Math.random() * cap; // full jitter } const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); // Usage const res = await requestWithRetry("https://api.anakin.io/v1/url-scraper", { method: "POST", headers: { "X-API-Key": "ak-your-key-here", "Content-Type": "application/json", }, body: JSON.stringify({ url: "https://example.com" }), }); const { jobId } = await res.json(); console.log(jobId); ``` </Tab> </Tabs> --- ## Rate limit handling Per-endpoint limits are documented on the [Rate Limits](/docs/documentation/rate-limits) page. The short version: most submit endpoints allow **60 requests/min per user**; AI evaluation is **10/min**; GET polling endpoints are not rate-limited. ### Response headers When a request is rate-limited, the API returns `429 Too Many Requests` with the standard error body. Some 429 responses (notably `/v1/browser-connect` over the limit on concurrent CDP sessions) include a `Retry-After` header indicating seconds to wait: ```http HTTP/1.1 429 Too Many Requests Retry-After: 5 Content-Type: application/json {"error": "rate_limit_exceeded", "message": "Too many requests. Please try again later."} ``` > **Heads up.** AnakinScraper does **not** currently emit the optional `X-RateLimit-Limit`, `X-RateLimit-Remaining`, or `X-RateLimit-Reset` headers. Don't rely on them — drive your retry loop off `Retry-After` (when present) or your own backoff. Surfacing these headers is on our roadmap. ### Reading `Retry-After` The `requestWithRetry` helpers above already honor `Retry-After`. If you only need to handle 429 specifically: <Tabs items={["Python", "Node.js"]}> <Tab value="Python"> ```python import time import requests resp = requests.post( "https://api.anakin.io/v1/url-scraper", headers={"X-API-Key": "ak-your-key-here"}, json={"url": "https://example.com"}, ) if resp.status_code == 429: wait = int(resp.headers.get("Retry-After", "5")) time.sleep(wait) resp = requests.post(...) # retry ``` </Tab> <Tab value="Node.js"> ```javascript let res = await fetch("https://api.anakin.io/v1/url-scraper", { method: "POST", ... }); if (res.status === 429) { const wait = Number(res.headers.get("Retry-After") ?? 5); await new Promise((r) => setTimeout(r, wait * 1000)); res = await fetch(...); // retry } ``` </Tab> </Tabs> --- ## Troubleshooting The following scenarios cover the failures we see most often in support tickets. ### "I'm getting 403 from the target site" The site has bot detection. Two levers, in order of effectiveness: 1. **Switch to the browser handler.** Add `"useBrowser": true` to your request — this routes through our managed, fingerprint-masked browser engine instead of plain HTTP. 2. **Set a country.** Add `"country": "US"` (or another ISO code) — a residential IP is picked from that region. If both fail, the site likely requires a logged-in session. Use [Browser Sessions](/docs/api-reference/browser-sessions/manual-sessions) to capture cookies once, then attach the session by ID. ### "Timeouts on a single-page app" Plain HTTP can't run JavaScript. Set `"useBrowser": true` so the scraper executes the page's JS before extracting content. For very slow SPAs, also increase `"waitForSelector"` or `"waitMs"` if your endpoint supports them. ### "Schema extraction returns the wrong fields" Agentic Search and JSON extraction are LLM-driven — better prompts and tighter schemas produce better output: - **Be explicit** in the prompt: name each field and describe its expected shape (e.g., "extract `price` as a number in USD, no currency symbol"). - **Provide examples** in the prompt for ambiguous fields. - **Tighten the schema.** Required JSON Schema fields force the model to produce them; optional fields tend to get omitted. - **Cap the schema at 50KB.** Larger schemas are rejected with `invalid_request`. ### "Job stuck in pending" A few possibilities: - **You're polling the wrong endpoint.** `POST /v1/url-scraper` returns a `jobId` you poll at `GET /v1/url-scraper/{id}`. The list is in [Polling Jobs](/docs/api-reference/polling-jobs). - **Worker fleet is saturated.** Pending → processing usually takes <5s. If it's been >60s, retry or [contact support](#reporting-unexpected-errors). - **The job died silently.** Stale jobs are auto-marked `failed` after 1 hour. If you see this, check the `error` field for the cause. ### "402 insufficient_credits when I just topped up" Credits are deducted on **completion**, but checked **upfront**. If you submitted a batch of 10 URLs and have 8 credits, the batch is rejected immediately even though some URLs would have come from cache (which costs 0). Top up enough for the worst case. ### "Got a 401 with a brand-new key" API keys take a few seconds to propagate. If a freshly-created key returns 401, wait 5–10 seconds and retry. If it persists, regenerate the key in the dashboard. ### "Different services return slightly different error shapes" A small number of older endpoints use minor variations on the canonical format. The HTTP status is always authoritative; the body always contains a string `error` field. Plan your error handling around the status code first, then the `error` code. ### "Wire job returned a `429` even though I've only made a few requests" `GET /v1/wire/jobs/{id}` is capped at **60/min per user** — unlike URL Scraper, which is unlimited. If you're polling many Wire jobs in parallel, stagger them or reduce poll frequency. See [Rate Limits](/docs/documentation/rate-limits) for the per-endpoint table. ### "Browser Connect closed unexpectedly" The service returns 429 once you have 50 concurrent CDP sessions. Close sessions when done and retry on `Retry-After`. If you saved a session, it must finish saving before another connection can attach to it (otherwise you'll see `session_not_saved`). --- ## Reporting unexpected errors If you hit a `500`, an unfamiliar `error` code, or behavior that contradicts this page: - **Capture the request:** method, URL, headers (redact the API key), body. - **Capture the response:** status, headers, body. - **Note the time** (UTC, to the second) — this lets us correlate against server logs. - **Email** support@anakin.io with the above. For Enterprise customers, see your dedicated channel. --- # Map (URL Discovery) (/docs/api-reference/map) The Map API discovers all URLs on a website. It fetches the page, parses sitemaps, and extracts links to build a complete URL inventory. Use it for site audits, content indexing, or as a precursor to [Crawl](/docs/api-reference/crawl). ### Features - **Sitemap + link extraction** — combines sitemap.xml parsing with in-page link discovery - **Subdomain support** — optionally include links to subdomains - **Search filter** — filter discovered URLs by keyword - **Browser rendering** — use headless Chrome for JS-rendered pages - **Up to 5,000 URLs** per request ### Endpoints <EndpointCard method="POST" path="/v1/map" description="Submit Map Job — discover URLs on a website" href="/docs/api-reference/map/submit-map-job" /> <EndpointCard method="GET" path="/v1/map/{id}" description="Get Map Result — retrieve discovered URLs" href="/docs/api-reference/map/get-map-result" /> --- # Website Monitoring (/docs/api-reference/monitoring) The Website Monitoring API watches a URL on a schedule, captures a snapshot on every check, and records a **change** whenever the page differs from the previous snapshot. Each detected change can fire a signed **webhook** and/or **email** alert. It reuses the same scraping pipeline as the [URL Scraper](/docs/api-reference/url-scraper) (proxies, browser rendering, authenticated sessions), so anything you can scrape, you can monitor. A monitor's **`scope`** is `page` *(default)* — watch one URL — `site` — crawl the whole site each run and track pages **added**, **removed**, and **changed** (see [Site monitors](/docs/api-reference/monitoring/site-monitors)) — or `wire` — run a managed [Wire](/docs/api-reference/monitoring/wire-monitors) API action each run and diff the JSON it returns, for sites with no public API of their own (see [Wire monitors](/docs/api-reference/monitoring/wire-monitors)). ### How it works 1. **Create a monitor** with a URL, a schedule (`intervalMinutes`), and what to watch. 2. On every check the monitor scrapes the page and stores a **snapshot**. 3. If the new snapshot differs from the last one, a **change** is recorded. 4. If alerts are configured, the change is delivered to your webhook / email. ### What to watch | `watchMode` | What it compares | Cost / check | |-------------|------------------|--------------| | `full_page` *(default)* | The whole page body, in the `watchFormat` you choose (`markdown`, `html`, or `cleaned_html`). | 2 credits | | `specific_data` | Only the fields in your `outputSchema` (JSON Schema), extracted with AI. Ideal for price / stock / status tracking. | 3 credits | Enabling **AI meaningful-change** (`aiMode: true`) filters out trivial noise (ads, timestamps) and summarizes real changes — it adds **+1 credit per check**. ### The Monitor object ```json { "id": "b1e7…", "userId": "u_123", "url": "https://example.com/product/123", "scope": "page", "maxPages": null, "maxDepth": null, "includePatterns": [], "excludePatterns": [], "excludedUrls": [], "watchMode": "specific_data", "watchFormat": "markdown", "useBrowser": false, "country": "us", "sessionId": null, "sessionExpired": false, "outputSchema": { "type": "object", "properties": { "price": { "type": "number" } }, "required": ["price"] }, "aiMode": true, "aiGoal": "only when the price drops or it goes out of stock", "intervalMinutes": 60, "isActive": true, "nextRunAt": "2026-01-01T12:00:00Z", "lastCheckedAt": "2026-01-01T11:00:00Z", "creditCostPerRun": 4, "alertWebhookUrl": "https://your-app.com/hooks/anakin", "alertWebhookSecret": "whsec_…", "alertEmails": "you@example.com", "lastAlertedAt": "2026-01-01T11:00:00Z", "createdAt": "2025-12-01T09:00:00Z", "updatedAt": "2025-12-15T09:00:00Z" } ``` | Field | Type | Description | |-------|------|-------------| | `id` | string | Monitor ID. | | `url` | string | The monitored URL (root URL for `site` scope; the Wire site's URL for `wire` scope). | | `scope` | string | `page`, `site`, or `wire`. See [Site monitors](/docs/api-reference/monitoring/site-monitors) and [Wire monitors](/docs/api-reference/monitoring/wire-monitors). | | `maxPages`, `maxDepth` | number \| null | Site scope: pages-per-run cap and crawl depth. | | `includePatterns`, `excludePatterns`, `excludedUrls` | string[] | Site scope: pages to track (hand-picked full URLs, crawled directly) and/or include/exclude globs, plus pages removed from tracking. | | `wireActionId` | string \| null | Wire scope: the Wire action run each check (e.g. `amazon.search_products`). | | `wireCatalogSlug` | string \| null | Wire scope: catalogue slug of the Wire site. | | `wireCredentialId` | string \| null | Wire scope: connected credential id, when the action needs auth. | | `wireParams` | object \| null | Wire scope: parameters passed to the action. | | `wireWatchPaths` | string[] \| null | Wire scope: JSON paths to diff instead of the whole response. | | `watchMode` | string | `full_page` or `specific_data`. | | `watchFormat` | string | For `full_page`: `markdown`, `html`, or `cleaned_html`. Always `markdown` for `specific_data`. | | `useBrowser` | boolean | Whether checks render with a headless browser. Forced `true` when a `sessionId` is set. | | `country` | string | Proxy country code. See [Supported Countries](/docs/api-reference/supported-countries). | | `sessionId` | string \| null | Saved [browser session](/docs/api-reference/browser-sessions) used for authenticated pages. | | `sessionExpired` | boolean | `true` when the saved session is no longer valid (the monitor is paused until re-auth). | | `outputSchema` | object \| null | JSON Schema of the tracked fields (`specific_data` mode). | | `aiMode` | boolean | AI meaningful-change filtering is on. | | `aiGoal` | string \| null | Optional natural-language description of which changes count as meaningful. | | `intervalMinutes` | number | Check frequency. Minimum `15`. | | `isActive` | boolean | Whether the monitor is running (`false` = paused). | | `expiresAt` | string \| null | Optional end date; the monitor auto-pauses when it passes. | | `expired` | boolean | `true` once the end date has passed (cleared on resume / new expiry). | | `nextRunAt` | string | ISO 8601 timestamp of the next scheduled check. | | `lastCheckedAt` | string \| null | ISO 8601 timestamp of the most recent check. | | `creditCostPerRun` | number | Credits deducted per check (2/3 by mode, +1 with `aiMode`). | | `alertWebhookUrl` | string \| null | Webhook that receives change alerts. | | `alertWebhookSecret` | string \| null | HMAC signing secret for the webhook (see [Alerts](/docs/api-reference/monitoring/alerts)). | | `alertEmails` | string \| null | Comma-separated email recipients. | | `lastAlertedAt` | string \| null | ISO 8601 timestamp of the last alert sent. | ### Limits - **Minimum interval:** 15 minutes. - **Active monitors per plan:** Free `5`, Pro `20`, Scale `100`. Creating or resuming past the cap returns `402 monitor_limit_reached`. ### Endpoints <EndpointCard method="POST" path="/v1/monitors" description="Create a monitor" href="/docs/api-reference/monitoring/create-monitor" /> <EndpointCard method="GET" path="/v1/monitors" description="List monitors, or get one by ID" href="/docs/api-reference/monitoring/list-monitors" /> <EndpointCard method="PUT" path="/v1/monitors/{id}" description="Update or delete a monitor" href="/docs/api-reference/monitoring/update-monitor" /> <EndpointCard method="POST" path="/v1/monitors/{id}/run" description="Pause, resume, or run a check now" href="/docs/api-reference/monitoring/control-monitor" /> <EndpointCard method="GET" path="/v1/monitors/{id}/changes" description="Snapshots & detected changes" href="/docs/api-reference/monitoring/snapshots-and-changes" /> <EndpointCard method="POST" path="/v1/monitors/{id}/test-alert" description="Alerts, deliveries & webhook signing" href="/docs/api-reference/monitoring/alerts" /> --- # Polling Jobs (/docs/api-reference/polling-jobs) Most AnakinScraper endpoints process jobs asynchronously. After submitting a request, you receive a `jobId` and poll a GET endpoint until the job completes. > **Skip polling with webhooks** — every async endpoint accepts an optional `webhook_url`, and you can register account-level endpoints that receive a signed `job.completed` / `job.failed` event the moment a job settles. See [Webhooks](/docs/api-reference/webhooks). --- ### How it works 1. **Submit** a POST request — you receive a `jobId` with status `pending` 2. **Poll** the corresponding GET endpoint with the `jobId` 3. **Check** the `status` field — repeat until `completed` or `failed` | Product | Submit | Poll | |---------|--------|------| | Wire | `POST /v1/wire/task` | `GET /v1/wire/jobs/{id}` | | URL Scraper | `POST /v1/url-scraper` | `GET /v1/url-scraper/{id}` | | URL Scraper (batch) | `POST /v1/url-scraper/batch` | `GET /v1/url-scraper/{id}` | | Agentic Search | `POST /v1/agentic-search` | `GET /v1/agentic-search/{id}` | > **Search API** (`POST /v1/search`) is synchronous — results are returned immediately, no polling needed. --- ### Recommended polling interval | Product | Interval | Typical completion | |---------|----------|--------------------| | Wire | 2–5 seconds | 3–30 seconds | | URL Scraper | 2–5 seconds | 3–15 seconds | | Agentic Search | 10 seconds | 1–5 minutes | --- ### Polling examples <Tabs items={["Python", "JavaScript"]}> <Tab value="Python"> ```python import requests import time def poll_job(endpoint, job_id, api_key, interval=5, timeout=300): """Poll a job until completed or failed.""" elapsed = 0 while elapsed < timeout: result = requests.get( f'https://api.anakin.io/v1/{endpoint}/{job_id}', headers={'X-API-Key': api_key} ) data = result.json() if data['status'] == 'completed': return data if data['status'] == 'failed': raise Exception(data.get('error', 'Job failed')) time.sleep(interval) elapsed += interval raise TimeoutError('Job polling timed out') # URL Scraper result = poll_job('url-scraper', 'job_abc123xyz', 'your_api_key', interval=3) print(result['markdown']) # Agentic Search (longer interval) result = poll_job('agentic-search', 'agentic_abc123xyz', 'your_api_key', interval=10, timeout=600) print(result['markdown']) ``` </Tab> <Tab value="JavaScript"> ```javascript async function pollJob(endpoint, jobId, apiKey, interval = 5000, timeout = 300000) { const start = Date.now(); while (Date.now() - start < timeout) { const res = await fetch(`https://api.anakin.io/v1/${endpoint}/${jobId}`, { headers: { 'X-API-Key': apiKey } }); const data = await res.json(); if (data.status === 'completed') return data; if (data.status === 'failed') throw new Error(data.error || 'Job failed'); await new Promise(r => setTimeout(r, interval)); } throw new Error('Job polling timed out'); } // URL Scraper const result = await pollJob('url-scraper', 'job_abc123xyz', 'your_api_key', 3000); console.log(result.markdown); // Agentic Search (longer interval) const report = await pollJob('agentic-search', 'agentic_abc123xyz', 'your_api_key', 10000, 600000); console.log(report.markdown); ``` </Tab> </Tabs> --- ### Status values | Status | Description | |--------|-------------| | `pending` | Job is queued, not yet started | | `queued` | Job is waiting for a worker (agentic search only) | | `processing` | Job is actively being processed | | `completed` | Results are ready — stop polling | | `failed` | Job encountered an error — stop polling | --- # Search API (/docs/api-reference/search) AI-powered web search that returns structured results with citations, snippets, and relevance scores. Results are returned immediately (synchronous). ### Features - **Synchronous** — results returned instantly, no polling needed - **AI-generated summaries** — get an answer alongside raw results - **Citations with scores** — relevance-ranked results with snippets ### Endpoints <EndpointCard method="POST" path="/v1/search" description="Search — perform an AI-powered web search" href="/docs/api-reference/search/search" /> --- # Supported Countries & Territories (/docs/api-reference/supported-countries) Use the `country` parameter in your API requests to route through a specific location. Codes follow ISO 3166-1 alpha-2 (lowercase). | Country | Code | |---------|------| | Afghanistan | `af` | | Aland | `ax` | | Albania | `al` | | Algeria | `dz` | | Andorra | `ad` | | Angola | `ao` | | Antarctica | `aq` | | Antigua and Barbuda | `ag` | | Argentina | `ar` | | Armenia | `am` | | Aruba | `aw` | | Australia | `au` | | Austria | `at` | | Azerbaijan | `az` | | Bahamas | `bs` | | Bahrain | `bh` | | Bangladesh | `bd` | | Barbados | `bb` | | Belarus | `by` | | Belgium | `be` | | Belize | `bz` | | Benin | `bj` | | Bermuda | `bm` | | Bhutan | `bt` | | Bolivia | `bo` | | Bonaire | `bq` | | Bosnia and Herzegovina | `ba` | | Botswana | `bw` | | Brazil | `br` | | British Indian Ocean Territory | `io` | | British Virgin Islands | `vg` | | Brunei | `bn` | | Bulgaria | `bg` | | Burkina Faso | `bf` | | Cambodia | `kh` | | Cameroon | `cm` | | Canada | `ca` | | Cape Verde | `cv` | | Cayman Islands | `ky` | | Chile | `cl` | | China | `cn` | | Colombia | `co` | | Cook Islands | `ck` | | Costa Rica | `cr` | | Croatia | `hr` | | Cuba | `cu` | | Curacao | `cw` | | Cyprus | `cy` | | Czech Republic | `cz` | | Democratic Republic of the Congo | `cd` | | Denmark | `dk` | | Djibouti | `dj` | | Dominican Republic | `do` | | East Timor | `tl` | | Ecuador | `ec` | | Egypt | `eg` | | El Salvador | `sv` | | Estonia | `ee` | | Eswatini | `sz` | | Ethiopia | `et` | | Faroe Islands | `fo` | | Fiji | `fj` | | Finland | `fi` | | France | `fr` | | French Guiana | `gf` | | French Polynesia | `pf` | | Gabon | `ga` | | Georgia | `ge` | | Germany | `de` | | Ghana | `gh` | | Gibraltar | `gi` | | Greece | `gr` | | Greenland | `gl` | | Grenada | `gd` | | Guadeloupe | `gp` | | Guam | `gu` | | Guatemala | `gt` | | Guernsey | `gg` | | Guinea | `gn` | | Guyana | `gy` | | Haiti | `ht` | | Honduras | `hn` | | Hong Kong | `hk` | | Hungary | `hu` | | Iceland | `is` | | India | `in` | | Indonesia | `id` | | Iran | `ir` | | Iraq | `iq` | | Ireland | `ie` | | Isle of Man | `im` | | Israel | `il` | | Italy | `it` | | Ivory Coast | `ci` | | Jamaica | `jm` | | Japan | `jp` | | Jersey | `je` | | Jordan | `jo` | | Kazakhstan | `kz` | | Kenya | `ke` | | Kosovo | `xk` | | Kuwait | `kw` | | Kyrgyzstan | `kg` | | Laos | `la` | | Latvia | `lv` | | Lebanon | `lb` | | Lesotho | `ls` | | Liberia | `lr` | | Libya | `ly` | | Liechtenstein | `li` | | Lithuania | `lt` | | Luxembourg | `lu` | | Macao | `mo` | | Madagascar | `mg` | | Malawi | `mw` | | Malaysia | `my` | | Maldives | `mv` | | Mali | `ml` | | Malta | `mt` | | Martinique | `mq` | | Mauritania | `mr` | | Mauritius | `mu` | | Mexico | `mx` | | Micronesia | `fm` | | Moldova | `md` | | Monaco | `mc` | | Mongolia | `mn` | | Montenegro | `me` | | Montserrat | `ms` | | Morocco | `ma` | | Mozambique | `mz` | | Myanmar (Burma) | `mm` | | Namibia | `na` | | Nepal | `np` | | Netherlands | `nl` | | New Caledonia | `nc` | | New Zealand | `nz` | | Nicaragua | `ni` | | Niger | `ne` | | Nigeria | `ng` | | North Macedonia | `mk` | | Northern Mariana Islands | `mp` | | Norway | `no` | | Oman | `om` | | Pakistan | `pk` | | Palestine | `ps` | | Panama | `pa` | | Papua New Guinea | `pg` | | Paraguay | `py` | | Peru | `pe` | | Philippines | `ph` | | Poland | `pl` | | Portugal | `pt` | | Puerto Rico | `pr` | | Qatar | `qa` | | Republic of the Congo | `cg` | | Reunion | `re` | | Romania | `ro` | | Russia | `ru` | | Rwanda | `rw` | | Saint Kitts and Nevis | `kn` | | Saint Lucia | `lc` | | Saint Martin | `mf` | | Saint Vincent and the Grenadines | `vc` | | Sao Tome and Principe | `st` | | Saudi Arabia | `sa` | | Senegal | `sn` | | Serbia | `rs` | | Sierra Leone | `sl` | | Singapore | `sg` | | Sint Maarten | `sx` | | Slovakia | `sk` | | Slovenia | `si` | | Solomon Islands | `sb` | | Somalia | `so` | | South Africa | `za` | | South Korea | `kr` | | Spain | `es` | | Sri Lanka | `lk` | | Suriname | `sr` | | Sweden | `se` | | Switzerland | `ch` | | Syria | `sy` | | Taiwan | `tw` | | Tajikistan | `tj` | | Tanzania | `tz` | | Thailand | `th` | | Togo | `tg` | | Tonga | `to` | | Trinidad and Tobago | `tt` | | Tunisia | `tn` | | Turkey | `tr` | | Turks and Caicos Islands | `tc` | | U.S. Virgin Islands | `vi` | | Uganda | `ug` | | Ukraine | `ua` | | United Arab Emirates | `ae` | | United Kingdom | `gb` | | United States | `us` | | Uruguay | `uy` | | Uzbekistan | `uz` | | Vanuatu | `vu` | | Venezuela | `ve` | | Vietnam | `vn` | | Yemen | `ye` | | Zambia | `zm` | | Zimbabwe | `zw` | --- ### Usage Example ```json { "url": "https://example.com", "country": "jp" } ``` This routes the request through a residential proxy in Japan. --- ### Programmatic Access You can also fetch this list programmatically: ```bash curl https://api.anakin.io/v1/countries ``` Returns a JSON array of all supported countries with their codes. --- # URL Scraper (/docs/api-reference/url-scraper) The URL Scraper is the core scraping API. Submit any URL and receive the scraped HTML, markdown, and optionally AI-extracted JSON data. Supports single URL and batch (up to 10 URLs) modes. ### Features - **Single & batch** scraping in one API - **30x faster** with intelligent caching - **Zero blocks** with anti-detection and proxy routing across [207 countries and territories](/docs/api-reference/supported-countries) - **AI JSON extraction** — structured data from any page - **Browser mode** — headless Chrome for JS-heavy sites and SPAs ### Endpoints <EndpointCard method="POST" path="/v1/url-scraper" description="Submit Scrape Job — scrape a single URL" href="/docs/api-reference/url-scraper/submit-scrape-job" /> <EndpointCard method="POST" path="/v1/url-scraper/scrape" description="Scrape — scrape a single URL and get the result inline (no polling)" href="/docs/api-reference/url-scraper/scrape" /> <EndpointCard method="POST" path="/v1/url-scraper/batch" description="Batch URL Scraping — scrape up to 10 URLs" href="/docs/api-reference/url-scraper/batch-url-scraping" /> <EndpointCard method="GET" path="/v1/url-scraper/{id}" description="Get Job Status & Results — poll for results" href="/docs/api-reference/url-scraper/get-job-status" /> --- # Webhooks (/docs/api-reference/webhooks) Every asynchronous AnakinScraper product can `POST` a **signed JSON event** to your server the moment work finishes — no polling loop required. One request format, one signature scheme, one [event taxonomy](/docs/api-reference/webhooks/events) across URL scraping, crawling, mapping, agentic search, Wire, website monitoring, and AI Visibility. > Webhooks complement [polling](/docs/api-reference/polling-jobs) — the poll endpoints keep working exactly as before. Use webhooks when you'd rather be told than ask. > **Results are delivered by reference, not by value.** Every terminal event carries `data.result_url` — an authenticated `GET` endpoint for the complete result, fetched with your API key. Small results are *also* inlined as `data.result` as a convenience, but only while the whole envelope stays under **256 KB**; past that the inline copy is dropped (`resultOmitted: "size"`), and some products (AI Visibility) never inline full results at all. **Build your receiver around `result_url`** — the inline copy is an optimization, not a contract. See [Event Types & Payloads](/docs/api-reference/webhooks/events) for the fetch pattern per product. --- ## Two ways to subscribe ### 1. Per-request `webhook_url` Pass an optional `webhook_url` in the body of any async submit endpoint and we notify that URL when **that job** settles: | Endpoint | Events delivered | |---|---| | [`POST /v1/url-scraper`](/docs/api-reference/url-scraper/submit-scrape-job) | `job.completed` / `job.failed` | | [`POST /v1/url-scraper/batch`](/docs/api-reference/url-scraper/batch-url-scraping) | `batch.completed` / `batch.failed` | | `POST /v1/web-scraper` | `job.completed` / `job.failed` | | [`POST /v1/map`](/docs/api-reference/map/submit-map-job) | `job.completed` / `job.failed` | | [`POST /v1/crawl`](/docs/api-reference/crawl/submit-crawl-job) | `job.completed` / `job.failed` | | [`POST /v1/agentic-search`](/docs/api-reference/agentic-search/submit-search) | `job.completed` / `job.failed` | | [`POST /v1/wire/task`](/docs/api-reference/wire/execute-task) | `wire.job.completed` / `wire.job.failed` | Per-request deliveries are signed with your account's **default signing secret** — fetch it with [`GET /v1/webhooks/signing-secret`](/docs/api-reference/webhooks/managing-endpoints#signing-secret), rotate it with [`POST /v1/webhooks/signing-secret/rotate`](/docs/api-reference/webhooks/managing-endpoints#signing-secret). ### 2. Registered endpoints Account-level destinations that receive **every matching event** across all products, with optional event filters — point one receiver at everything, or split events across services. Each endpoint has its **own** `whsec_…` secret; an empty filter subscribes to all events, and you can register up to **10** endpoints per account. Manage them in the dashboard (**Account → Webhooks**) or via the [management API](/docs/api-reference/webhooks/managing-endpoints). > **Sync endpoints emit events too** — `POST /v1/search` and `POST /v1/url-scraper/scrape` return results inline, but still emit `job.completed` / `job.failed` events to registered endpoints, so one receiver can observe every job uniformly. (`webhook_url` itself is not accepted on `POST /v1/search`.) --- ## Request format Every delivery is an HTTP `POST` to your URL with these headers: | Header | Example | Notes | |---|---|---| | `Content-Type` | `application/json` | Body is JSON. | | `User-Agent` | `AnakinScraper-Webhooks/1` | Identifies our sender. | | `X-Anakin-Timestamp` | `1783009089` | Unix seconds — **fresh on every attempt**. | | `X-Anakin-Delivery-Id` | `d81f64c2-…` | **Stable across retries** — use it to de-duplicate. | | `X-Anakin-Event` | `job.completed` | The event type — same as `type` in the body. | | `X-Anakin-Signature` | `sha256=7987…` | `sha256=` + hex `HMAC_SHA256(secret, rawBody)`. | The body is the [event envelope](/docs/api-reference/webhooks/events): ```json { "id": "evt_a1b2c3…", "type": "job.completed", "createdAt": "2026-07-13T10:00:00Z", "data": { } } ``` The one exception is `monitor.change`, which keeps its original flat payload for backward compatibility — see [Event Types](/docs/api-reference/webhooks/events#monitorchange). Per-request timeout is **10 seconds** — acknowledge quickly (see [Best practices](#best-practices)). ### Verify the signature Your webhook URL is public: anyone who learns it could `POST` forged events. So every delivery is **HMAC-SHA256 signed** with a secret only you and we hold — the secret never travels in the request, only the signature does. Verification happens **on your side**: recompute the signature from the received body and your copy of the secret, then compare. Match → genuinely from us, untampered → process it. No match → forged or wrong secret → reject it (return `401`, ignore the body). Verifying is optional but strongly recommended for any endpoint that acts on the data. Which secret signs a delivery: | Delivery | Signed with | |---|---| | To a registered endpoint | That endpoint's own `whsec_…` secret | | Per-request `webhook_url` | Your account's [default signing secret](/docs/api-reference/webhooks/managing-endpoints#signing-secret) | The signature is computed over the **exact raw bytes** of the body, so verify against the raw body, never a re-serialized object (JSON key order or whitespace would differ and the check would fail). <Tabs items={["Node (Express)", "Python (Flask)"]}> <Tab value="Node (Express)"> ```js import express from "express" import crypto from "crypto" const app = express() app.use("/anakin-webhook", express.raw({ type: "application/json" })) // raw bytes, not parsed app.post("/anakin-webhook", (req, res) => { const secret = process.env.ANAKIN_WEBHOOK_SECRET // whsec_… (endpoint secret or default signing secret) const got = req.header("X-Anakin-Signature") || "" const expected = "sha256=" + crypto.createHmac("sha256", secret).update(req.body).digest("hex") const ok = got.length === expected.length && crypto.timingSafeEqual(Buffer.from(got), Buffer.from(expected)) // constant-time compare if (!ok) return res.status(401).send("bad signature") const event = JSON.parse(req.body.toString()) // de-dupe on req.header("X-Anakin-Delivery-Id"), then process asynchronously res.status(200).send("ok") }) ``` </Tab> <Tab value="Python (Flask)"> ```python import hmac, hashlib, os from flask import Flask, request, abort app = Flask(__name__) @app.post("/anakin-webhook") def webhook(): secret = os.environ["ANAKIN_WEBHOOK_SECRET"].encode() raw = request.get_data() # raw bytes — do NOT use request.json here expected = "sha256=" + hmac.new(secret, raw, hashlib.sha256).hexdigest() got = request.headers.get("X-Anakin-Signature", "") if not hmac.compare_digest(expected, got): # constant-time compare abort(401) event = request.get_json() # de-dupe on request.headers.get("X-Anakin-Delivery-Id"), then process return "", 200 ``` </Tab> </Tabs> --- ## Delivery, retries & idempotency - **Success:** your endpoint returns **2xx** → recorded as `success`. - **Permanent failure (no retry):** any **4xx other than 408 / 429** (e.g. 400, 401, 404) → we assume the endpoint is misconfigured and stop → recorded as `exhausted`. - **Retried with backoff:** a connection error / timeout, any **5xx**, a **408**, or a **429** → retried up to **6 attempts** (≈8.5 h total), then marked `exhausted`: | After attempt | Wait before next | |---|---| | 1 | 1 min | | 2 | 5 min | | 3 | 30 min | | 4 | 2 h | | 5 | 6 h | - **Each attempt** times out after **10 seconds** and follows at most **3 redirects**. - **At-least-once — de-duplicate.** A delivery can arrive more than once (e.g. you processed it but replied slowly, so we retried). De-dupe on `X-Anakin-Delivery-Id`, which is stable across retries. - **Delivery log & resend.** Every attempt is recorded in your account-wide [delivery log](/docs/api-reference/webhooks/managing-endpoints#delivery-log), retained for **30 days**. `failed` / `exhausted` deliveries can be resent — byte-identical body and signature — from the dashboard or via [`POST /v1/webhooks/deliveries/{id}/resend`](/docs/api-reference/webhooks/managing-endpoints#delivery-log). --- ## Callback URLs - **HTTPS is strongly recommended**; plain `http` is allowed. - Maximum length is **2048 characters**. - **Private and internal addresses are rejected** (e.g. `localhost`, private IP ranges). --- ## Best practices - **Verify the signature** with a constant-time compare, over the **raw** body. - **Respond 2xx fast** (within 10s). Acknowledge first, do heavy work asynchronously — slow endpoints get retried and look like failures. - **Be idempotent** — key processing on `X-Anakin-Delivery-Id`. - **Return the right status** — 2xx = accepted; 5xx / 408 / 429 to ask for a retry; any other 4xx tells us to stop permanently, so use it only for genuinely bad requests. - **Don't rely on inline results** — `data.result` is inlined only when small, and AI Visibility events never inline full answers; `data.result_url` is always present. Fetch it whenever `resultOmitted` is set (or whenever there's no inline `result`). - **Keep secrets secret** — treat them like passwords; [rotate](/docs/api-reference/webhooks/managing-endpoints#signing-secret) the default secret if it leaks. - **Don't assume ordering** — deliveries can overlap under retries; use `createdAt` / `X-Anakin-Timestamp` if you need sequence. --- ## In this section - [Event Types & Payloads](/docs/api-reference/webhooks/events) — the envelope, the full taxonomy, and per-event `data` fields with examples - [Managing Endpoints](/docs/api-reference/webhooks/managing-endpoints) — the management API: registered endpoints, the delivery log, resend, and signing secrets - [Website Monitor Webhooks](/docs/integrations/website-monitoring/webhooks) — the monitor-specific guide (flat `monitor.change` payload, snapshot fetching) --- # Wire (/docs/api-reference/wire) Run pre-built automation actions across hundreds of popular websites. Each action handles browser rendering, authentication, and structured data extraction. Jobs are processed asynchronously — submit a task and poll for results. ### How It Works 1. **Discover an action** — list catalogs or search to find an `action_id` and its parameter schema 2. **Submit a task** — call `POST /v1/wire/task` with the `action_id` and your parameters 3. **Poll for results** — use the returned `job_id` to check status and retrieve data Browse available actions from the [Wire dashboard](/wire) or via the discovery endpoints below. > **No API key? Read-only Wire is free.** Discovery — `GET /v1/wire/catalog` and `GET /v1/wire/resolve` (also served as `/search`) — is public, and you can **run** read-only actions with **no key and no account** via the synchronous [Zero Touch](/docs/integrations/ai-agents/zero-touch) endpoint `POST /v1/wire-run` — the result comes back inline, no polling. The `POST /v1/wire/task` flow documented below is the **keyed, asynchronous** mode: required for write actions, account-connected runs, and durable jobs. ### Endpoints <EndpointCard method="GET" path="/v1/wire/catalog" description="List Catalogs — every supported website with its action count" href="/docs/api-reference/wire/list-catalogs" /> <EndpointCard method="GET" path="/v1/wire/catalog/{slug}" description="Catalog Details — single catalog plus its full action list & schemas" href="/docs/api-reference/wire/get-catalog" /> <EndpointCard method="GET" path="/v1/wire/resolve" description="Resolve Actions — find an action_id by intent, catalog, or category (alias: /search)" href="/docs/api-reference/wire/search-actions" /> <EndpointCard method="POST" path="/v1/wire/task" description="Execute Task — run a pre-built action and get a job ID" href="/docs/api-reference/wire/execute-task" /> <EndpointCard method="GET" path="/v1/wire/jobs/{id}" description="Get Job Status — poll for status and retrieve results" href="/docs/api-reference/wire/get-job" /> ### Identities & credentials An **identity** is a named account on a website (e.g. "Personal Amazon", "Work Amazon"). Each identity holds one or more **credentials** (cookies, API keys, etc.) — the encrypted auth data needed to run tasks as that account. To use multi-auth: list your identities, copy the `credential_id` you want, pass it as `credential_id` on `POST /v1/wire/task`. Identities and credentials are typically created via the [Wire dashboard](/wire/identities); for credentials-mode catalogs (email/password sign-in), API consumers can also sign in directly with `POST /login`. <EndpointCard method="GET" path="/v1/wire/identities" description="List all your identities (or one by ID) with their credentials inline" href="/docs/api-reference/wire/list-identities" /> <EndpointCard method="POST" path="/v1/wire/login" description="Create Identity / Login — find-or-create an identity and sign in (password or vault locator) in one call" href="/docs/api-reference/wire/login" /> <EndpointCard method="POST" path="/v1/wire/credentials/verify" description="Verify Identity / Re-Login — refresh the saved login for an identity (vault-sourced or manual)" href="/docs/api-reference/wire/verify-credential" /> ### Identity sources An **identity source** is a connection to a vault you own — 1Password or Azure Key Vault — that Wire reads credentials from at sign-in time, so no password is typed into Wire. The secret never enters Wire's database: it's read into memory for a single login run and dropped. What's stored is a pointer to it. Start with [Provider Setup](/docs/api-reference/wire/provider-setup) — creating the service account or service principal that grants Wire read access. The rest of the lifecycle is available over the API, and also from the [dashboard](/wire/identity-sources). <EndpointCard method="GET" path="/v1/wire/identity-sources/providers" description="List connectable providers and the fields each one needs" href="/docs/api-reference/wire/list-providers" /> <EndpointCard method="POST" path="/v1/wire/identity-sources" description="Connect a vault — verified before anything is stored" href="/docs/api-reference/wire/create-identity-source" /> <EndpointCard method="GET" path="/v1/wire/identity-sources" description="List your connected sources" href="/docs/api-reference/wire/list-identity-sources" /> <EndpointCard method="GET" path="/v1/wire/identity-sources/{id}" description="One source, plus how many identities depend on it" href="/docs/api-reference/wire/get-identity-source" /> <EndpointCard method="PATCH" path="/v1/wire/identity-sources/{id}" description="Rename or rotate the credential — bindings survive" href="/docs/api-reference/wire/update-identity-source" /> <EndpointCard method="POST" path="/v1/wire/identity-sources/{id}/verify" description="Re-test the stored credential and refresh its health" href="/docs/api-reference/wire/verify-identity-source" /> <EndpointCard method="DELETE" path="/v1/wire/identity-sources/{id}" description="Disconnect, keeping or deleting the sourced identities" href="/docs/api-reference/wire/delete-identity-source" /> <EndpointCard method="GET" path="/v1/wire/identity-sources/{id}/identities" description="List the identities sourced from a connection" href="/docs/api-reference/wire/list-source-identities" /> Browse a connected vault to find the entry to bind. Both endpoints are provider-agnostic — a container is a 1Password vault or an Azure Key Vault; an entry is a login item or a secret: <EndpointCard method="GET" path="/v1/wire/identity-sources/{id}/containers" description="List containers the source can read" href="/docs/api-reference/wire/list-source-containers" /> <EndpointCard method="GET" path="/v1/wire/identity-sources/{id}/containers/{container_id}/entries" description="List (domain-filtered) entries in a container" href="/docs/api-reference/wire/list-source-entries" /> Then bind: pass `source_id` and a [`source_ref`](/docs/api-reference/wire/login#the-source_ref-locator) to [POST /v1/wire/login](/docs/api-reference/wire/login). ### Build new actions Need an action for a site that isn't in the catalog yet? Request a build — Wire generates a scraper from your description, auto-tests it, and publishes it so you can run it with `POST /task`. <EndpointCard method="POST" path="/v1/wire/build-request" description="Request a Build — generate a brand-new action for a site that isn't in the catalog yet" href="/docs/api-reference/wire/create-build-request" /> ### Errors Common error responses on [POST /v1/wire/task](/docs/api-reference/wire/execute-task): | Status | Code | When | Action | |--------|------|------|--------| | `401` | `AUTH_REQUIRED` | Action needs auth and you have no credential for this catalog | Visit the `connect_url` in the response to connect your account | | `401` | `AUTH_EXPIRED` | The `credential_id` exists but its session is no longer valid (cookies expired, token revoked, password changed) | Have the user reconnect — `GET /identities` will show `status: "expired"` on the credential | | `403` | `FORBIDDEN` | The `credential_id` doesn't belong to you, or belongs to a different catalog than the action | Re-fetch valid credential IDs via [GET /v1/wire/identities](/docs/api-reference/wire/list-identities) | | `402` | `INSUFFICIENT_CREDITS` | Your account balance can't cover the action's `credits_per_call` | Top up credits or use a smaller-cost action | | `500` | `EXECUTION_FAILED` | Transient submission failure — engine couldn't enqueue the task. Credits are not deducted | Retry the request; if it persists, contact support | --- # GET Get Results (/docs/api-reference/agentic-search/get-search-result) <EndpointBanner method="GET" path="https://api.anakin.io/v1/agentic-search/{id}" /> Retrieve the status and results of an agentic search job. Agentic searches run through multiple stages and may take several minutes — poll every 10 seconds. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The job ID returned from the submit endpoint | --- ### Response — Processing <StatusBadge code={202} text="Accepted" /> ```json { "job_id": "3f8aa45d-6ea3-4107-88ce-7f39ecf48a84", "status": "pending", "message": "Job is pending", "created_at": "2024-01-01T12:00:00.000Z" } ``` ### Response — Completed <StatusBadge code={200} text="OK" /> ```json { "id": "3f8aa45d-6ea3-4107-88ce-7f39ecf48a84", "status": "completed", "jobType": "agentic_search", "generatedJson": { "summary": "Summary of the research findings...", "structured_data": { "developments": [ { "title": "Quantum Computing Advances", "description": "Recent developments in quantum computing...", "organization": "IBM", "date": "2024-01" } ] }, "data_schema": { "description": "Schema for structured data extraction", "fields": { "developments": { "type": "array", "description": "List of developments" } } } }, "createdAt": "2024-01-01T12:00:00.000Z", "completedAt": "2024-01-01T12:05:00.000Z", "durationMs": 33500 } ``` --- ### Response Fields | Field | Type | Description | |-------|------|-------------| | `id` | string | Unique identifier for the job | | `status` | string | `pending`, `processing`, `completed`, or `failed` | | `jobType` | string | Always `agentic_search` for this endpoint | | `generatedJson` | object | The full agentic search result (see below) | | `createdAt` | string | ISO 8601 timestamp of job creation | | `completedAt` | string | ISO 8601 timestamp of completion | | `durationMs` | number | Total processing time in milliseconds | ### generatedJson Fields | Field | Type | Description | |-------|------|-------------| | `summary` | string | Concise summary of findings | | `structured_data` | object | Dynamic structured data matching `data_schema.fields` | | `data_schema` | object | Schema describing the structured data format | ### Job Statuses | Status | Description | |--------|-------------| | `pending` | Job is queued | | `processing` | Research pipeline is running | | `completed` | Research report is ready | | `failed` | Job encountered an error | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X GET https://api.anakin.io/v1/agentic-search/3f8aa45d-6ea3-4107-88ce-7f39ecf48a84 \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests job_id = "3f8aa45d-6ea3-4107-88ce-7f39ecf48a84" result = requests.get( f'https://api.anakin.io/v1/agentic-search/{job_id}', headers={'X-API-Key': 'your_api_key'} ) data = result.json() if data['status'] == 'completed': result_data = data['generatedJson'] print(f"Summary: {result_data['summary']}") print(f"Schema: {result_data['data_schema']}") print(f"Data: {result_data['structured_data']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const jobId = '3f8aa45d-6ea3-4107-88ce-7f39ecf48a84'; const res = await fetch(`https://api.anakin.io/v1/agentic-search/${jobId}`, { headers: { 'X-API-Key': 'your_api_key' } }); const data = await res.json(); if (data.status === 'completed') { const resultData = data.generatedJson; console.log(resultData.summary); console.log(resultData.data_schema); console.log(resultData.structured_data); } ``` </Tab> </Tabs> For polling patterns, see the [Polling Jobs](/docs/api-reference/polling-jobs) reference. --- # POST Research Query (/docs/api-reference/agentic-search/submit-search) <EndpointBanner method="POST" path="https://api.anakin.io/v1/agentic-search" /> Start an agentic research pipeline. The job runs through 4 stages (query refinement, web search, citation scraping, analysis) and may take several minutes to complete. Poll for results using [GET /v1/agentic-search/\{id\}](/docs/api-reference/agentic-search/get-search-result). --- ### Request Body ```json { "prompt": "Comprehensive analysis of quantum computing trends", "webhook_url": "https://your-app.com/hooks/anakin" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `prompt` **required** | string | Research query or question | | `webhook_url` | string | URL to `POST` a signed `job.completed` / `job.failed` event to when the research job settles — handy since agentic searches can run for minutes. See [Webhooks](/docs/api-reference/webhooks). | --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "job_id": "3f8aa45d-6ea3-4107-88ce-7f39ecf48a84", "status": "pending", "message": "Agentic search job queued successfully", "created_at": "2024-01-01T12:00:00.000Z" } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `job_id` | string | Unique identifier for the agentic search job | | `status` | string | Job status (`pending`) | | `message` | string | Confirmation message | | `created_at` | string | ISO 8601 timestamp of job creation | Use the `job_id` with [GET /v1/agentic-search/\{id\}](/docs/api-reference/agentic-search/get-search-result) to poll for results. Agentic searches typically take longer than standard scrapes — poll every 10 seconds. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/agentic-search \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "prompt": "Comprehensive analysis of quantum computing trends" }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/agentic-search', headers={'X-API-Key': 'your_api_key'}, json={ 'prompt': 'Comprehensive analysis of quantum computing trends' } ) data = response.json() print(f"Agentic search submitted: {data['job_id']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/agentic-search', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ prompt: 'Comprehensive analysis of quantum computing trends' }) }); const data = await response.json(); console.log(data.job_id); ``` </Tab> </Tabs> --- # GET Search (/docs/api-reference/ai-visibility/get-search) <EndpointBanner method="GET" path="https://api.anakin.io/v1/ai-visibility/search/{id}" /> Returns the search's status and every per-source result persisted so far. While `status` is `running`, results appear incrementally as each engine answers — poll every 2–5 seconds until `completed`. > This endpoint is also the `result_url` target of the `ai.search.completed` / `ai.search.failed` [webhook events](/docs/api-reference/webhooks/events). Deliveries inline only bounded per-source previews — the **full answers live here**, so webhook receivers should `GET` this URL with their API key after each event. --- ### Response <StatusBadge code={200} text="OK" /> ```json { "search_id": "3d9db7ff-6ea3-4107-88ce-7f39ecf48a84", "status": "completed", "country": "us", "results": [ { "source": "chatgpt", "status": "completed", "summary": "As of July 2026, these are widely regarded as…", "full_content": "{\"prompt\":\"…\",\"answer_text\":\"…\"}", "latency_ms": 9100, "credits_used": 1 }, { "source": "gemini", "status": "failed", "error": "Timed out waiting for the source to answer" } ] } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `status` | string | `running` → `completed`, or `failed` when every source fails (a search still settles as `completed` when only some sources fail). Runs stuck past 10 minutes are marked `failed` automatically | | `country` | string | Search geography the run executed from (set at submit; retries reuse it) | | `synthesis` | string | Optional — cross-source agreement/divergence summary. Generated automatically once at least two sources complete; omitted for single-source runs | | `results[].source` | string | Source slug | | `results[].status` | string | `completed`, `failed`, or `timed_out` (the engine didn't answer within the run window — retrying a timed-out source reconciles the original job before re-billing) | | `results[].summary` | string | Extracted answer summary | | `results[].full_content` | string | The engine's raw response payload (JSON string) | | `results[].latency_ms` | number | Measured engine execution time | | `results[].verdict` | string | Optional — AI-assessed `consensus` or `outlier`, set by the same automatic synthesis pass (absent for single-source runs) | | `results[].credits_used` | number | Credits billed for this source (0 when failed) | | `results[].error` | string | Failure reason, when `status` is `failed` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl https://api.anakin.io/v1/ai-visibility/search/3d9db7ff-6ea3-4107-88ce-7f39ecf48a84 \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests, time while True: data = requests.get( f"https://api.anakin.io/v1/ai-visibility/search/{search_id}", headers={"X-API-Key": "your_api_key"}, ).json() if data["status"] == "completed": break time.sleep(2) ``` </Tab> <Tab value="JavaScript"> ```javascript let data; do { await new Promise((r) => setTimeout(r, 2000)); data = await ( await fetch(`https://api.anakin.io/v1/ai-visibility/search/${searchId}`, { headers: { "X-API-Key": "your_api_key" }, }) ).json(); } while (data.status !== "completed"); ``` </Tab> </Tabs> --- # GET List Searches (/docs/api-reference/ai-visibility/list-searches) <EndpointBanner method="GET" path="https://api.anakin.io/v1/ai-visibility/searches" /> Lists your most recent searches with per-source statuses and total credit cost. Because searches run asynchronously, this endpoint is how you recover results for runs that finished after your client disconnected. --- ### Response <StatusBadge code={200} text="OK" /> ```json { "searches": [ { "search_id": "3d9db7ff-6ea3-4107-88ce-7f39ecf48a84", "query": "Best coding agents in 2026", "status": "completed", "created_at": "2026-07-15 12:00:00+00", "credits_used": 3, "sources": [ { "source": "chatgpt", "status": "completed" }, { "source": "gemini", "status": "completed" }, { "source": "google-ai-overview", "status": "completed" } ] } ] } ``` Returns your 20 most recent searches. Fetch full results per run via [GET /v1/ai-visibility/search/\{id\}](/docs/api-reference/ai-visibility/get-search). --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl https://api.anakin.io/v1/ai-visibility/searches \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests searches = requests.get( "https://api.anakin.io/v1/ai-visibility/searches", headers={"X-API-Key": "your_api_key"}, ).json()["searches"] ``` </Tab> <Tab value="JavaScript"> ```javascript const { searches } = await ( await fetch("https://api.anakin.io/v1/ai-visibility/searches", { headers: { "X-API-Key": "your_api_key" }, }) ).json(); ``` </Tab> </Tabs> --- # GET List Sources (/docs/api-reference/ai-visibility/list-sources) <EndpointBanner method="GET" path="https://api.anakin.io/v1/ai-visibility/sources" /> Lists the enabled AI sources. Slugs are what you pass in `sources` when [submitting a search](/docs/api-reference/ai-visibility/submit-search). The roster is managed platform-side — new engines appear here as they land in the Wire catalogue. --- ### Response <StatusBadge code={200} text="OK" /> ```json { "sources": [ { "slug": "chatgpt", "label": "ChatGPT" }, { "slug": "gemini", "label": "Gemini" }, { "slug": "google-ai-overview", "label": "Google AI Overview" } ] } ``` --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl https://api.anakin.io/v1/ai-visibility/sources \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests sources = requests.get( "https://api.anakin.io/v1/ai-visibility/sources", headers={"X-API-Key": "your_api_key"}, ).json()["sources"] ``` </Tab> <Tab value="JavaScript"> ```javascript const { sources } = await ( await fetch("https://api.anakin.io/v1/ai-visibility/sources", { headers: { "X-API-Key": "your_api_key" }, }) ).json(); ``` </Tab> </Tabs> --- # POST Retry Source (/docs/api-reference/ai-visibility/retry-source) <EndpointBanner method="POST" path="https://api.anakin.io/v1/ai-visibility/search/{id}/retry" /> Re-runs a single source against an existing search's prompt, replacing that source's stored result. The call is synchronous — it returns when the engine answers (or fails). The retried source is billed again on success, and its previous consensus/outlier verdict is cleared. Retries are rate-limited to 30 per minute per account. One exception to re-billing: if the source previously ended `timed_out`, the retry first checks whether the original engine job finished after the run window closed — if it did, that answer is adopted **without a new charge**; only a genuinely fresh re-run bills again. --- ### Request Body ```json { "source": "gemini" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `source` **required** | string | The source slug to re-run | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "source": "gemini", "status": "completed", "summary": "The smartphone class of 2026 has delivered…", "full_content": "{\"prompt\":\"…\",\"answer_text\":\"…\"}", "latency_ms": 21400, "credits_used": 1 } ``` Returns the updated result for that source only — the same shape as an entry in `results[]` from [GET /v1/ai-visibility/search/\{id\}](/docs/api-reference/ai-visibility/get-search). --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/ai-visibility/search/3d9db7ff-6ea3-4107-88ce-7f39ecf48a84/retry \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{"source": "gemini"}' ``` </Tab> <Tab value="Python"> ```python import requests result = requests.post( f"https://api.anakin.io/v1/ai-visibility/search/{search_id}/retry", headers={"X-API-Key": "your_api_key"}, json={"source": "gemini"}, ).json() ``` </Tab> <Tab value="JavaScript"> ```javascript const result = await ( await fetch(`https://api.anakin.io/v1/ai-visibility/search/${searchId}/retry`, { method: "POST", headers: { "X-API-Key": "your_api_key", "Content-Type": "application/json" }, body: JSON.stringify({ source: "gemini" }), }) ).json(); ``` </Tab> </Tabs> --- # POST Submit Search (/docs/api-reference/ai-visibility/submit-search) <EndpointBanner method="POST" path="https://api.anakin.io/v1/ai-visibility/search" /> Start an AI Visibility search. The response returns immediately with a `search_id` and `status: "running"`; each source's answer is persisted the moment its engine finishes. Poll [GET /v1/ai-visibility/search/\{id\}](/docs/api-reference/ai-visibility/get-search) or subscribe to the `ai.search.completed` webhook event — note that webhook deliveries carry bounded previews only, with the full answers always fetched from the event's `result_url` (see [Event Types & Payloads](/docs/api-reference/webhooks/events)). Every submit runs fresh — results are never served from cache, because AI engine answers change constantly. Submitting the same prompt twice creates two separate runs, each billed normally. Submissions are rate-limited to 15 per minute per account, and `query` is capped at 2,000 characters (longer requests return `400 invalid_request`). --- ### Request Body ```json { "query": "Best coding agents in 2026", "sources": ["chatgpt", "gemini", "google-ai-overview"], "country": "us" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `query` **required** | string | The prompt sent verbatim to every selected AI engine | | `sources` | string[] | Source slugs to query (see [List Sources](/docs/api-reference/ai-visibility/list-sources)). Omit to query every enabled source | | `country` | string | Search geography — 2-letter ISO code (e.g. `us`, `in`), same geo targeting as the [URL scraper](/docs/api-reference/url-scraper/submit-scrape-job). Engines are queried from that region; answers (and whether Google serves an AI Overview at all) vary by geography. Defaults to `us`; retries of a source reuse the run's country | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "search_id": "3d9db7ff-6ea3-4107-88ce-7f39ecf48a84", "status": "running", "results": [], "country": "us" } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `search_id` | string | Identifier to poll for this run's results | | `status` | string | Always `running` — results arrive per source as the engines answer | | `results` | array | Always empty at submission — poll to see results fill in | | `country` | string | The normalized search geography this run executes from | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/ai-visibility/search \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "query": "Best coding agents in 2026", "sources": ["chatgpt", "gemini"] }' ``` </Tab> <Tab value="Python"> ```python import requests resp = requests.post( "https://api.anakin.io/v1/ai-visibility/search", headers={"X-API-Key": "your_api_key"}, json={"query": "Best coding agents in 2026", "sources": ["chatgpt", "gemini"]}, ) search_id = resp.json()["search_id"] ``` </Tab> <Tab value="JavaScript"> ```javascript const resp = await fetch("https://api.anakin.io/v1/ai-visibility/search", { method: "POST", headers: { "X-API-Key": "your_api_key", "Content-Type": "application/json" }, body: JSON.stringify({ query: "Best coding agents in 2026", sources: ["chatgpt", "gemini"], }), }); const { search_id } = await resp.json(); ``` </Tab> </Tabs> --- # WS Geo-Targeting (/docs/api-reference/browser-api/geo-targeting) ## Country-routed sessions Pass `?country=XX` (ISO 3166-1 alpha-2) when connecting to route your session through a residential proxy in that country: ``` wss://api.anakin.io/v1/browser-connect?country=GB ``` ## Aligned fingerprint signals When you pass `?country=`, the browser's **timezone, locale, language, and Accept-Language all match the proxy's country**. Anti-bot systems compare these against the IP — a UK IP that reports `UTC` and `en-US` is an obvious bot signal. With `?country=GB` every signal lines up. What websites see for `?country=GB`: | Signal | What websites see | |---|---| | Exit IP | UK residential (e.g. BT, Virgin Media, Sky) | | `navigator.language` | `en-GB` | | `navigator.languages` | `["en-GB"]` | | `Accept-Language` request header | `en-GB` | | `Intl.DateTimeFormat().resolvedOptions().timeZone` | `Europe/London` | | `new Date().getTimezoneOffset()` | `-60` (BST) or `0` (GMT) depending on the date | | `Intl.NumberFormat` default locale | `en-GB` (formats currency as £) | | `Intl.NumberFormat(undefined, {style:'currency',currency:'GBP'})` | `£1,234.56` | | WebRTC ICE candidates | mDNS-obfuscated (no real IP leak) | | `navigator.geolocation` | denied by default (requires explicit user grant) | ## Supported countries Per-country timezone and locale defaults: | Code | Timezone | Locale | |---|---|---| | `US` | America/New_York | en-US | | `GB` | Europe/London | en-GB | | `DE` | Europe/Berlin | de-DE | | `FR` | Europe/Paris | fr-FR | | `ES` | Europe/Madrid | es-ES | | `IT` | Europe/Rome | it-IT | | `NL` | Europe/Amsterdam | nl-NL | | `SE` | Europe/Stockholm | sv-SE | | `JP` | Asia/Tokyo | ja-JP | | `KR` | Asia/Seoul | ko-KR | | `CN` | Asia/Shanghai | zh-CN | | `IN` | Asia/Kolkata | en-IN | | `SG` | Asia/Singapore | en-SG | | `HK` | Asia/Hong_Kong | zh-HK | | `AU` | Australia/Sydney | en-AU | | `NZ` | Pacific/Auckland | en-NZ | | `CA` | America/Toronto | en-CA | | `MX` | America/Mexico_City | es-MX | | `BR` | America/Sao_Paulo | pt-BR | | `AR` | America/Argentina/Buenos_Aires | es-AR | | `AE` | Asia/Dubai | ar-AE | | `SA` | Asia/Riyadh | ar-SA | | `ZA` | Africa/Johannesburg | en-ZA | Country codes not in this list still route through a residential proxy in that country (proxy availability permitting), but timezone and locale fall back to the browser's defaults (`UTC` and `en-US`). Open a support ticket if you need a country added to the alignment table. If `?country=` is omitted, the session is **not geo-pinned**: it does not go through a residential proxy, and timezone/locale stay at the browser defaults (`UTC`, `en-US`). Pass a country whenever the target site cares about where you are coming from. **Checking your exit IP:** some IP-echo services (ipinfo.io, api.ipify.org, httpbin.org) block residential proxy ranges and fail with `ERR_TUNNEL_CONNECTION_FAILED` — that is the echo service, not your session. Use `https://lumtest.com/myip.json` instead. ## Example: GB session <Tabs items={["Playwright (Python)", "Playwright (Node.js)"]}> <Tab value="Playwright (Python)"> ```python import asyncio from playwright.async_api import async_playwright async def main(): async with async_playwright() as p: browser = await p.chromium.connect_over_cdp( "wss://api.anakin.io/v1/browser-connect?country=GB", headers={"X-API-Key": "your_api_key"}, ) page = browser.contexts[0].pages[0] # IP, timezone, language all align to GB info = await page.evaluate(""" ({ tz: Intl.DateTimeFormat().resolvedOptions().timeZone, lang: navigator.language, offset: new Date().getTimezoneOffset(), }) """) print(info) # {tz: 'Europe/London', lang: 'en-GB', offset: 0} await browser.close() asyncio.run(main()) ``` </Tab> <Tab value="Playwright (Node.js)"> ```javascript const { chromium } = require('playwright'); (async () => { const browser = await chromium.connectOverCDP( 'wss://api.anakin.io/v1/browser-connect?country=GB', { headers: { 'X-API-Key': 'your_api_key' } } ); const page = browser.contexts()[0].pages()[0]; const info = await page.evaluate(() => ({ tz: Intl.DateTimeFormat().resolvedOptions().timeZone, lang: navigator.language, offset: new Date().getTimezoneOffset(), })); console.log(info); // {tz: 'Europe/London', lang: 'en-GB', offset: 0} await browser.close(); })(); ``` </Tab> </Tabs> ## Sticky exit IP within a session A single session keeps the same exit IP for its duration. Reconnecting issues a fresh proxy by default — pair with [Saved Sessions](/docs/api-reference/browser-api/saved-sessions) if you need IP stickiness across reconnects (the saved session's proxy is preserved). --- # Limits & Billing (/docs/api-reference/browser-api/limits) ## Billing Browser API sessions are billed at **1 credit per 2 minutes**, rounded up. A session that lasts 3 minutes costs 2 credits. A session that lasts 4 minutes 1 second also costs 2 credits (you're billed at the start of each 2-minute window). | | | |---|---| | Rate | 1 credit / 2 min | | First credit | Charged on connect (covers minutes 0-2) | | Subsequent credits | Charged at the start of each new 2-minute window | | Recording | **Free** — included in the per-minute rate | | Geo-targeting | **Free** — no per-country surcharge | | Saved-session load | **Free** — no extra charge for loading cookies | | Auto-disconnect | When your credit balance reaches 0 | ## Limits | Parameter | Value | Notes | |---|---|---| | Max session duration | 2 hours | Hard cap; session ends automatically | | Idle timeout | ~2 minutes | No CDP messages from client → disconnect | | Max message size | 50 MB | Per CDP frame | | Concurrent connections | per-account plan limit | See [pricing](https://anakin.io/pricing) | | Authentication | `X-API-Key` header **or** `?api_key=` query param | Header preferred; query param needed for clients that can't set custom WS headers | ### Avoiding the idle timeout The connection is closed if your script is silent for **about 2 minutes**. Long workflow pauses (MFA wait, slow API polls, OAuth callbacks) need a keepalive — call any cheap browser API every ~60 seconds: ```python import asyncio # during a long wait... while waiting_for_mfa(): await page.evaluate("() => 0") # cheap keepalive await asyncio.sleep(60) ``` Playwright's `page.wait_for_function()` and `page.wait_for_selector()` also keep the session alive while they're polling — preferred for any "wait until X appears" loop. The cleanest practice is **`browser.close()`** as soon as you're done — don't leave idle connections open just because you can. --- ## Not supported - **Selenium / ChromeDriver** — only CDP-native clients (Playwright, Puppeteer, chromedp) are supported. --- ## Error responses Authentication and capacity errors come back as standard HTTP responses **before** the WebSocket upgrade, so your CDP client receives them as connection failures. Saved-session errors (404 / 409 / 422) are raised after the upgrade and arrive as a WebSocket **close frame with code 1008** whose reason is `"<status>: <json body>"`, e.g. `404: {"error": "session not found, not yet saved, or in use", ...}` — Playwright surfaces it in the `connectOverCDP` error message. | Status | Error | Cause | |---|---|---| | 401 | `X-API-Key required` | No API key sent | | 403 | `auth rejected: invalid_api_key` | API key not recognised | | 402 | `insufficient credits` | Credit balance is 0 | | 404 | `session not found` | `?session_id` / `?session_name` doesn't resolve | | 409 | `session is being automated` | The session you're loading is in use by another job | | 422 | `session has no stored data` | Session was created but never saved | | 429 | Rate-limited | Too many active connections for this account | | 503 | `browser service unavailable` | Transient — retry with backoff | | 503 | `session saving not configured` | Saved sessions temporarily unavailable (contact support) | --- # WS Recording (/docs/api-reference/browser-api/recording) Browser API can record each session as a WebM video. Recording is **opt-in** — pass `?record=true` when connecting and the video is finalized + uploaded when you disconnect. ``` wss://api.anakin.io/v1/browser-connect?record=true ``` Recording is free — no extra credits beyond the standard 1 credit / 2 minute rate. ## Retrieving recordings After disconnecting, your recording is available via the API: ```bash # List all recordings curl https://api.anakin.io/v1/recordings \ -H "X-API-Key: your_api_key" # Get a specific recording (includes a presigned video URL) # The path param accepts either the `id` (UUID) or the `connId` (rec-…). curl https://api.anakin.io/v1/recordings/rec-123 \ -H "X-API-Key: your_api_key" # Download the video directly — 302-redirects to a 1-hour presigned URL curl -OL https://api.anakin.io/v1/recordings/rec-123/download \ -H "X-API-Key: your_api_key" ``` The detail endpoint returns a **presigned video URL valid for 1 hour**: ```json { "id": "abc-123", "connId": "rec-123", "duration": 45, "fileSize": 304205, "status": "completed", "videoUrl": "https://.../recording.webm", "createdAt": "2026-04-01T12:00:00Z" } ``` Recordings are also browsable from your [dashboard](/recordings). ## Pairing with other features Combine recording with [saved sessions](/docs/api-reference/browser-api/saved-sessions) — both work transparently on the same connection: ``` wss://api.anakin.io/v1/browser-connect?record=true&save_session=my-login&save_url=https://amazon.com ``` The recording captures the entire session including the auto-save handshake at the end. ## Specs | Property | Value | |---|---| | Format | WebM (VP8) | | Frame rate | 15 fps | | Resolution | 1920 × 1080 | | Retention | 7 days | | Cost | Included (no extra credits) | | Typical size | ~50 MB for a 2-hour session | | Availability | Recording is finalized a few seconds after you disconnect | ## Default Recording is **off by default**. If you don't pass `?record=true`, no video is captured and nothing is uploaded. Billing is unchanged either way — recording is a privacy / storage knob, not a billing one. --- # WS Saved Sessions (/docs/api-reference/browser-api/saved-sessions) Saved sessions let your Browser API connections start **with cookies and localStorage already loaded** — no need to log in inside every script. The flow has two steps: **save** a session once (log in, then disconnect — Anakin captures the auth state), then **load** that session on every future run. The same flow is available in your [dashboard](https://anakin.io/dashboard) and via the [`/v1/sessions` API](/docs/api-reference/browser-sessions). --- ## 1. Save a session on disconnect Pass `?save_session=<name>` and `?save_url=<target>` when connecting. Log in inside your script, then close the browser — the session's cookies + localStorage are captured automatically. ``` wss://api.anakin.io/v1/browser-connect?save_session=my-login&save_url=https://your-target-site.com ``` <Tabs items={["Playwright (Python)", "Playwright (Node.js)"]}> <Tab value="Playwright (Python)"> ```python import asyncio from playwright.async_api import async_playwright API_KEY = "your_api_key" async def main(): async with async_playwright() as p: browser = await p.chromium.connect_over_cdp( "wss://api.anakin.io/v1/browser-connect?save_session=my-login&save_url=https://your-target-site.com", headers={"X-API-Key": API_KEY}, ) page = browser.contexts[0].pages[0] # Log in await page.goto("https://your-target-site.com/login") await page.fill("input[name='username']", "your_username") await page.fill("input[name='password']", "your_password") await page.click("button[type='submit']") await page.wait_for_load_state("networkidle") # Disconnect — cookies + localStorage are auto-saved await browser.close() asyncio.run(main()) ``` </Tab> <Tab value="Playwright (Node.js)"> ```javascript const { chromium } = require('playwright'); const API_KEY = 'your_api_key'; (async () => { const browser = await chromium.connectOverCDP( 'wss://api.anakin.io/v1/browser-connect?save_session=my-login&save_url=https://your-target-site.com', { headers: { 'X-API-Key': API_KEY } } ); const page = browser.contexts()[0].pages()[0]; // Log in await page.goto('https://your-target-site.com/login'); await page.fill("input[name='username']", 'your_username'); await page.fill("input[name='password']", 'your_password'); await page.click("button[type='submit']"); await page.waitForLoadState('networkidle'); // Disconnect — session is auto-saved await browser.close(); })(); ``` </Tab> </Tabs> On the next run, load with `?session_name=my-login` (see below). ### Save parameters | Parameter | Required | Description | |---|---|---| | `save_session` | Yes | Name for the saved session (must be unique per user) | | `save_url` | Yes | Website URL the session belongs to (e.g. `https://amazon.com`) | ### Save error responses | Status | Error | Meaning | |---|---|---| | 503 | `session saving not configured` | Saved sessions are temporarily unavailable (contact support) | Names are not enforced unique on save: connecting again with the same `save_session` name creates a second saved session with that name, and `?session_name=` lookups become ambiguous. Use a fresh name per save, or delete the old session first from your [dashboard](https://anakin.io/dashboard) or via `DELETE /v1/sessions/{id}`. --- ## 2. Load a saved session Pass `?session_id=<uuid>` or `?session_name=<name>` when connecting: ``` wss://api.anakin.io/v1/browser-connect?session_name=my-login ``` The browser launches with your saved cookies and localStorage already loaded — your script can navigate straight to authenticated pages without logging in again. The session's original exit IP is reused on every load, so logins protected by IP-binding stay valid. <Tabs items={["Playwright (Python)", "Playwright (Node.js)", "Puppeteer (Node.js)"]}> <Tab value="Playwright (Python)"> ```python import asyncio from playwright.async_api import async_playwright API_KEY = "your_api_key" SESSION_ID = "your_session_id" # from dashboard or GET /v1/sessions async def main(): async with async_playwright() as p: browser = await p.chromium.connect_over_cdp( f"wss://api.anakin.io/v1/browser-connect?session_id={SESSION_ID}", headers={"X-API-Key": API_KEY}, ) page = browser.contexts[0].pages[0] # Navigate to an authenticated page — cookies are pre-loaded await page.goto("https://amazon.com/your-orders", wait_until="domcontentloaded") print("Title:", await page.title()) await browser.close() asyncio.run(main()) ``` </Tab> <Tab value="Playwright (Node.js)"> ```javascript const { chromium } = require('playwright'); const API_KEY = 'your_api_key'; const SESSION_ID = 'your_session_id'; (async () => { const browser = await chromium.connectOverCDP( `wss://api.anakin.io/v1/browser-connect?session_id=${SESSION_ID}`, { headers: { 'X-API-Key': API_KEY } } ); const page = browser.contexts()[0].pages()[0]; await page.goto('https://amazon.com/your-orders', { waitUntil: 'domcontentloaded' }); console.log('Title:', await page.title()); await browser.close(); })(); ``` </Tab> <Tab value="Puppeteer (Node.js)"> ```javascript const puppeteer = require('puppeteer-core'); const API_KEY = 'your_api_key'; const SESSION_ID = 'your_session_id'; (async () => { const browser = await puppeteer.connect({ browserWSEndpoint: `wss://api.anakin.io/v1/browser-connect?session_id=${SESSION_ID}`, headers: { 'X-API-Key': API_KEY }, }); const page = (await browser.pages())[0] || await browser.newPage(); await page.goto('https://amazon.com/your-orders', { waitUntil: 'domcontentloaded', timeout: 30000, }); console.log('Title:', await page.title()); await browser.disconnect(); })(); ``` </Tab> </Tabs> You can use the session **name** instead of ID: ``` wss://api.anakin.io/v1/browser-connect?session_name=my-login ``` ### Load parameters | Parameter | Description | |---|---| | `session_id` | UUID of the saved session | | `session_name` | Name of the saved session (must be unique per user) | If both are provided, `session_id` takes precedence. If neither is provided, a fresh browser is launched. ### Load error responses Session validation fails the connection with a WebSocket **close code 1008** whose reason is `"<status>: <json body>"` (see [Limits & Billing → Error responses](/docs/api-reference/browser-api/limits#error-responses)): | Status | Error | Meaning | |---|---|---| | 404 | `session not found` | Session ID or name doesn't exist for your account | | 422 | `session has no stored data — save it first` | Session was created but never saved — log in and save first | | 409 | `session is being automated` | Session is currently in use by an automation job | --- ## Tips - **Save is on disconnect.** Cookies and localStorage are captured when you close the browser, so always end your script with `browser.close()` (or `browser.disconnect()` for Puppeteer). - **localStorage is captured per open tab.** Cookies are browser-wide, but localStorage is read only from tabs that are still open on that origin at disconnect. Leave the logged-in page open (don't navigate it elsewhere or close it) before `browser.close()` if you need its localStorage saved. - **Need a confirmed save with a UI?** Use the [interactive browser session](/docs/api-reference/browser-sessions) flow instead. - **Listing and deleting saved sessions** is available in your [dashboard](https://anakin.io/dashboard) and via the [`/v1/sessions` API](/docs/api-reference/browser-sessions). --- # GET Get Crawl Result (/docs/api-reference/crawl/get-crawl-result) <EndpointBanner method="GET" path="https://api.anakin.io/v1/crawl/{id}" /> Retrieve the status and results of a crawl job. Use this to poll for completion after submitting a [crawl request](/docs/api-reference/crawl/submit-crawl-job). --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The job ID returned from the submit endpoint | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "id": "job_abc123xyz", "status": "completed", "url": "https://example.com", "totalPages": 3, "completedPages": 3, "results": [ { "url": "https://example.com", "status": "completed", "html": "<html>...</html>", "markdown": "# Home page content...", "durationMs": 2000 }, { "url": "https://example.com/blog", "status": "completed", "html": "<html>...</html>", "markdown": "# Blog index...", "durationMs": 1500 }, { "url": "https://example.com/blog/post-1", "status": "failed", "error": "Connection timeout", "durationMs": 5000 } ], "createdAt": "2024-01-01T12:00:00Z", "completedAt": "2024-01-01T12:00:15Z", "durationMs": 15000 } ``` --- ### Response Fields | Field | Type | Description | |-------|------|-------------| | `status` | string | `pending`, `processing`, `completed`, or `failed` | | `url` | string | The starting URL submitted for crawling | | `totalPages` | number | Total pages discovered and attempted | | `completedPages` | number | Pages successfully scraped | | `results` | array | Per-page results. Only present when completed. | | `error` | string | Error message. Only present when the entire job failed. | | `durationMs` | number | Total processing time in milliseconds. | ### Per-Page Result Fields | Field | Type | Description | |-------|------|-------------| | `url` | string | The page URL | | `status` | string | `completed` or `failed` | | `html` | string | Raw HTML content. Only when page completed. | | `markdown` | string | Markdown version of the content. Only when page completed. | | `error` | string | Error message. Only when page failed. | | `durationMs` | number | Per-page processing time in milliseconds. | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X GET https://api.anakin.io/v1/crawl/job_abc123xyz \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests import time job_id = "job_abc123xyz" while True: result = requests.get( f'https://api.anakin.io/v1/crawl/{job_id}', headers={'X-API-Key': 'your_api_key'} ) data = result.json() if data['status'] == 'completed': print(f"Crawled {data['completedPages']}/{data['totalPages']} pages:") for page in data['results']: if page['status'] == 'completed': print(f" {page['url']} — {len(page['markdown'])} chars") else: print(f" {page['url']} — FAILED: {page['error']}") break elif data['status'] == 'failed': print(f"Error: {data['error']}") break time.sleep(2) ``` </Tab> <Tab value="JavaScript"> ```javascript const jobId = 'job_abc123xyz'; const poll = async () => { const res = await fetch(`https://api.anakin.io/v1/crawl/${jobId}`, { headers: { 'X-API-Key': 'your_api_key' } }); const data = await res.json(); if (data.status === 'completed') { console.log(`Crawled ${data.completedPages}/${data.totalPages} pages:`); data.results.forEach(page => { if (page.status === 'completed') { console.log(` ${page.url} — ${page.markdown.length} chars`); } else { console.log(` ${page.url} — FAILED: ${page.error}`); } }); } else if (data.status === 'failed') { console.error(data.error); } else { setTimeout(poll, 2000); } }; poll(); ``` </Tab> </Tabs> For polling patterns, see the [Polling Jobs](/docs/api-reference/polling-jobs) reference. --- # POST Submit Crawl Job (/docs/api-reference/crawl/submit-crawl-job) <EndpointBanner method="POST" path="https://api.anakin.io/v1/crawl" /> Submit a website for multi-page crawling. The job discovers URLs and scrapes each page. Use the returned `jobId` to [poll for results](/docs/api-reference/crawl/get-crawl-result). --- ### Request Body ```json { "url": "https://example.com", "maxPages": 10, "includePatterns": ["/blog/*"], "excludePatterns": ["/admin/*"], "country": "us", "useBrowser": false, "webhook_url": "https://your-app.com/hooks/anakin" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `url` **required** | string | The starting URL to crawl from. Must be valid HTTP/HTTPS. | | `maxPages` | number | Maximum pages to crawl. Default `10`, max `100`. | | `depth` | number | Link-discovery depth before scraping. Default `1`, max `5`. Higher depth surfaces more candidate URLs (still capped by `maxPages`). | | `includePatterns` | string[] | URL glob patterns to include. Only URLs matching at least one pattern are crawled. | | `excludePatterns` | string[] | URL glob patterns to exclude. URLs matching any pattern are skipped. | | `country` | string | Country code for proxy routing. Default `"us"`. See [Supported Countries](/docs/api-reference/supported-countries). | | `useBrowser` | boolean | Use headless Chrome for rendering. Default `false`. Best for JS-heavy sites. | | `sessionId` | string | Browser session ID for authenticated crawling. See [Browser Sessions](/docs/api-reference/browser-sessions). | | `webhook_url` | string | URL to `POST` a signed `job.completed` / `job.failed` event to when the crawl settles — skip polling. See [Webhooks](/docs/api-reference/webhooks). | --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "jobId": "job_abc123xyz", "status": "pending" } ``` The job is processed asynchronously. Use the `jobId` with [GET /v1/crawl/\{id\}](/docs/api-reference/crawl/get-crawl-result) to check status and retrieve results. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/crawl \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "maxPages": 20, "includePatterns": ["/blog/*"], "country": "us" }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/crawl', headers={'X-API-Key': 'your_api_key'}, json={ 'url': 'https://example.com', 'maxPages': 20, 'includePatterns': ['/blog/*'], 'excludePatterns': ['/admin/*'], 'country': 'us' } ) data = response.json() print(f"Job submitted: {data['jobId']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/crawl', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ url: 'https://example.com', maxPages: 20, includePatterns: ['/blog/*'], excludePatterns: ['/admin/*'], country: 'us' }) }); const data = await response.json(); console.log(data.jobId); ``` </Tab> </Tabs> --- # GET Get Map Result (/docs/api-reference/map/get-map-result) <EndpointBanner method="GET" path="https://api.anakin.io/v1/map/{id}" /> Retrieve the status and results of a map job. Use this to poll for completion after submitting a [map request](/docs/api-reference/map/submit-map-job). --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The job ID returned from the submit endpoint | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "id": "job_abc123xyz", "status": "completed", "url": "https://example.com", "links": [ "https://example.com/about", "https://example.com/blog", "https://example.com/blog/post-1", "https://example.com/contact", "https://example.com/pricing" ], "totalLinks": 5, "externalLinks": [ "https://partner-site.com/page" ], "totalExternalLinks": 1, "createdAt": "2024-01-01T12:00:00Z", "completedAt": "2024-01-01T12:00:03Z", "durationMs": 3000 } ``` --- ### Response Fields | Field | Type | Description | |-------|------|-------------| | `status` | string | `pending`, `processing`, `completed`, or `failed` | | `url` | string | The original URL submitted for discovery | | `links` | string[] | Discovered same-domain URLs (includes subdomains when `includeSubdomains` was set). Only present when completed. | | `totalLinks` | number | Total count of same-domain URLs discovered. | | `externalLinks` | string[] | Discovered URLs on other domains (subdomains count as external unless `includeSubdomains` was set). Present when `includeExternalLinks` was set — but internal links fill `limit` first, so external links are only returned when internal links didn't already reach the cap. | | `totalExternalLinks` | number | Total count of external URLs discovered. | | `error` | string | Error message. Only present when failed. | | `durationMs` | number | Processing time in milliseconds. | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X GET https://api.anakin.io/v1/map/job_abc123xyz \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests import time job_id = "job_abc123xyz" while True: result = requests.get( f'https://api.anakin.io/v1/map/{job_id}', headers={'X-API-Key': 'your_api_key'} ) data = result.json() if data['status'] == 'completed': print(f"Found {data['totalLinks']} URLs:") for link in data['links']: print(f" {link}") break elif data['status'] == 'failed': print(f"Error: {data['error']}") break time.sleep(1) ``` </Tab> <Tab value="JavaScript"> ```javascript const jobId = 'job_abc123xyz'; const poll = async () => { const res = await fetch(`https://api.anakin.io/v1/map/${jobId}`, { headers: { 'X-API-Key': 'your_api_key' } }); const data = await res.json(); if (data.status === 'completed') { console.log(`Found ${data.totalLinks} URLs:`); data.links.forEach(link => console.log(` ${link}`)); } else if (data.status === 'failed') { console.error(data.error); } else { setTimeout(poll, 1000); } }; poll(); ``` </Tab> </Tabs> For polling patterns, see the [Polling Jobs](/docs/api-reference/polling-jobs) reference. --- # POST Submit Map Job (/docs/api-reference/map/submit-map-job) <EndpointBanner method="POST" path="https://api.anakin.io/v1/map" /> Submit a URL for discovery. The job is processed asynchronously — use the returned `jobId` to [poll for results](/docs/api-reference/map/get-map-result). --- ### Request Body ```json { "url": "https://example.com", "includeSubdomains": false, "includeExternalLinks": false, "limit": 100, "depth": 2, "limitPerLevel": 100, "search": "", "useBrowser": false, "webhook_url": "https://your-app.com/hooks/anakin" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `url` **required** | string | The URL to discover links from. Must be valid HTTP/HTTPS. | | `includeSubdomains` | boolean | Include links to subdomains (e.g. `blog.example.com`). Default `false`. | | `includeExternalLinks` | boolean | Also collect links to other domains (returned separately as `externalLinks`). Default `false`. | | `limit` | number | Maximum URLs to return. Default `100`, max `5000`. | | `depth` | number | How many link hops to follow from the seed. Default `2`, max `5`. | | `limitPerLevel` | number | Max pages fetched per depth level beyond the seed. Default `100`, max `500`. | | `search` | string | Filter discovered URLs — only return URLs containing this string. | | `useBrowser` | boolean | Use headless Chrome to render the page before extracting links. Default `false`. Best for JS-heavy sites. | | `sessionId` | string | Browser session ID for authenticated discovery. See [Browser Sessions](/docs/api-reference/browser-sessions). | | `webhook_url` | string | URL to `POST` a signed `job.completed` / `job.failed` event to when the map job settles — skip polling. See [Webhooks](/docs/api-reference/webhooks). | --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "jobId": "job_abc123xyz", "status": "pending" } ``` The job is processed asynchronously. Use the `jobId` with [GET /v1/map/\{id\}](/docs/api-reference/map/get-map-result) to check status and retrieve results. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/map \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "includeSubdomains": false, "limit": 100 }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/map', headers={'X-API-Key': 'your_api_key'}, json={ 'url': 'https://example.com', 'includeSubdomains': False, 'limit': 100, 'search': '/blog/' } ) data = response.json() print(f"Job submitted: {data['jobId']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/map', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ url: 'https://example.com', includeSubdomains: false, limit: 100, search: '/blog/' }) }); const data = await response.json(); console.log(data.jobId); ``` </Tab> </Tabs> --- # Alerts, Deliveries & Webhooks (/docs/api-reference/monitoring/alerts) When a monitor detects a change, it delivers an alert to the channels you configured (`alertWebhookUrl` and/or `alertEmails`). Every attempt is logged as a **delivery**. --- ## Webhook payload Anakin POSTs this JSON body to your `alertWebhookUrl` on each detected change: ```json { "type": "monitor.change", "monitorId": "b1e7c2a4-…", "url": "https://example.com/product/123", "watchMode": "specific_data", "changeId": "c_9a1…", "changedAt": "2026-01-01T12:00:00Z", "changedFields": ["price"], "summary": "The price dropped from $19.99 to $14.99.", "diff": { "before": { "price": 19.99 }, "after": { "price": 14.99 } } } ``` Each request carries these headers: | Header | Description | |--------|-------------| | `X-Anakin-Signature` | `sha256=` + hex HMAC-SHA256 of the **raw request body**, keyed with the monitor's `alertWebhookSecret`. | | `X-Anakin-Timestamp` | Unix seconds when the request was signed. | | `X-Anakin-Delivery-Id` | The **change** id being delivered — stable across retries of the same change, so use it as your idempotency key. | **Verify the signature** by recomputing the HMAC over the exact raw body and comparing (constant-time) to `X-Anakin-Signature`: ```python import hmac, hashlib def verify(raw_body: bytes, signature_header: str, secret: str) -> bool: expected = "sha256=" + hmac.new(secret.encode(), raw_body, hashlib.sha256).hexdigest() return hmac.compare_digest(expected, signature_header) ``` For the full webhook integration guide (retries, delivery semantics), see [Website Monitoring Webhooks](/docs/integrations/website-monitoring/webhooks). --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors/{id}/test-alert" /> Send a sample alert to the monitor's configured channels so you can verify your endpoint and signature handling before a real change fires. ### Response <StatusBadge code={200} text="OK" /> ```json { "success": true, "status": 200 } ``` `status` is the HTTP status your webhook returned (or the email send result). Errors: `400 invalid_request` if no alert channel is configured; `502` if delivery failed (the body includes `status` and a `message`). --- <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}/deliveries" /> List recent alert-delivery attempts, newest first (up to 200). ### Response <StatusBadge code={200} text="OK" /> ```json { "deliveries": [ { "id": "d_5f2…", "monitorId": "b1e7c2a4-…", "changeId": "c_9a1…", "event": "change", "channel": "webhook", "url": "https://your-app.com/hooks/anakin", "status": "success", "httpStatus": 200, "attempts": 1, "createdAt": "2026-01-01T12:00:01Z" } ] } ``` | Field | Type | Description | |-------|------|-------------| | `event` | string | `change` or `test`. | | `channel` | string | `webhook` or `email`. | | `url` | string | Destination — the webhook URL or the comma-separated recipients. | | `status` | string | `pending`, `success`, `failed`, or `exhausted` (retries used up). | | `httpStatus` | number \| null | HTTP status returned by the webhook. | | `attempts` | number | Delivery attempts so far. | | `error` | string \| null | Failure reason, when failed. | | `nextAttemptAt` | string \| null | When the next retry is scheduled (while `pending`). | --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors/{id}/deliveries/{deliveryId}/retry" /> Re-queue a failed or exhausted delivery for another attempt. ### Response <StatusBadge code={200} text="OK" /> ```json { "success": true } ``` Returns `404 not_found` if the delivery doesn't belong to this monitor. --- ### Code Examples <Tabs items={["cURL", "Python"]}> <Tab value="cURL"> ```bash # Fire a test alert curl -X POST https://api.anakin.io/v1/monitors/b1e7c2a4-.../test-alert -H "X-API-Key: your_api_key" # Inspect deliveries curl https://api.anakin.io/v1/monitors/b1e7c2a4-.../deliveries -H "X-API-Key: your_api_key" # Retry one curl -X POST https://api.anakin.io/v1/monitors/b1e7c2a4-.../deliveries/d_5f2.../retry -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests headers = {"X-API-Key": "your_api_key"} mid = "b1e7c2a4-..." print(requests.post(f"https://api.anakin.io/v1/monitors/{mid}/test-alert", headers=headers).json()) deliveries = requests.get(f"https://api.anakin.io/v1/monitors/{mid}/deliveries", headers=headers).json()["deliveries"] for d in deliveries: if d["status"] in ("failed", "exhausted"): requests.post(f"https://api.anakin.io/v1/monitors/{mid}/deliveries/{d['id']}/retry", headers=headers) ``` </Tab> </Tabs> --- # POST Pause / Resume / End Date / Run Now (/docs/api-reference/monitoring/control-monitor) These action endpoints control a monitor's schedule without editing its full configuration. --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors/{id}/pause" /> Pause the schedule. The monitor stops checking until resumed. Returns the updated [Monitor object](/docs/api-reference/monitoring) with `isActive: false`. --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors/{id}/resume" /> Resume a paused or **expired** monitor. Returns the updated [Monitor object](/docs/api-reference/monitoring) with `isActive: true` and `expired: false`. Optionally start a fresh end date by sending a body — `{ "expiresAt": "2026-08-06" }` runs it until that date; omit the body (or send `null`) to resume with **no end date**. A future ISO 8601 timestamp or `YYYY-MM-DD` is accepted. Returns `402 monitor_limit_reached` if resuming would exceed your plan's active-monitor cap. --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors/{id}/expiry" /> Change **only** the end date of a running monitor — extend it or remove it — without pausing/resuming. Body `{ "expiresAt": "2026-09-05" }` sets a new future date; `{ "expiresAt": null }` (or an empty value) removes the end date so it runs indefinitely. Returns the updated [Monitor object](/docs/api-reference/monitoring). After an end date passes, a background sweep auto-pauses the monitor and sets `expired: true` (a distinct state from a manual pause) — `resume` it to start a new window. --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors/{id}/run" /> Queue an **immediate** check now, independent of the schedule (the regular cadence is unaffected). A successful run also clears a stale `sessionExpired` flag. ### Response <StatusBadge code={200} text="OK" /> ```json { "success": true, "jobId": "job_abc123xyz" } ``` | Field | Type | Description | |-------|------|-------------| | `jobId` | string | The scrape job enqueued for this check (url/site monitors). | For **wire monitors** the check runs against the Wire engine instead of the scrape queue, so the response carries no `jobId`: ```json { "success": true, "message": "Wire check started" } ``` **Errors** | Status | `error` | When | |--------|---------|------| | `409` | `session_expired` | The monitor's saved browser session is no longer valid. Re-authenticate and re-select the session, then run again. | | `404` | `not_found` | The monitor doesn't exist or isn't yours. | | `503` | `wire_unavailable` | Wire monitors only — the Wire engine couldn't be reached; try again shortly. | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/monitors/b1e7c2a4-.../pause -H "X-API-Key: your_api_key" curl -X POST https://api.anakin.io/v1/monitors/b1e7c2a4-.../resume -H "X-API-Key: your_api_key" curl -X POST https://api.anakin.io/v1/monitors/b1e7c2a4-.../run -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests headers = {"X-API-Key": "your_api_key"} base = "https://api.anakin.io/v1/monitors/b1e7c2a4-..." requests.post(f"{base}/pause", headers=headers) requests.post(f"{base}/resume", headers=headers) run = requests.post(f"{base}/run", headers=headers).json() print("Check queued:", run["jobId"]) ``` </Tab> <Tab value="JavaScript"> ```javascript const headers = { "X-API-Key": "your_api_key" }; const base = "https://api.anakin.io/v1/monitors/b1e7c2a4-..."; await fetch(`${base}/pause`, { method: "POST", headers }); await fetch(`${base}/resume`, { method: "POST", headers }); const run = await (await fetch(`${base}/run`, { method: "POST", headers })).json(); console.log("Check queued:", run.jobId); ``` </Tab> </Tabs> --- # POST Create Monitor (/docs/api-reference/monitoring/create-monitor) <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors" /> Create a monitor that checks a URL every `intervalMinutes` and records a change whenever the page differs from the previous check. A monitor has a **`scope`**: `page` *(default)* watches a single URL, `site` crawls the whole site each run and reports pages **added**, **removed**, and **changed**, and `wire` runs a managed [Wire](/docs/api-reference/monitoring/wire-monitors) API action each run and diffs the JSON it returns. See [Site monitors](/docs/api-reference/monitoring/site-monitors) and [Wire monitors](/docs/api-reference/monitoring/wire-monitors) for the scope-specific config, endpoints, and change shape. --- ### Request Body ```json { "url": "https://example.com/product/123", "watchMode": "specific_data", "outputSchema": { "type": "object", "properties": { "price": { "type": "number" }, "inStock": { "type": "boolean" } }, "required": ["price"] }, "intervalMinutes": 60, "aiMode": true, "aiGoal": "only when the price drops or it goes out of stock", "country": "us", "alertWebhookUrl": "https://your-app.com/hooks/anakin", "alertEmails": "you@example.com" } ``` Only `url` and `intervalMinutes` are **required**. Every other field is optional — the defaults below are used when you omit it. | Parameter | Type | Description | |-----------|------|-------------| | `url` **required** | string | Page (or, for `site` scope, the root URL) to monitor. Must start with `http://` or `https://`. | | `intervalMinutes` **required** | number | How often to check, in minutes. Minimum `15`. | | `scope` *optional* | string | `page` *(default)* — watch this one URL. `site` — crawl the whole site each run and diff the page set. `wire` — run a [Wire](/docs/api-reference/monitoring/wire-monitors) action each run and diff its JSON. | | `maxPages` *site only* | number | Organic crawl budget — how many **discovered** pages to crawl per run, *on top of* any pinned `includePatterns` URLs. Total per run is capped at `50`. Bills **1 credit per page** crawled. | | `maxDepth` *site only* | number | Link hops to follow from the root during the organic crawl (`1`–`5`). Default `2`. | | `includePatterns` *site only* | string[] | **Pin** pages to always track. A **full URL** is crawled **directly** every run — fetched even if discovery misses it — *in addition to* the organic crawl of up to `maxPages` more pages (this is how the dashboard's Inspect/Import hand-picking is stored). A **bare path glob** (e.g. `/blog/*`) filters the organically-discovered pages. URLs must be on the monitor's **own site** — off-site URLs are dropped. | | `excludePatterns` *site only* | string[] | Skip URLs matching these globs. | | `wireActionId` **required for `wire`** | string | The Wire action to run each check, e.g. `amazon.search_products`. See [Wire monitors](/docs/api-reference/monitoring/wire-monitors). | | `wireCatalogSlug` *wire only* | string | Catalogue slug of the Wire site the action belongs to, e.g. `amazon`. Optional but recommended. | | `wireCredentialId` *wire only* | string | A connected [identity's](/docs/api-reference/monitoring/wire-monitors) credential id, required only when the action's auth mode is `required`. | | `wireParams` *wire only* | object | Parameters passed to the action, e.g. `{ "query": "usb-c cable" }`. Shape depends on the action. | | `wireWatchPaths` *wire only* | string[] | JSON paths from the action's result to diff (e.g. `["results.0.rank"]`). Empty = diff the whole payload. See [Wire monitors](/docs/api-reference/monitoring/wire-monitors#watch-specific-fields). | | `watchMode` *optional* | string | `full_page` *(default)* or `specific_data`. Site monitors always use `full_page`; wire monitors always diff the returned JSON (`specific_data`, set server-side). | | `watchFormat` *optional* | string | `full_page` only — what to diff: `markdown` *(default)*, `html`, or `cleaned_html`. Ignored for `specific_data`. | | `outputSchema` *required for `specific_data`* | object | JSON Schema of the fields to track (AI-extracted). Only used — and required — in `specific_data` mode; ignored for `full_page`. | | `aiMode` *optional* | boolean | Filter trivial noise and summarize meaningful changes. Adds **+1 credit/check**. Requires an LLM-enabled server. Default `false`. | | `aiGoal` *optional* | string | Optional even when `aiMode` is on. A natural-language hint that *narrows* what counts as meaningful (e.g. "only when the price drops or it goes out of stock"). Omit it and the AI flags any meaningful change on its own. | | `useBrowser` *optional* | boolean | Render with a headless browser (needed for JS-heavy pages). Default `false`. Forced `true` when `sessionId` is set. | | `country` *optional* | string | Proxy country code. Default `"us"`. See [Supported Countries](/docs/api-reference/supported-countries). | | `sessionId` *optional* | string | Saved [browser session](/docs/api-reference/browser-sessions) for monitoring logged-in pages. | | `isActive` *optional* | boolean | Start active or paused. Default `true`. | | `expiresAt` *optional* | string | End date — an ISO 8601 timestamp or `YYYY-MM-DD` date. After it passes, the monitor auto-pauses (`expired: true`) and can be resumed. Must be in the **future**. Null/omitted = **no end date** (runs until you pause it or run out of credits). | | `alertWebhookUrl` *optional* | string | Webhook to POST each detected change to (HMAC-signed). See [Alerts](/docs/api-reference/monitoring/alerts). | | `alertEmails` *optional* | string | Comma-separated email recipients. Requires email alerts to be enabled on the server. | --- ### Response <StatusBadge code={201} text="Created" /> Returns the full [Monitor object](/docs/api-reference/monitoring). When you set `alertWebhookUrl`, the response includes a freshly generated `alertWebhookSecret` — **store it now**, it's what you use to verify webhook signatures. ```json { "id": "b1e7c2a4-…", "url": "https://example.com/product/123", "watchMode": "specific_data", "intervalMinutes": 60, "isActive": true, "aiMode": true, "creditCostPerRun": 4, "nextRunAt": "2026-01-01T12:00:00Z", "alertWebhookUrl": "https://your-app.com/hooks/anakin", "alertWebhookSecret": "whsec_9f2c…", "createdAt": "2026-01-01T11:00:00Z", "updatedAt": "2026-01-01T11:00:00Z" } ``` The first check runs on the next scheduler tick (usually within a minute). Poll [`GET /v1/monitors/{id}`](/docs/api-reference/monitoring/list-monitors) for `lastCheckedAt`, or list [changes](/docs/api-reference/monitoring/snapshots-and-changes). **Errors** | Status | `error` | When | |--------|---------|------| | `400` | `invalid_request` | Missing/invalid `url`, `intervalMinutes < 15`, bad `watchMode`/`watchFormat`, missing `outputSchema` for `specific_data`, or malformed emails. | | `402` | `monitor_limit_reached` | You're at your plan's active-monitor cap (Free 5 / Pro 20 / Scale 100). | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/monitors \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com/product/123", "watchMode": "specific_data", "outputSchema": { "type": "object", "properties": { "price": { "type": "number" } }, "required": ["price"] }, "intervalMinutes": 60, "alertWebhookUrl": "https://your-app.com/hooks/anakin" }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( "https://api.anakin.io/v1/monitors", headers={"X-API-Key": "your_api_key"}, json={ "url": "https://example.com/product/123", "watchMode": "specific_data", "outputSchema": { "type": "object", "properties": {"price": {"type": "number"}}, "required": ["price"], }, "intervalMinutes": 60, "alertWebhookUrl": "https://your-app.com/hooks/anakin", }, ) monitor = res.json() print("Created", monitor["id"], "— save secret:", monitor.get("alertWebhookSecret")) ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch("https://api.anakin.io/v1/monitors", { method: "POST", headers: { "X-API-Key": "your_api_key", "Content-Type": "application/json" }, body: JSON.stringify({ url: "https://example.com/product/123", watchMode: "specific_data", outputSchema: { type: "object", properties: { price: { type: "number" } }, required: ["price"] }, intervalMinutes: 60, alertWebhookUrl: "https://your-app.com/hooks/anakin", }), }); const monitor = await res.json(); console.log("Created", monitor.id, "— save secret:", monitor.alertWebhookSecret); ``` </Tab> </Tabs> --- # GET List & Get Monitors (/docs/api-reference/monitoring/list-monitors) <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors" /> List all monitors you own. ### Response <StatusBadge code={200} text="OK" /> ```json { "monitors": [ { "id": "b1e7c2a4-…", "url": "https://example.com/product/123", "watchMode": "specific_data", "intervalMinutes": 60, "isActive": true, "sessionExpired": false, "aiMode": true, "creditCostPerRun": 4, "nextRunAt": "2026-01-01T12:00:00Z", "lastCheckedAt": "2026-01-01T11:00:00Z", "createdAt": "2025-12-01T09:00:00Z", "updatedAt": "2025-12-15T09:00:00Z" } ] } ``` Each item is a full [Monitor object](/docs/api-reference/monitoring). --- <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}" /> Fetch a single monitor by ID. ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The monitor ID. | ### Response <StatusBadge code={200} text="OK" /> Returns the [Monitor object](/docs/api-reference/monitoring). Returns `404 not_found` if the monitor doesn't exist or isn't yours. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # List all monitors curl https://api.anakin.io/v1/monitors \ -H "X-API-Key: your_api_key" # Get one curl https://api.anakin.io/v1/monitors/b1e7c2a4-... \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests headers = {"X-API-Key": "your_api_key"} monitors = requests.get("https://api.anakin.io/v1/monitors", headers=headers).json()["monitors"] for m in monitors: print(m["id"], m["url"], "active" if m["isActive"] else "paused") one = requests.get(f"https://api.anakin.io/v1/monitors/{monitors[0]['id']}", headers=headers).json() print(one["lastCheckedAt"]) ``` </Tab> <Tab value="JavaScript"> ```javascript const headers = { "X-API-Key": "your_api_key" }; const { monitors } = await (await fetch("https://api.anakin.io/v1/monitors", { headers })).json(); monitors.forEach((m) => console.log(m.id, m.url, m.isActive ? "active" : "paused")); const one = await (await fetch(`https://api.anakin.io/v1/monitors/${monitors[0].id}`, { headers })).json(); console.log(one.lastCheckedAt); ``` </Tab> </Tabs> --- # GET Site Runs & Pages (/docs/api-reference/monitoring/site-monitors) A **site monitor** (`scope: "site"`) crawls the whole site on each interval, compares the set of pages against the previous run, and reports pages **added**, **removed**, and **changed** (with a per‑page content diff). It reuses the same schedule, alerting (webhook + email), and AI meaningful‑change filtering as page monitors — the unit of work is a crawl instead of a single fetch. Create one by setting `scope: "site"` and the crawl config on [`POST /v1/monitors`](/docs/api-reference/monitoring/create-monitor): ```json { "url": "https://example.com", "scope": "site", "maxPages": 25, "maxDepth": 2, "excludePatterns": ["/cart", "/checkout"], "intervalMinutes": 720, "alertWebhookUrl": "https://your-app.com/hooks/anakin" } ``` **Pinned pages + organic crawl.** By default the monitor crawls the site organically — discovered URLs are sorted before the `maxPages` cap, so a stable site yields the same pages each run. To guarantee specific pages are always watched, **pin them**: put their full URLs in `includePatterns` (the dashboard's "Inspect" and "Import" flows do this). Pinned URLs are crawled **directly** every run (fetched even if discovery misses them) **in addition to** the organic crawl of up to `maxPages` more pages — total capped at 50/run. Pinned URLs must be on the monitor's own site. You can also **remove** pages you don't care about (below) so their crawl budget goes to others. --- ### Credits & limits - **1 credit per page** crawled per run (a browser‑rendered crawl costs more, same as the URL scraper). - `maxPages` is capped at **50** per run. - Site monitors have their own active‑monitor caps per plan, lower than page monitors since each run crawls many pages: **Free 1 · Pro 5 · Scale 20**. --- ### The change payload For a site monitor, each entry from [`GET /v1/monitors/{id}/changes`](/docs/api-reference/monitoring/snapshots-and-changes) has a `diffJson` of type `"site"`: ```json { "type": "site", "runId": "…", "counts": { "added": 2, "removed": 1, "changed": 4, "pages": 25 }, "added": [{ "url": "https://example.com/new", "afterHash": "9f3a…" }], "removed": [{ "url": "https://example.com/gone", "beforeHash": "1b7c…" }], "changed": [{ "url": "https://example.com/pricing", "beforeHash": "1b7c…", "afterHash": "9f3a…", "summary": "Price rose from $19 to $24." }] } ``` Each entry carries the page's content hashes: `afterHash` (added & changed) and `beforeHash` (removed & changed). Fetch either side's page body with the content endpoint below to render a before/after diff. `summary` on a changed entry is present only when AI mode is on. The same digest is delivered to your webhook/email as one alert per run. --- ### Page content (for diffs) <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}/content?hash=…" /> Fetch a tracked page's stored body by its **content hash** — the `beforeHash` / `afterHash` from a change entry above. Use it to render before/after text diffs; you pass a hash, never a storage path. ### Response <StatusBadge code={200} text="OK" /> ```json { "available": true, "content": "# Pricing\n\n…" } ``` | Field | Type | Description | |-------|------|-------------| | `available` | boolean | `false` when no body is stored for that hash (e.g. storage not configured). | | `content` | string | The page body in the monitor's `watchFormat`. Empty when `available` is `false`. | `hash` must be a 64‑character content hash; a missing or malformed `hash` returns `400`. --- ### List runs <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}/runs" /> Each site‑monitor check is a **run**. Returns the runs newest‑first with their set‑diff counts. ```json { "runs": [ { "id": "…", "jobId": "…", "status": "completed", "startedAt": "2026-07-06T12:00:00Z", "finishedAt": "2026-07-06T12:01:30Z", "pageCount": 25, "addedCount": 2, "removedCount": 1, "changedCount": 4 } ] } ``` --- ### Tracked pages <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}/pages" /> The pages currently tracked (from the latest run) and the URLs you've removed from tracking. ```json { "tracked": ["https://example.com", "https://example.com/pricing"], "excluded": ["https://example.com/blog"] } ``` <EndpointBanner method="DELETE" path="https://api.anakin.io/v1/monitors/{id}/pages?url=…" /> Stop tracking a page. It's excluded from every future crawl (freeing budget for other pages) and never reported as removed. Returns `{ "success": true, "excludedUrls": [...] }`. <EndpointBanner method="POST" path="https://api.anakin.io/v1/monitors/{id}/pages/restore?url=…" /> Resume tracking a previously‑removed page. Returns `{ "success": true, "excludedUrls": [...] }`. --- # GET Snapshots & Changes (/docs/api-reference/monitoring/snapshots-and-changes) Every check stores a **snapshot**. When a snapshot differs from the previous one, a **change** is recorded. These endpoints let you read that history. --- <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}/changes" /> List detected changes for a monitor, newest first (up to 200). ### Response <StatusBadge code={200} text="OK" /> ```json { "changes": [ { "id": "c_9a1…", "monitorId": "b1e7c2a4-…", "snapshotId": "s_77b…", "changedAt": "2026-01-01T12:00:00Z", "summary": "The price dropped from $19.99 to $14.99.", "diffJson": { "type": "fields", "changedFields": ["price"], "before": { "price": 19.99 }, "after": { "price": 14.99 }, "aiMeaningful": [ { "description": "Price dropped", "before": "$19.99", "after": "$14.99" } ] }, "createdAt": "2026-01-01T12:00:00Z" } ] } ``` | Field | Type | Description | |-------|------|-------------| | `changedAt` | string | When the change was detected. | | `summary` | string \| null | Human-readable summary (AI-generated when `aiMode` is on). | | `diffJson` | object | Structured diff (see below). | **`diffJson` fields** | Field | Type | Description | |-------|------|-------------| | `type` | string | `fields` (specific-data monitors, incl. wire), `content` (full-page monitors), or `site` (site monitors). | | `changedFields` | string[] | Field keys that changed (`fields` diffs). | | `before` / `after` | object | Extracted values before/after (`fields` diffs). | | `beforeSnapshotId` / `afterSnapshotId` | string | Snapshot IDs for a `content` text diff — fetch each side's body via the content endpoint below. | | `stats` | object | `{ "adds": number, "dels": number }` — line counts, present on `content` diffs only. | | `aiMeaningful` | array | AI-curated substantive changes, each `{ description, before, after }` (present in `aiMode`). | --- <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}/snapshots" /> List captured snapshots, newest first (up to 200). ### Response <StatusBadge code={200} text="OK" /> ```json { "snapshots": [ { "id": "s_77b…", "monitorId": "b1e7c2a4-…", "capturedAt": "2026-01-01T12:00:00Z", "contentHash": "a1b2c3…", "extractedData": { "price": 14.99 }, "createdAt": "2026-01-01T12:00:00Z" } ] } ``` | Field | Type | Description | |-------|------|-------------| | `contentHash` | string | Hash of the compared content — identical hashes mean no change. | | `extractedData` | object \| null | Extracted fields (`specific_data` mode). | To read a `full_page` snapshot's stored body (for rendering a text diff), call the snapshot content endpoint below with the snapshot's `id` — you never handle a storage path. --- <EndpointBanner method="GET" path="https://api.anakin.io/v1/monitors/{id}/snapshots/{snapshotId}/content" /> Fetch the stored page body for a `full_page` snapshot — use it to render before/after text diffs. Returns an empty body for `specific_data` snapshots (their data is in `extractedData`). ### Response <StatusBadge code={200} text="OK" /> ```json { "available": true, "content": "# Home page content…", "capturedAt": "2026-01-01T12:00:00Z", "contentHash": "a1b2c3…" } ``` | Field | Type | Description | |-------|------|-------------| | `available` | boolean | `false` when no body is stored (e.g. `specific_data`, or storage not configured). | | `content` | string | The captured body in the monitor's `watchFormat`. Empty when `available` is `false`. | --- ### Code Examples <Tabs items={["cURL", "Python"]}> <Tab value="cURL"> ```bash curl https://api.anakin.io/v1/monitors/b1e7c2a4-.../changes -H "X-API-Key: your_api_key" curl https://api.anakin.io/v1/monitors/b1e7c2a4-.../snapshots -H "X-API-Key: your_api_key" curl https://api.anakin.io/v1/monitors/b1e7c2a4-.../snapshots/s_77b.../content -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests headers = {"X-API-Key": "your_api_key"} mid = "b1e7c2a4-..." changes = requests.get(f"https://api.anakin.io/v1/monitors/{mid}/changes", headers=headers).json()["changes"] for ch in changes: print(ch["changedAt"], ch.get("summary")) ``` </Tab> </Tabs> --- # PUT / DELETE Update & Delete Monitor (/docs/api-reference/monitoring/update-monitor) <EndpointBanner method="PUT" path="https://api.anakin.io/v1/monitors/{id}" /> Update a monitor. The body is the **same shape as [Create](/docs/api-reference/monitoring/create-monitor)** — send the full desired configuration, not a partial patch. ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The monitor ID. | ### Request Body Identical to [Create Monitor](/docs/api-reference/monitoring/create-monitor). A few notes on update semantics: - **Signing secret is preserved.** Changing `alertWebhookUrl` keeps the existing `alertWebhookSecret`; a secret is only generated the first time you add a webhook. - **Changing `watchMode` or `watchFormat` re-baselines the monitor** — stored snapshots are cleared so the next check starts fresh (no spurious "changed" alert). - **Pausing:** send `"isActive": false` (or use the dedicated [pause endpoint](/docs/api-reference/monitoring/control-monitor)). ### Response <StatusBadge code={200} text="OK" /> Returns the updated [Monitor object](/docs/api-reference/monitoring). `404 not_found` if it isn't yours; `400 invalid_request` on validation failure. --- <EndpointBanner method="DELETE" path="https://api.anakin.io/v1/monitors/{id}" /> Permanently delete a monitor and all of its snapshots and change history. This cannot be undone. ### Response <StatusBadge code={200} text="OK" /> ```json { "success": true } ``` Returns `404 not_found` if the monitor doesn't exist or isn't yours. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # Update — change the interval to every 4 hours curl -X PUT https://api.anakin.io/v1/monitors/b1e7c2a4-... \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com/product/123", "watchMode": "full_page", "intervalMinutes": 240 }' # Delete curl -X DELETE https://api.anakin.io/v1/monitors/b1e7c2a4-... \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests headers = {"X-API-Key": "your_api_key"} mid = "b1e7c2a4-..." requests.put( f"https://api.anakin.io/v1/monitors/{mid}", headers=headers, json={"url": "https://example.com/product/123", "watchMode": "full_page", "intervalMinutes": 240}, ) requests.delete(f"https://api.anakin.io/v1/monitors/{mid}", headers=headers) ``` </Tab> <Tab value="JavaScript"> ```javascript const headers = { "X-API-Key": "your_api_key", "Content-Type": "application/json" }; const mid = "b1e7c2a4-..."; await fetch(`https://api.anakin.io/v1/monitors/${mid}`, { method: "PUT", headers, body: JSON.stringify({ url: "https://example.com/product/123", watchMode: "full_page", intervalMinutes: 240 }), }); await fetch(`https://api.anakin.io/v1/monitors/${mid}`, { method: "DELETE", headers }); ``` </Tab> </Tabs> --- # Wire Monitors (/docs/api-reference/monitoring/wire-monitors) A **wire monitor** (`scope: "wire"`) runs a managed **[Wire](/docs/api-reference/wire)** API action on each interval and diffs the JSON it returns. Use it when the thing you want to watch lives behind a login, an anti-bot wall, or a site with no public API of its own — Wire returns clean structured data, and the monitor tracks it for changes. It reuses the same schedule, alerting (webhook + email), and AI meaningful-change filtering as page and site monitors. The only difference is where the data comes from: instead of scraping a URL, each check submits a Wire **task** and diffs the returned object. ### How it works 1. Pick a Wire **action** from the [catalogue](/docs/api-reference/wire) (e.g. `amazon.search_products`) and, if it needs auth, a connected **identity**. 2. On every check the monitor submits a Wire task with your `wireParams` and waits for the result. 3. The returned JSON is stored as a **snapshot** (as `specific_data`). 4. If it differs from the previous result, a **change** is recorded and any configured alerts fire. The whole returned object is compared — turn on `aiMode` to be alerted only on meaningful changes (and get a plain-language summary), optionally narrowed with `aiGoal`. --- ### Create a wire monitor Set `scope: "wire"` and `wireActionId` on [`POST /v1/monitors`](/docs/api-reference/monitoring/create-monitor). `url` is still required — use the Wire site's URL; it's only used for display and as the monitor's title. ```json { "url": "https://www.amazon.com", "scope": "wire", "wireCatalogSlug": "amazon", "wireActionId": "amazon.search_products", "wireParams": { "query": "usb-c cable", "country": "US" }, "intervalMinutes": 720, "aiMode": true, "aiGoal": "only when a new top result appears or a price changes", "alertWebhookUrl": "https://your-app.com/hooks/anakin" } ``` | Field | Required | Description | |-------|----------|-------------| | `scope` | yes | Must be `"wire"`. | | `wireActionId` | yes | The action to run each check, e.g. `amazon.search_products`. Browse actions in the [Wire catalogue](/docs/api-reference/wire). | | `wireCatalogSlug` | recommended | Catalogue slug of the Wire site the action belongs to, e.g. `amazon`. | | `wireCredentialId` | when the action's auth is `required` | The credential id of a connected [identity](/docs/api-reference/wire) for that site. Actions with auth `none` don't take one; `optional` actions return richer data with one. | | `wireParams` | depends on the action | Parameters passed to the action. The shape is defined by the action (each has `name` / `type` / `required`). | | `wireWatchPaths` | optional | JSON paths (from the action's result) to diff. **Empty = diff the whole payload.** See [Watch specific fields](#watch-specific-fields). | `watchMode` is forced to `specific_data` server-side and `outputSchema` is ignored — the Wire engine already returns structured JSON, so there's nothing to extract. Any `maxPages` / `maxDepth` / crawl fields are ignored too. > **Auth.** On the dashboard you never handle an API key — pick the site, action, params and identity in the **Monitor via Wire** flow and it's wired to your account automatically. Over the REST API you authenticate with your `X-API-Key` as usual; the scheduler calls the Wire engine on your behalf using the same account, so the action runs as **you** (your identities, your credits). --- ### Credits - `creditCostPerRun` is reported as a **nominal `1`** on the monitor object. The real cost is the action's own `creditsPerCall`, billed by the Wire engine per task. - Turning on `aiMode` adds **+1 credit per check** for the meaningful-change pass. - A failed Wire task (the action errors or times out) is not billed a monitor credit and does not record a change. --- ### Watch specific fields By default a wire monitor diffs the **whole** returned payload. For actions that return volatile or list-shaped data (a search result set, rankings, rotating image URLs), that can flag a change on almost every check. Narrow it with **`wireWatchPaths`** — a list of JSON paths into the result; only those are compared. Each path is dot-separated, with numeric array indices and a `*` wildcard over arrays: ```json { "url": "https://www.imdb.com", "scope": "wire", "wireCatalogSlug": "imdb", "wireActionId": "im_search", "wireParams": { "query": "fast and furious" }, "wireWatchPaths": ["results.0.id", "results.0.rank", "results.*.title"], "intervalMinutes": 720 } ``` - `results.0.rank` — the top result's rank. - `results.*.title` — every result's title (as an ordered list). - Omit `wireWatchPaths` (or send `[]`) to diff the whole payload. The diff then reports changes keyed by path, and AI meaningful-change (if on) only considers the selected fields. On the dashboard, click **Inspect output** in the monitor form to run the action once, preview its JSON, and tick the fields to watch. > Non-JSON action output (rare) is always diffed whole; `wireWatchPaths` is ignored for it. --- ### The change payload A wire monitor's `diffJson` is a `fields` diff whose `before` / `after` are the full JSON objects the action returned, and `changedFields` are the top-level keys that changed. From [`GET /v1/monitors/{id}/changes`](/docs/api-reference/monitoring/snapshots-and-changes): ```json { "type": "fields", "changedFields": ["results"], "before": { "results": [{ "title": "USB-C Cable 3ft", "price": 8.99 }] }, "after": { "results": [{ "title": "USB-C Cable 3ft", "price": 6.49 }] }, "aiMeaningful": [ { "description": "Top result price dropped", "before": "$8.99", "after": "$6.49" } ] } ``` The snapshot's `extractedData` holds the full returned object for each check, so you can pull the raw result history from [snapshots](/docs/api-reference/monitoring/snapshots-and-changes). `aiMeaningful` and `summary` are present only when `aiMode` is on. --- ### Limits & notes - **Minimum interval:** 15 minutes, same as other monitors. - Wire monitors count against your active-monitor cap (Free `5` / Pro `20` / Scale `100`). - If the action requires auth and its identity's credential expires, checks fail until you reconnect it in [Wire](/docs/api-reference/wire) — the monitor keeps trying on schedule. - Manage everything else — pause/resume, run-now, alerts — with the same [endpoints](/docs/api-reference/monitoring) as page monitors. --- ### Example <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/monitors \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://www.amazon.com", "scope": "wire", "wireCatalogSlug": "amazon", "wireActionId": "amazon.search_products", "wireParams": { "query": "usb-c cable" }, "intervalMinutes": 720, "aiMode": true, "alertWebhookUrl": "https://your-app.com/hooks/anakin" }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( "https://api.anakin.io/v1/monitors", headers={"X-API-Key": "your_api_key"}, json={ "url": "https://www.amazon.com", "scope": "wire", "wireCatalogSlug": "amazon", "wireActionId": "amazon.search_products", "wireParams": {"query": "usb-c cable"}, "intervalMinutes": 720, "aiMode": True, "alertWebhookUrl": "https://your-app.com/hooks/anakin", }, ) monitor = res.json() print("Created wire monitor", monitor["id"]) ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch("https://api.anakin.io/v1/monitors", { method: "POST", headers: { "X-API-Key": "your_api_key", "Content-Type": "application/json" }, body: JSON.stringify({ url: "https://www.amazon.com", scope: "wire", wireCatalogSlug: "amazon", wireActionId: "amazon.search_products", wireParams: { query: "usb-c cable" }, intervalMinutes: 720, aiMode: true, alertWebhookUrl: "https://your-app.com/hooks/anakin", }), }); const monitor = await res.json(); console.log("Created wire monitor", monitor.id); ``` </Tab> </Tabs> --- # POST Search (/docs/api-reference/search/search) <EndpointBanner method="POST" path="https://api.anakin.io/v1/search" /> Perform an AI-powered web search. Returns an AI-generated summary alongside structured search results with citations, snippets, and relevance scores. Results are returned synchronously. --- ### Request Body ```json { "prompt": "latest AI developments 2024", "limit": 5 } ``` | Parameter | Type | Description | |-----------|------|-------------| | `prompt` **required** | string | Search query or question | | `limit` | number | Maximum number of results to return. Default `5`, max `20`. | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "id": "63385e99-3ef5-4667-84a7-e7b398ec8e06", "results": [ { "url": "https://example.com/article", "title": "AI Developments 2024", "snippet": "Recent advancements in AI...", "date": "2024-01-15", "last_updated": "2024-01-20" } ] } ``` ### Response Fields | Field | Type | Description | |-------|------|-------------| | `id` | string | Unique identifier for the search request | | `results` | array | Array of search result objects | | `results[].url` | string | Source URL | | `results[].title` | string | Page title | | `results[].snippet` | string | Relevant text excerpt | | `results[].date` | string | Publication date (when available) | | `results[].last_updated` | string | Last updated date (when available) | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/search \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "prompt": "latest AI developments 2024", "limit": 5 }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/search', headers={'X-API-Key': 'your_api_key'}, json={ 'prompt': 'latest AI developments 2024', 'limit': 5 } ) data = response.json() print(f"Search ID: {data['id']}") for result in data['results']: print(f"\nTitle: {result['title']}") print(f"URL: {result['url']}") print(f"Snippet: {result['snippet']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/search', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ prompt: 'latest AI developments 2024', limit: 5 }) }); const data = await response.json(); console.log(`Search ID: ${data.id}`); data.results.forEach(r => console.log(r.title, r.url)); ``` </Tab> </Tabs> --- # POST Scrape URLs (/docs/api-reference/url-scraper/batch-url-scraping) <EndpointBanner method="POST" path="https://api.anakin.io/v1/url-scraper/batch" /> Submit up to 10 URLs for scraping in a single request. All URLs are processed in parallel. Use the returned `jobId` to [poll for results](/docs/api-reference/url-scraper/get-job-status). --- ### Request Body ```json { "urls": [ "https://example.com/page1", "https://example.com/page2", "https://example.com/page3" ], "country": "us", "useBrowser": false, "generateJson": false, "webhook_url": "https://your-app.com/hooks/anakin" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `urls` **required** | string[] | Array of URLs to scrape (1–10). | | `country` | string | Country code for proxy routing. Default `"us"`. See [Supported Countries](/docs/api-reference/supported-countries) (207 locations). | | `useBrowser` | boolean | Use headless Chrome with Playwright. Default `false`. | | `generateJson` | boolean | AI-extract structured JSON from the content. Default `false`. | | `sessionId` | string | Browser session ID for authenticated batch scraping. See [Browser Sessions](/docs/api-reference/browser-sessions). | | `webhook_url` | string | URL to `POST` a signed `batch.completed` / `batch.failed` event to when the whole batch settles (URLs inside a batch don't fire individual `job.*` events). See [Webhooks](/docs/api-reference/webhooks). | --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "jobId": "batch_abc123", "status": "pending" } ``` You receive a parent job ID that tracks overall batch progress. Use [GET /v1/url-scraper/\{id\}](/docs/api-reference/url-scraper/get-job-status) to poll for results — the response will include a `results` array with individual URL outcomes. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/url-scraper/batch \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "urls": [ "https://example.com/page1", "https://example.com/page2", "https://example.com/page3" ], "country": "us", "useBrowser": false, "generateJson": true }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/url-scraper/batch', headers={'X-API-Key': 'your_api_key'}, json={ 'urls': [ 'https://example.com/page1', 'https://example.com/page2', 'https://example.com/page3' ], 'country': 'us', 'useBrowser': False, 'generateJson': True } ) data = response.json() print(f"Batch job submitted: {data['jobId']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/url-scraper/batch', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ urls: [ 'https://example.com/page1', 'https://example.com/page2', 'https://example.com/page3' ], country: 'us', useBrowser: false, generateJson: true }) }); const data = await response.json(); console.log(data.jobId); ``` </Tab> </Tabs> --- # GET Get Results (/docs/api-reference/url-scraper/get-job-status) <EndpointBanner method="GET" path="https://api.anakin.io/v1/url-scraper/{id}" /> Retrieve the status and results of a scrape job. Use this to poll for completion after submitting a [single URL](/docs/api-reference/url-scraper/submit-scrape-job) or [batch](/docs/api-reference/url-scraper/batch-url-scraping) scrape request. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The job ID returned from the submit endpoint | --- ### Response — Single URL Job <StatusBadge code={200} text="OK" /> ```json { "id": "job_abc123xyz", "status": "completed", "url": "https://example.com", "jobType": "url_scraper", "country": "us", "html": "<html>...</html>", "cleanedHtml": "<div>...</div>", "markdown": "# Page content...", "generatedJson": { "data": {} }, "links": [{ "href": "https://example.com/about", "text": "About" }], "images": [{ "src": "https://example.com/logo.png", "alt": "Logo" }], "summary": "A concise AI-generated summary of the page.", "screenshotUrl": "https://.../screenshot.png", "fullPageScreenshotUrl": "https://.../fullpage.png", "cached": false, "error": null, "createdAt": "2024-01-01T12:00:00Z", "completedAt": "2024-01-01T12:00:05Z", "durationMs": 5000 } ``` Which content fields appear depends on the `formats` you requested (see [Submit a Scrape Job](/docs/api-reference/url-scraper/submit-scrape-job)) — every content field is omitted when it wasn't requested or produced. `screenshotUrl` / `fullPageScreenshotUrl` are API-key-authed download endpoints — you must `GET` them with your `X-API-Key` header. See [Download Screenshot](/docs/api-reference/url-scraper/get-screenshot). ### Response — Batch Job <StatusBadge code={200} text="OK" /> ```json { "id": "batch_abc123", "status": "completed", "jobType": "batch_url_scraper", "country": "us", "urls": ["https://example.com/page1", "https://example.com/page2"], "results": [ { "index": 0, "url": "https://example.com/page1", "status": "completed", "html": "<html>...</html>", "cleanedHtml": "<div>...</div>", "markdown": "# Content...", "generatedJson": { "data": {} }, "cached": false, "durationMs": 3000 }, { "index": 1, "url": "https://example.com/page2", "status": "failed", "error": "Connection timeout", "durationMs": 5000 } ], "createdAt": "2024-01-01T12:00:00Z", "completedAt": "2024-01-01T12:00:10Z", "durationMs": 10000 } ``` --- ### Response Fields | Field | Type | Description | |-------|------|-------------| | `status` | string | `pending`, `processing`, `completed`, or `failed` | | `html` | string | Raw HTML content. Only present when completed. | | `cleanedHtml` | string | Cleaned HTML with non-essential elements removed. | | `markdown` | string | Markdown version of the content. | | `generatedJson` | object | AI-extracted structured JSON. Only when `generateJson: true` (or an `outputSchema`) was set. | | `links` | object[] | Extracted hyperlinks, each `{ href, text }`. Only when the `links` format was requested. | | `images` | object[] | Extracted images, each `{ src, alt }`. Only when the `images` format was requested. | | `summary` | string | AI-generated summary. Only when the `summary` format was requested. | | `screenshotUrl` | string | API-key-authed URL to download the viewport screenshot (`GET` with `X-API-Key`). Only when `screenshot` was requested. See [Download Screenshot](/docs/api-reference/url-scraper/get-screenshot). | | `fullPageScreenshotUrl` | string | API-key-authed URL to download the full-page screenshot (`GET` with `X-API-Key`). Only when `screenshotFullPage` was requested. See [Download Screenshot](/docs/api-reference/url-scraper/get-screenshot). | | `cached` | boolean | `true` if served from cache. | | `error` | string | Error message. Only present when failed. | | `durationMs` | number | Processing time in milliseconds. | | `results` | array | Batch jobs only — array of per-URL results. | ### Job Statuses | Status | Description | |--------|-------------| | `pending` | Job is queued | | `processing` | Job is being executed | | `completed` | Results are ready | | `failed` | Job encountered an error | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X GET https://api.anakin.io/v1/url-scraper/job_abc123xyz \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests job_id = "job_abc123xyz" result = requests.get( f'https://api.anakin.io/v1/url-scraper/{job_id}', headers={'X-API-Key': 'your_api_key'} ) data = result.json() if data['status'] == 'completed': print(data['markdown']) ``` </Tab> <Tab value="JavaScript"> ```javascript const jobId = 'job_abc123xyz'; const res = await fetch(`https://api.anakin.io/v1/url-scraper/${jobId}`, { headers: { 'X-API-Key': 'your_api_key' } }); const data = await res.json(); if (data.status === 'completed') { console.log(data.markdown); } ``` </Tab> </Tabs> For polling patterns, see the [Polling Jobs](/docs/api-reference/polling-jobs) reference. --- # GET Download Screenshot (/docs/api-reference/url-scraper/get-screenshot) <EndpointBanner method="GET" path="https://api.anakin.io/v1/url-scraper/{id}/screenshot" /> Download the PNG screenshot captured for a completed scrape job. This endpoint streams the raw image bytes — it is **not** a public S3 link. Every request must be authenticated with your API key, and you can only fetch screenshots for jobs your key owns. You get here from the [job result](/docs/api-reference/url-scraper/get-job-status): when you request the `screenshot` and/or `screenshotFullPage` formats, the completed job returns `screenshotUrl` and `fullPageScreenshotUrl`, both of which point at this endpoint. --- ### Capture → Download Flow 1. Submit a scrape with a screenshot format — this auto-enables the headless browser: ```json { "url": "https://example.com", "formats": ["screenshot", "screenshotFullPage"] } ``` 2. [Poll the job](/docs/api-reference/url-scraper/get-job-status) until `status` is `completed`. The result includes `screenshotUrl` (viewport) and `fullPageScreenshotUrl` (full page). 3. `GET` that URL **with your `X-API-Key` header** to download the PNG. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The job ID whose screenshot you want. For batch jobs, use the per-result child job ID from the batch response. | ### Query Parameters | Parameter | Type | Description | |-----------|------|-------------| | `type` *optional* | string | `viewport` (default) or `fullpage`. Must match a format that was requested when the job was submitted. Any other value returns `400`. | --- ### Authentication **This endpoint is API-key authenticated and ownership-scoped — it is not open.** You **must** send your API key on every request (`X-API-Key: your_api_key`). Requests with no key return `401`; requests for a job that your key does not own return `403`. The `screenshotUrl` / `fullPageScreenshotUrl` fields returned by the job-status endpoint are download endpoints on `api.anakin.io`, not pre-signed S3 URLs — pasting them into a browser without your API key header will fail with `401`. --- ### Response <StatusBadge code={200} text="OK" /> Raw PNG image bytes. | Header | Value | |--------|-------| | `Content-Type` | `image/png` | | `Cache-Control` | `private, max-age=300` | | Status | Meaning | |--------|---------| | `200` | Screenshot streamed as `image/png`. | | `400` | Missing `id`, or `type` is not `viewport`/`fullpage`. | | `401` | Missing or invalid API key. | | `403` | The job exists but is not owned by your API key. | | `404` | Job or requested screenshot does not exist (e.g. the format wasn't requested, or the job hasn't completed). | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # Viewport screenshot (default) curl -X GET "https://api.anakin.io/v1/url-scraper/job_abc123xyz/screenshot" \ -H "X-API-Key: your_api_key" \ --output screenshot.png # Full-page screenshot curl -X GET "https://api.anakin.io/v1/url-scraper/job_abc123xyz/screenshot?type=fullpage" \ -H "X-API-Key: your_api_key" \ --output fullpage.png ``` </Tab> <Tab value="Python"> ```python import requests job_id = "job_abc123xyz" resp = requests.get( f'https://api.anakin.io/v1/url-scraper/{job_id}/screenshot', headers={'X-API-Key': 'your_api_key'}, params={'type': 'viewport'}, # or 'fullpage' ) resp.raise_for_status() with open('screenshot.png', 'wb') as f: f.write(resp.content) ``` </Tab> <Tab value="JavaScript"> ```javascript const jobId = 'job_abc123xyz'; const res = await fetch( `https://api.anakin.io/v1/url-scraper/${jobId}/screenshot?type=viewport`, { headers: { 'X-API-Key': 'your_api_key' } } ); if (!res.ok) throw new Error(`Screenshot fetch failed: ${res.status}`); const buffer = Buffer.from(await res.arrayBuffer()); require('fs').writeFileSync('screenshot.png', buffer); ``` </Tab> </Tabs> The URLs returned in `screenshotUrl` / `fullPageScreenshotUrl` already encode the correct `id` and `type`, so you can fetch them directly — just remember to attach your `X-API-Key` header. --- # POST Scrape URL (Inline) (/docs/api-reference/url-scraper/scrape) <EndpointBanner method="POST" path="https://api.anakin.io/v1/url-scraper/scrape" /> Submit a single URL and **block until the scrape finishes**, returning the full result in the same response — no polling required. Convenient for quick, interactive use where you'd rather wait than poll. For batch jobs or long-running scrapes, prefer the asynchronous [POST /v1/url-scraper](/docs/api-reference/url-scraper/submit-scrape-job) + [poll](/docs/api-reference/url-scraper/get-job-status) flow instead. **Note:** the request holds the connection open until the scrape completes (up to ~90s). If the job doesn't finish in time, you get a `202` whose body has the same shape as a poll response (an `id` + a non-terminal `status`) to fall back to polling — see [Timeout](#timeout) below. Set generous client/proxy read timeouts. Because each request holds a connection open, this endpoint is **rate limited more tightly than the async endpoint** (20 requests/min per user). For high throughput, use the async [submit](/docs/api-reference/url-scraper/submit-scrape-job) + [poll](/docs/api-reference/url-scraper/get-job-status) flow instead. Latency tracks the normal scrape pipeline, so it can rise when the platform is under heavy load. --- ### Request Body Identical to the asynchronous endpoint. ```json { "url": "https://example.com", "country": "us", "useBrowser": false, "generateJson": false } ``` | Parameter | Type | Description | |-----------|------|-------------| | `url` **required** | string | The URL to scrape. Must be valid HTTP/HTTPS. | | `country` | string | Country code for proxy routing. Default `"us"`. See [Supported Countries](/docs/api-reference/supported-countries) (207 locations). | | `useBrowser` | boolean | Use headless Chrome with Playwright. Default `false`. Best for JS-heavy sites. | | `generateJson` | boolean | AI-extract structured JSON from the content. Default `false`. | | `sessionId` | string | Browser session ID for scraping authenticated pages. See [Browser Sessions](/docs/api-reference/browser-sessions). | --- ### Response <StatusBadge code={200} text="OK" /> Returns the completed job with its content inline (same shape as [GET /v1/url-scraper/\{id\}](/docs/api-reference/url-scraper/get-job-status)): ```json { "id": "job_abc123xyz", "status": "completed", "url": "https://example.com", "jobType": "url_scraper", "country": "us", "html": "<html>...</html>", "cleanedHtml": "<div>...</div>", "markdown": "# Page content...", "generatedJson": { "data": {} }, "cached": false, "error": null, "createdAt": "2024-01-01T12:00:00Z", "completedAt": "2024-01-01T12:00:05Z", "durationMs": 5000 } ``` A scrape that fails also returns `200` with `status: "failed"` and a populated `error` field — check `status`, not just the HTTP code. #### Timeout <StatusBadge code={202} text="Accepted" /> If the scrape doesn't finish within the wait budget, the response has the **same shape as a poll response** — an `id` and a non-terminal `status` — so you can [poll](/docs/api-reference/url-scraper/get-job-status) `GET /v1/url-scraper/{id}` for the result. (The content fields are simply absent until the job completes.) ```json { "id": "job_abc123xyz", "status": "processing", "url": "https://example.com", "jobType": "url_scraper", "country": "us", "createdAt": "2024-01-01T12:00:00Z" } ``` The `id` field is identical across the `200` result and this `202` — your client reads `id` either way; only `status` tells you whether to use the inline content or keep polling. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/url-scraper/scrape \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ --max-time 120 \ -d '{ "url": "https://example.com", "country": "us", "useBrowser": false, "generateJson": false }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/url-scraper/scrape', headers={'X-API-Key': 'your_api_key'}, json={ 'url': 'https://example.com', 'country': 'us', 'useBrowser': False, 'generateJson': True, }, timeout=120, ) data = response.json() if data.get('status') == 'completed': print(data['markdown']) else: # 202 timeout fallback — poll GET /v1/url-scraper/{id} print(f"Still running: {data['id']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/url-scraper/scrape', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ url: 'https://example.com', country: 'us', useBrowser: false, generateJson: true }) }); const data = await response.json(); if (data.status === 'completed') { console.log(data.markdown); } else { // 202 timeout fallback — poll GET /v1/url-scraper/{id} console.log('Still running:', data.id); } ``` </Tab> </Tabs> --- # POST Scrape URL (/docs/api-reference/url-scraper/submit-scrape-job) <EndpointBanner method="POST" path="https://api.anakin.io/v1/url-scraper" /> Submit a single URL for scraping. The job is processed asynchronously — use the returned `jobId` to [poll for results](/docs/api-reference/url-scraper/get-job-status). --- ### Request Body ```json { "url": "https://example.com", "formats": ["markdown", "html", "links"], "country": "us", "useBrowser": false, "generateJson": false, "webhook_url": "https://your-app.com/hooks/anakin" } ``` Only `url` is required — every other field is optional. | Parameter | Type | Description | |-----------|------|-------------| | `url` **required** | string | The URL to scrape. Must be valid HTTP/HTTPS. | | `formats` *optional* | string[] | Which outputs to produce: `markdown`, `html`, `cleanedHtml`, `links`, `images`, `summary`, `screenshot`, `screenshotFullPage`, `json`. Defaults to markdown + HTML + cleaned HTML. | | `country` *optional* | string | Country code for proxy routing. Default `"us"`. See [Supported Countries](/docs/api-reference/supported-countries) (207 locations). | | `useBrowser` *optional* | boolean | Use headless Chrome with Playwright. Default `false`. Best for JS-heavy sites. | | `generateJson` *optional* | boolean | AI-extract structured JSON from the content. Default `false`. | | `outputSchema` *optional* | object | JSON Schema describing the fields to extract. Implies `generateJson: true`. | | `sessionId` *optional* | string | Browser session ID for scraping authenticated pages. See [Browser Sessions](/docs/api-reference/browser-sessions). | | `webhook_url` *optional* | string | URL to `POST` a signed `job.completed` / `job.failed` event to when the job settles — skip polling. See [Webhooks](/docs/api-reference/webhooks). | | `actions` *optional* | object[] | Ordered browser actions executed before the page is captured. Implies `useBrowser: true` and costs **+1 credit**. See [Browser Actions](#browser-actions) below. | --- ### Browser Actions Pass an `actions` array to interact with the page before it is captured — click "load more" buttons, dismiss banners, scroll lazy-loaded content into view, or fill in a search box. Actions run in order in a real browser. ```json { "url": "https://example.com/products", "actions": [ { "type": "wait_for", "selector": ".product-grid" }, { "type": "click", "selector": "button.load-more" }, { "type": "wait", "milliseconds": 1500 }, { "type": "scroll", "direction": "down" } ] } ``` | Type | Fields | Description | |------|--------|-------------| | `wait` | `milliseconds` **required** | Pause for a fixed duration (1–15000ms). | | `wait_for` | `selector` **required**, `milliseconds` *optional* | Wait until a CSS selector appears (default max wait 10000ms). | | `click` | `selector` **required** | Click the first element matching a CSS selector. Missing elements are skipped. | | `scroll` | `direction` *optional* (`up`/`down`, default `down`), `selector` *optional* | Scroll the page (or a specific element) by ~one viewport. | | `write` | `text` **required** | Type text into the currently focused element (max 1000 chars). | | `press` | `key` **required** | Press a keyboard key, e.g. `Enter`, `Tab`. | **Limits:** max 15 actions per request; each wait capped at 15 seconds; total wait time across all actions capped at 30 seconds; selectors capped at 500 characters. Requests exceeding a limit are rejected with `400`. --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "jobId": "job_abc123xyz", "status": "pending" } ``` The job is processed asynchronously. Use the `jobId` with [GET /v1/url-scraper/\{id\}](/docs/api-reference/url-scraper/get-job-status) to check status and retrieve results. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/url-scraper \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://example.com", "country": "us", "useBrowser": false, "generateJson": false }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/url-scraper', headers={'X-API-Key': 'your_api_key'}, json={ 'url': 'https://example.com', 'country': 'us', 'useBrowser': False, 'generateJson': True } ) data = response.json() print(f"Job submitted: {data['jobId']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/url-scraper', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ url: 'https://example.com', country: 'us', useBrowser: false, generateJson: true }) }); const data = await response.json(); console.log(data.jobId); ``` </Tab> </Tabs> --- # Event Types & Payloads (/docs/api-reference/webhooks/events) Every webhook delivery carries exactly one event. This page documents each event type and the payload it carries. For headers, signature verification, and retry semantics, see the [Webhooks overview](/docs/api-reference/webhooks). --- ## The envelope All events **except `monitor.change`** share one envelope: ```json { "id": "evt_a1b2c3…", "type": "job.completed", "createdAt": "2026-07-13T10:00:00Z", "data": { } } ``` | Field | Type | Description | |-------|------|-------------| | `id` | string | Unique event ID (`evt_…`). | | `type` | string | The event type — same as the `X-Anakin-Event` header. | | `createdAt` | string | When the event was created (ISO 8601). | | `data` | object | Event-specific fields, documented per event below. | `monitor.change` keeps its **original flat payload** (no envelope) for backward compatibility — see [below](#monitorchange). --- ## Event taxonomy | Event | Fires when | |-------|------------| | `job.completed` | An async job finishes successfully — `data.jobType` tells you which product | | `job.failed` | An async job fails | | `batch.completed` | A batch URL-scrape settles successfully (fires once per batch) | | `batch.failed` | A batch URL-scrape fails as a whole | | `wire.job.completed` | A Wire task finishes successfully | | `wire.job.failed` | A Wire task fails | | `monitor.change` | A Website Monitor detects a change | | `ai.search.completed` | An AI Visibility search settles with at least one source answered | | `ai.search.failed` | An AI Visibility search settles with every source failed | | `webhook.test` | You trigger a test delivery from the dashboard or API | Registered endpoints receive the events matched by their filter (empty filter = all); a per-request `webhook_url` receives only the events for that job. Fetch this list programmatically with [`GET /v1/webhooks/events`](/docs/api-reference/webhooks/managing-endpoints#event-list). --- ## `job.completed` / `job.failed` Fired when an asynchronous job settles. `data.jobType` distinguishes the product: `url_scraper`, `web_scraper`, `map`, `crawl`, `agentic_search`, or `search`. ### `data` fields | Field | Type | Description | |-------|------|-------------| | `jobId` | string | The job ID returned at submission. | | `jobType` | string | `url_scraper`, `web_scraper`, `map`, `crawl`, `agentic_search`, or `search`. | | `status` | string | `"completed"` or `"failed"`. | | `url` | string | The submitted URL. | | `country` | string | Proxy country — present when set on the job. | | `cached` | boolean | Present when the result was served from cache. | | `creditsUsed` | number | Credits charged — present when known. | | `durationMs` | number | Processing time in milliseconds — present when known. | | `createdAt` | string | When the job was created. | | `completedAt` | string | When the job settled. | | `result_url` | string | **Always present.** The canonical authenticated `GET` endpoint for the full result — fetch it with your API key. | | `result` | object | The full canonical result document, inlined **only when the whole envelope is ≤ 256 KB**. | | `resultOmitted` | string | `"size"` or `"unavailable"` — present when `result` was dropped. | | `error` | string | Failure reason — `job.failed` only. | Treat the inline `result` as an optimization: when the result document is small it's included so you never make a second request; when it would push the envelope past **256 KB** it's dropped and `resultOmitted: "size"` is set instead. `result_url` is always there and always canonical — when in doubt, fetch it: | Job | `result_url` | |-----|--------------| | `url_scraper`, `batch_url_scraper` | `https://api.anakin.io/v1/url-scraper/{jobId}` | | `web_scraper` | `https://api.anakin.io/v1/web-scraper/{jobId}` | | `map` | `https://api.anakin.io/v1/map/{jobId}` | | `crawl` | `https://api.anakin.io/v1/crawl/{jobId}` | | `agentic_search` | `https://api.anakin.io/v1/agentic-search/{jobId}` | | `search` | `https://api.anakin.io/v1/request/{jobId}` | | Wire tasks (`wire.job.*`) | `https://api.anakin.io/v1/wire/jobs/{jobId}` | Whatever the job type, follow the `result_url` from the event rather than constructing URLs yourself. > **Fetching the complete result** — `result_url` is a normal authenticated endpoint: `GET` it with your API key, exactly like polling. This is the one pattern to build into every receiver, because inlining is best-effort and size-dependent: > > ```bash > curl "https://api.anakin.io/v1/url-scraper/0b0e5e7e-…" \ > -H "X-API-Key: your_api_key" > ``` > > A receiver that handles `result_url` works for **every** event and **every** result size; a receiver that only reads inline `result` silently breaks the day a result crosses 256 KB. `job.failed` events **never** inline results — read `error`, or fetch `result_url` for the full failure details. ### Example — `job.completed` with the result inlined ```json { "id": "evt_5f2a9c…", "type": "job.completed", "createdAt": "2026-07-13T10:00:05Z", "data": { "jobId": "0b0e5e7e-4c2a-4b8e-9c1d-2f3a4b5c6d7e", "jobType": "url_scraper", "status": "completed", "url": "https://example.com", "country": "us", "cached": false, "creditsUsed": 1, "durationMs": 5000, "createdAt": "2026-07-13T10:00:00Z", "completedAt": "2026-07-13T10:00:05Z", "result_url": "https://api.anakin.io/v1/url-scraper/0b0e5e7e-4c2a-4b8e-9c1d-2f3a4b5c6d7e", "result": { "id": "0b0e5e7e-4c2a-4b8e-9c1d-2f3a4b5c6d7e", "status": "completed", "url": "https://example.com", "jobType": "url_scraper", "country": "us", "html": "<html>...</html>", "cleanedHtml": "<div>...</div>", "markdown": "# Page content...", "cached": false, "error": null, "createdAt": "2026-07-13T10:00:00Z", "completedAt": "2026-07-13T10:00:05Z", "durationMs": 5000 } } } ``` `result` is the same document the poll endpoint returns — here, the shape of [GET /v1/url-scraper/\{id\}](/docs/api-reference/url-scraper/get-job-status). ### Example — `job.completed` with the result omitted A multi-page crawl result easily exceeds the envelope limit, so the event arrives reference-only: ```json { "id": "evt_8d41e0…", "type": "job.completed", "createdAt": "2026-07-13T10:04:41Z", "data": { "jobId": "9f8e7d6c-5b4a-4c3d-8e2f-1a0b9c8d7e6f", "jobType": "crawl", "status": "completed", "url": "https://example.com", "country": "us", "creditsUsed": 20, "durationMs": 281000, "createdAt": "2026-07-13T10:00:00Z", "completedAt": "2026-07-13T10:04:41Z", "result_url": "https://api.anakin.io/v1/crawl/9f8e7d6c-5b4a-4c3d-8e2f-1a0b9c8d7e6f", "resultOmitted": "size" } } ``` ### Example — `job.failed` ```json { "id": "evt_c77b12…", "type": "job.failed", "createdAt": "2026-07-13T10:02:11Z", "data": { "jobId": "0b0e5e7e-4c2a-4b8e-9c1d-2f3a4b5c6d7e", "jobType": "url_scraper", "status": "failed", "url": "https://example.com", "country": "us", "createdAt": "2026-07-13T10:00:00Z", "completedAt": "2026-07-13T10:02:11Z", "result_url": "https://api.anakin.io/v1/url-scraper/0b0e5e7e-4c2a-4b8e-9c1d-2f3a4b5c6d7e", "error": "Connection timeout" } } ``` > **Sync endpoints emit these too** — `POST /v1/search` and `POST /v1/url-scraper/scrape` return results inline in the HTTP response, but registered endpoints subscribed to `job.completed` / `job.failed` still receive the events, so a single receiver observes every job. --- ## `batch.completed` / `batch.failed` Fired **once per batch** when the whole [batch scrape](/docs/api-reference/url-scraper/batch-url-scraping) settles — the individual URLs inside a batch do **not** fire `job.*` events. Per-URL failures in an otherwise settled batch show up in the `failed` count on `batch.completed`; `batch.failed` fires when the batch as a whole fails. ### `data` fields | Field | Type | Description | |-------|------|-------------| | `jobId` | string | The batch job ID returned at submission. | | `jobType` | string | Always `batch_url_scraper`. | | `status` | string | `"completed"` or `"failed"`. | | `urlsTotal` | number | URLs submitted in the batch. | | `completed` | number | URLs that scraped successfully. | | `failed` | number | URLs that failed. | | `creditsUsed` | number | Credits charged for the batch. | | `result_url` | string | `GET` endpoint for the full batch result with per-URL outcomes. | Batch events **never** inline results — fetch `result_url` for the per-URL outcomes. ### Example — `batch.completed` ```json { "id": "evt_2b90fe…", "type": "batch.completed", "createdAt": "2026-07-13T10:00:12Z", "data": { "jobId": "a28d0097-6828-4a2f-a242-dd5658656771", "jobType": "batch_url_scraper", "status": "completed", "urlsTotal": 3, "completed": 2, "failed": 1, "creditsUsed": 2, "result_url": "https://api.anakin.io/v1/url-scraper/a28d0097-6828-4a2f-a242-dd5658656771" } } ``` --- ## `wire.job.completed` / `wire.job.failed` Fired when a [Wire task](/docs/api-reference/wire/execute-task) settles — pass `webhook_url` at submission, or subscribe a registered endpoint. Delivered within **~10 seconds** of completion. ### `data` fields | Field | Type | Description | |-------|------|-------------| | `jobId` | string | The Wire job ID. | | `actionId` | string | The action that ran (e.g. `li_profile_scrape`). | | `actionName` | string | Human-readable action name. | | `catalogSlug` | string | The catalog the action belongs to. | | `status` | string | `"completed"` or `"failed"`. | | `creditsUsed` | number | Credits charged. | | `durationMs` | number | Execution time in milliseconds. | | `error` | string | Failure reason — `wire.job.failed` only. | | `result_url` | string | **Always present** — `https://api.anakin.io/v1/wire/jobs/{jobId}`. | | `result` | object | The full job result (same shape as [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job)), inlined when small — the same **256 KB** envelope rule applies. | ### Example — `wire.job.completed` ```json { "id": "evt_77aa0d…", "type": "wire.job.completed", "createdAt": "2026-07-13T10:00:09Z", "data": { "jobId": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "actionId": "li_profile_scrape", "actionName": "LinkedIn Profile Scrape", "catalogSlug": "linkedin", "status": "completed", "creditsUsed": 2, "durationMs": 8420, "result_url": "https://api.anakin.io/v1/wire/jobs/7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "result": { "status": "completed", "data": { "name": "Jane Doe", "headline": "Software Engineer at Acme Corp", "location": "San Francisco, CA", "connections": 500 }, "credits_used": 2, "execution_ms": 8420 } } } ``` On `wire.job.failed`, `error` is present and no result is inlined. --- ## `monitor.change` Website-monitoring alerts keep their **original flat payload** — `type` at the top level, no envelope — so existing receivers keep working unchanged: ```json { "type": "monitor.change", "monitorId": "62e555fb-…", "url": "https://example.com/product/123", "watchMode": "specific_data", "changeId": "c1e8fbac-…", "changedAt": "2026-07-13T10:00:00Z", "changedFields": ["price"], "summary": "The price dropped from $19.99 to $14.99.", "diff": { "before": { "price": 19.99 }, "after": { "price": 14.99 } } } ``` Registered endpoints subscribed to `monitor.change` receive these deliveries too, in this same flat shape. For the full payload schema (`watchMode` variants, `diff`, AI summaries), fetching before/after snapshots, and monitor-specific setup, see [Website Monitor Webhooks](/docs/integrations/website-monitoring/webhooks). --- ## `ai.search.completed` / `ai.search.failed` Fired when an [AI Visibility search](/docs/api-reference/ai-visibility) settles — every selected source has either answered or failed. `ai.search.completed` means at least one source answered; `ai.search.failed` means all of them failed. > **Note — full answers are never inlined.** Unlike `job.*` and `wire.job.*` events, AI Visibility deliveries don't carry a `result` object at all: each source's answer can run to hundreds of KB, so the event inlines only a bounded per-source `summary` (≤ 500 chars). To get the complete answers, `GET` the `result_url` with your API key — it is always present. ### `data` fields | Field | Type | Description | |-------|------|-------------| | `searchId` | string | The search ID returned at submission. | | `query` | string | The prompt that was fanned out. | | `status` | string | `"completed"` or `"failed"`. | | `country` | string | Search geography the run executed from. | | `sources` | object[] | One entry per source — see below. | | `creditsUsed` | number | Total credits charged across all sources. | | `synthesis` | string | Optional — cross-source synthesis, only present when the AI synthesis pass is enabled for the run (off by default). | | `completedAt` | string | When the search settled. | | `result_url` | string | **Always present** — `https://api.anakin.io/v1/ai-visibility/search/{searchId}`. Fetch this for the full answers. | Each `sources[]` entry: | Field | Type | Description | |-------|------|-------------| | `source` | string | Source slug (`chatgpt`, `gemini`, `google_ai_overview`, …). | | `status` | string | `"completed"` or `"failed"`. | | `latencyMs` | number | Measured engine latency. | | `creditsUsed` | number | Credits for this source (0 when failed). | | `summary` | string | Bounded answer preview (≤ 500 chars), completed sources only — **not** the full answer. | | `error` | string | Failure reason — failed sources only. | ### Example — `ai.search.completed` ```json { "id": "evt_9f2c1d…", "type": "ai.search.completed", "createdAt": "2026-07-15T12:03:41Z", "data": { "searchId": "3d9db7ff-1c2e-4b8a-9f0d-5e6a7b8c9d0e", "query": "Best coding agents in 2026", "status": "completed", "country": "us", "creditsUsed": 3, "sources": [ { "source": "chatgpt", "status": "completed", "latencyMs": 9100, "creditsUsed": 1, "summary": "The leading coding agents in 2026 are…" }, { "source": "gemini", "status": "completed", "latencyMs": 27100, "creditsUsed": 2, "summary": "Several agents dominate the space…" }, { "source": "google_ai_overview", "status": "failed", "latencyMs": 150000, "creditsUsed": 0, "error": "source timed out" } ], "result_url": "https://api.anakin.io/v1/ai-visibility/search/3d9db7ff-1c2e-4b8a-9f0d-5e6a7b8c9d0e" } } ``` Then fetch the complete answers (full text per source): ```bash curl "https://api.anakin.io/v1/ai-visibility/search/3d9db7ff-1c2e-4b8a-9f0d-5e6a7b8c9d0e" \ -H "X-API-Key: your_api_key" ``` The response is documented at [GET /v1/ai-visibility/search/\{id\}](/docs/api-reference/ai-visibility/get-search). --- ## `webhook.test` Sent when you press **Test** in the dashboard (**Account → Webhooks**) or call [`POST /v1/webhooks/{id}/test`](/docs/api-reference/webhooks/managing-endpoints#manage-endpoints). ```json { "id": "evt_f0e1d2…", "type": "webhook.test", "createdAt": "2026-07-13T10:00:00Z", "data": { "endpointId": "9b1c22e4-…", "message": "Test delivery from anakin.io — your endpoint is reachable and this body is signed with your endpoint secret." } } ``` --- # Managing Endpoints (/docs/api-reference/webhooks/managing-endpoints) **Registered endpoints** are account-level webhook destinations: each one has a URL, an optional event filter, and its own `whsec_…` signing secret. Everything on this page can also be done from the dashboard under **Account → Webhooks**. All endpoints require your API key in the `X-API-Key` header. - Up to **10** registered endpoints per account. - An empty (or omitted) `events` filter subscribes to **all events** — see the [event taxonomy](/docs/api-reference/webhooks/events). - URLs: **HTTPS strongly recommended** (`http` allowed), max **2048 characters**, private/internal addresses rejected. --- ## Manage endpoints <EndpointBanner method="POST" path="https://api.anakin.io/v1/webhooks" /> Create a registered endpoint. ### Request Body ```json { "url": "https://your-app.com/hooks/anakin", "description": "Production receiver", "events": ["job.completed", "job.failed"] } ``` | Parameter | Type | Description | |-----------|------|-------------| | `url` **required** | string | The URL we `POST` deliveries to. | | `description` | string | A label shown in the dashboard and API listings. | | `events` | string[] | Event filter. Empty or omitted = **all events**. | ### Response <StatusBadge code={201} text="Created" /> ```json { "id": "9b1c22e4-…", "url": "https://your-app.com/hooks/anakin", "description": "Production receiver", "events": ["job.completed", "job.failed"], "isActive": true, "secret": "whsec_…", "createdAt": "2026-07-13T10:00:00Z" } ``` The response includes the endpoint's **signing secret** (`whsec_…`) — copy it into your receiver to [verify deliveries](/docs/api-reference/webhooks#verify-the-signature). Creating an endpoint beyond the limit of 10, or with an invalid URL, returns `400`. --- <EndpointBanner method="GET" path="https://api.anakin.io/v1/webhooks" /> List your registered endpoints. <StatusBadge code={200} text="OK" /> ```json { "endpoints": [ { "id": "9b1c22e4-…", "url": "https://your-app.com/hooks/anakin", "description": "Production receiver", "events": ["job.completed", "job.failed"], "isActive": true, "createdAt": "2026-07-13T10:00:00Z" } ] } ``` --- <EndpointBanner method="PATCH" path="https://api.anakin.io/v1/webhooks/{id}" /> Update an endpoint. All fields are optional — only the fields you send change. ### Request Body ```json { "events": ["job.completed", "job.failed", "monitor.change"], "isActive": true } ``` | Parameter | Type | Description | |-----------|------|-------------| | `url` | string | New destination URL. | | `description` | string | New label. | | `events` | string[] | **Replaces** the event filter. Empty = all events. | | `isActive` | boolean | `false` pauses deliveries to this endpoint without deleting it. | ### Response <StatusBadge code={200} text="OK" /> Returns the updated endpoint. A missing or non-owned endpoint returns `404`. --- <EndpointBanner method="DELETE" path="https://api.anakin.io/v1/webhooks/{id}" /> Delete an endpoint. Deliveries to it stop immediately. <StatusBadge code={200} text="OK" /> ```json { "success": true } ``` --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/webhooks/{id}/test" /> Send a [`webhook.test`](/docs/api-reference/webhooks/events#webhooktest) event to the endpoint so you can verify reachability and signature handling before real traffic — same as the dashboard's **Test** button. <StatusBadge code={200} text="OK" /> ```json { "deliveryId": "d_7c0f31…", "httpStatus": 200, "success": true } ``` `httpStatus` is the status your endpoint returned. --- ## Delivery log Every delivery attempt — to registered endpoints, to per-request `webhook_url`s, and for monitor alerts — is recorded in one account-wide log, retained for **30 days**. <EndpointBanner method="GET" path="https://api.anakin.io/v1/webhooks/deliveries" /> List recent deliveries across your account, newest first. ### Query Parameters | Parameter | Type | Description | |-----------|------|-------------| | `endpointId` | string | Only deliveries to one registered endpoint. | | `event` | string | Filter by event type (e.g. `job.failed`). | | `status` | string | `pending`, `success`, `failed`, or `exhausted`. | | `jobId` | string | Deliveries produced by one job. | | `limit` | number | Maximum rows to return. | ### Response <StatusBadge code={200} text="OK" /> ```json { "deliveries": [ { "id": "d_7c0f31…", "endpointId": "9b1c22e4-…", "event": "job.completed", "jobId": "job_abc123xyz", "url": "https://your-app.com/hooks/anakin", "status": "success", "httpStatus": 200, "attempts": 1, "createdAt": "2026-07-13T10:00:06Z" } ] } ``` | Field | Type | Description | |-------|------|-------------| | `id` | string | Delivery ID — equals the `X-Anakin-Delivery-Id` header. | | `endpointId` | string \| null | The registered endpoint, or `null` for per-request `webhook_url` deliveries. | | `event` | string | The event type delivered. | | `jobId` | string \| null | The job that produced the event, when applicable. | | `status` | string | `pending`, `success`, `failed`, or `exhausted` (retries used up). | | `httpStatus` | number \| null | HTTP status your endpoint returned. | | `attempts` | number | Delivery attempts so far. | | `error` | string \| null | Failure reason, when failed. | | `nextAttemptAt` | string \| null | When the next retry is scheduled (while `pending`). | --- <EndpointBanner method="GET" path="https://api.anakin.io/v1/webhooks/{id}/deliveries" /> Same response shape, scoped to one registered endpoint. --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/webhooks/deliveries/{id}/resend" /> Re-send a `failed` or `exhausted` delivery. The body and signature are **byte-identical** to the original (only `X-Anakin-Timestamp` is fresh), and `X-Anakin-Delivery-Id` is unchanged — so your de-duplication keeps working. Also available from the dashboard. <StatusBadge code={200} text="OK" /> ```json { "success": true } ``` --- ## Signing secret Your account's **default signing secret** signs per-request `webhook_url` deliveries. (Registered endpoints each have their own secret, returned at creation.) <EndpointBanner method="GET" path="https://api.anakin.io/v1/webhooks/signing-secret" /> <StatusBadge code={200} text="OK" /> ```json { "secret": "whsec_…" } ``` --- <EndpointBanner method="POST" path="https://api.anakin.io/v1/webhooks/signing-secret/rotate" /> Generate a new default signing secret. Subsequent per-request deliveries are signed with the new secret — update your receiver with the returned value immediately. <StatusBadge code={200} text="OK" /> ```json { "secret": "whsec_…" } ``` --- ## Event list <EndpointBanner method="GET" path="https://api.anakin.io/v1/webhooks/events" /> The machine-readable [event taxonomy](/docs/api-reference/webhooks/events) — useful for building event pickers. <StatusBadge code={200} text="OK" /> ```json { "events": [ "job.completed", "job.failed", "batch.completed", "batch.failed", "wire.job.completed", "wire.job.failed", "monitor.change", "ai.search.completed", "ai.search.failed", "webhook.test" ] } ``` --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # Create an endpoint (the response includes its whsec_… secret) curl -X POST https://api.anakin.io/v1/webhooks \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "url": "https://your-app.com/hooks/anakin", "description": "Production receiver", "events": ["job.completed", "job.failed"] }' # Fire a test delivery curl -X POST https://api.anakin.io/v1/webhooks/9b1c22e4-.../test -H "X-API-Key: your_api_key" # Inspect recent failures curl "https://api.anakin.io/v1/webhooks/deliveries?status=failed&limit=20" -H "X-API-Key: your_api_key" # Resend one curl -X POST https://api.anakin.io/v1/webhooks/deliveries/d_7c0f31.../resend -H "X-API-Key: your_api_key" # Rotate the default signing secret (per-request webhook_url deliveries) curl -X POST https://api.anakin.io/v1/webhooks/signing-secret/rotate -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests headers = {"X-API-Key": "your_api_key"} base = "https://api.anakin.io/v1/webhooks" # Create an endpoint endpoint = requests.post(base, headers=headers, json={ "url": "https://your-app.com/hooks/anakin", "description": "Production receiver", "events": ["job.completed", "job.failed"], }).json() print(endpoint["id"], endpoint["secret"]) # store the whsec_… secret # Fire a test delivery print(requests.post(f"{base}/{endpoint['id']}/test", headers=headers).json()) # Resend anything that failed deliveries = requests.get(f"{base}/deliveries", headers=headers, params={"status": "failed", "limit": 20}).json()["deliveries"] for d in deliveries: requests.post(f"{base}/deliveries/{d['id']}/resend", headers=headers) ``` </Tab> <Tab value="JavaScript"> ```javascript const headers = { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }; const base = 'https://api.anakin.io/v1/webhooks'; // Create an endpoint const endpoint = await fetch(base, { method: 'POST', headers, body: JSON.stringify({ url: 'https://your-app.com/hooks/anakin', description: 'Production receiver', events: ['job.completed', 'job.failed'] }) }).then(r => r.json()); console.log(endpoint.id, endpoint.secret); // store the whsec_… secret // Fire a test delivery const test = await fetch(`${base}/${endpoint.id}/test`, { method: 'POST', headers }).then(r => r.json()); console.log(test.httpStatus, test.success); // Resend anything that failed const { deliveries } = await fetch(`${base}/deliveries?status=failed&limit=20`, { headers }).then(r => r.json()); for (const d of deliveries) { await fetch(`${base}/deliveries/${d.id}/resend`, { method: 'POST', headers }); } ``` </Tab> </Tabs> --- # POST Build Request (/docs/api-reference/wire/create-build-request) <EndpointBanner method="POST" path="https://api.anakin.io/v1/wire/build-request" /> Request a brand-new action for a website that doesn't have one yet. Wire's builder generates a scraper from your `goal`, auto-tests it against the live site, and publishes it to the catalog on success. The call is asynchronous — it returns immediately with `status: "pending"`. Credits are charged on submission and refunded automatically if the build fails. Track progress by listing your build requests (`GET /v1/wire/build-requests`) or from the [Wire dashboard](/wire). Once a request reaches `success`, run its action with [POST /v1/wire/task](/docs/api-reference/wire/execute-task). --- ### Request Body ```json { "website_url": "https://example.com", "goal": "Extract product name, price, and availability from a product page" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `website_url` **required** | string | The site to build an action for. The domain is extracted automatically (leading `www.` stripped) | | `goal` **required** | string | Natural-language description of what the action should do or extract. Be specific — the builder synthesizes the scraper from this | | `catalog_id` | string (UUID) | Attach the action to an existing catalog instead of creating one. Find IDs in [GET /v1/wire/catalog](/docs/api-reference/wire/list-catalogs) | | `visibility` | string | `"private"` (default) or `"public"`. Private actions are only visible to your account | | `force` | boolean | Defaults to `false`. If similar actions already exist for the domain the request returns `409 ACTION_EXISTS`; set `true` to build anyway | --- ### Response <StatusBadge code={201} text="Created" /> ```json { "status": "ok", "build_request": { "id": "b1a2c3d4-e5f6-7890-abcd-ef1234567890", "user_id": "9f8e7d6c-5b4a-3210-fedc-ba9876543210", "website_url": "https://example.com", "domain": "example.com", "goal": "Extract product name, price, and availability from a product page", "visibility": "private", "status": "pending", "credits_charged": 25, "priority": 0, "team": null, "tags": [], "created_at": "2026-05-28T12:00:00Z", "updated_at": "2026-05-28T12:00:00Z" } } ``` | Field | Type | Description | |-------|------|-------------| | `id` | string (UUID) | The build request ID — use it to track the build | | `status` | string | Starts at `pending`, then ends at `success` (action is live) or `failed` (credits refunded) | | `credits_charged` | integer | Credits held for this build. Refunded if it fails | | `action_id` | string | Present once published — pass it to [POST /v1/wire/task](/docs/api-reference/wire/execute-task) | | `error`, `error_type` | string | Populated when `status` is `failed` | --- ### Error Responses | Code | HTTP | When | |------|------|------| | `INVALID_INPUT` | 400 | `website_url` or `goal` missing, `website_url` malformed, or `visibility` isn't `private`/`public` | | `BLOCKED_WEBSITE` | 400 | The site isn't supported for scraping | | `INSUFFICIENT_CREDITS` | 402 | Your balance can't cover the build cost. Response includes `balance` and `required` | | `ACTION_EXISTS` | 409 | Similar actions already exist for this domain. Response includes `existing_actions`; set `force: true` to proceed | | `BUILD_LIMIT_REACHED` | 429 | Too many pending build requests (max 3). Wait for one to finish | | `INTERNAL_ERROR` | 500 | Failed to create the build request. Any charged credits are refunded | **Action exists** — `409 Conflict` ```json { "status": "error", "error": { "code": "ACTION_EXISTS", "message": "Similar actions already exist for this website. Set force=true to proceed anyway.", "existing_actions": [ { "action_id": "example_com_product_detail", "name": "Product Detail", "description": "Extract product name, price, and availability" } ] } } ``` --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST https://api.anakin.io/v1/wire/build-request \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "website_url": "https://example.com", "goal": "Extract product name, price, and availability from a product page" }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( 'https://api.anakin.io/v1/wire/build-request', headers={'X-API-Key': 'your_api_key'}, json={ 'website_url': 'https://example.com', 'goal': 'Extract product name, price, and availability from a product page' } ) data = response.json() print(f"Build request: {data['build_request']['id']} ({data['build_request']['status']})") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('https://api.anakin.io/v1/wire/build-request', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ website_url: 'https://example.com', goal: 'Extract product name, price, and availability from a product page' }) }); const { build_request } = await response.json(); console.log('Build request:', build_request.id, build_request.status); ``` </Tab> </Tabs> --- ### Limits There is no per-minute rate limit on this endpoint. Instead, concurrency is bounded: if you submit while you already have the maximum number of pending build requests, the call returns `BUILD_LIMIT_REACHED` (`429`) — wait for one to finish before submitting another. Each build request costs credits (currently 25), refunded automatically if the build fails. ### Related - [GET /v1/wire/catalog](/docs/api-reference/wire/list-catalogs) — find a `catalog_id` to attach the new action to - [POST /v1/wire/task](/docs/api-reference/wire/execute-task) — run the action once the build succeeds --- # POST Connect Source (/docs/api-reference/wire/create-identity-source) <EndpointBanner method="POST" path="/identity-sources" /> Connects a vault you own so [logins](/docs/api-reference/wire/login) can read credentials from it instead of you typing a password into Wire. The body is generic: `provider`, `display_name`, an optional `auth_method`, and then whatever fields that provider declares — **at the top level, not nested**. Read those fields from [GET /v1/wire/identity-sources/providers](/docs/api-reference/wire/list-providers) rather than hardcoding them. > **A failed connect stores nothing.** Wire proves the credential against the provider *before* persisting — for Azure, against every vault URL you supplied. There is no half-built connection to clean up, and a rejected credential never becomes a broken source. Credentials are encrypted at rest (AES-256-GCM) and never returned by any endpoint. --- ### Request Body | Parameter | Type | Description | |-----------|------|-------------| | `provider` **required** | string | Provider key from [list providers](/docs/api-reference/wire/list-providers) — `1password` or `azure_key_vault` | | `display_name` **required** | string | Your label for the connection. Not required to be unique — renames after rotation are common | | `auth_method` | string | Which of the provider's `methods[].id` to use. Omit for the provider's default (`methods[0]`) | | *provider fields* | string | Every field the chosen method declares, top-level. Values must be strings — a non-string is rejected rather than coerced | **1Password** — one field, a Service Account token: ```json { "provider": "1password", "display_name": "Engineering vault", "token": "ops_..." } ``` Create the token under **Developer Tools → Service Accounts** in 1Password, granting it read access to the vaults you want Wire to use — see [Provider Setup](/docs/api-reference/wire/provider-setup#1password). **Azure Key Vault** — an Entra service principal plus the vaults it may read: ```json { "provider": "azure_key_vault", "display_name": "Prod Key Vault", "auth_method": "service_principal", "tenant_id": "00000000-0000-0000-0000-000000000000", "client_id": "00000000-0000-0000-0000-000000000000", "client_secret": "...", "vault_urls": "https://my-vault.vault.azure.net\nhttps://other-vault.vault.azure.net" } ``` `vault_urls` is newline-separated. Azure has no API for "which vaults may this principal read", so you list them — and the app must hold the **Key Vault Secrets User** role on each. That role assignment is the step most often missed: without it the connection authenticates but reads nothing, and connect fails naming the vault. Full walkthrough: [Provider Setup](/docs/api-reference/wire/provider-setup#azure-key-vault). --- ### Response <StatusBadge code={201} text="Created" /> ```json { "status": "ok", "identity_source": { "id": "f1e2d3c4-0000-0000-0000-000000000000", "user_id": "8a7b6c5d-0000-0000-0000-000000000000", "provider": "1password", "display_name": "Engineering vault", "config": { "auth_method": "service_account" }, "scope_metadata": { "vaults": [ { "id": "abcd1234", "name": "Engineering" } ] }, "status": "active", "created_at": "2026-06-01T09:00:00Z", "updated_at": "2026-06-01T09:00:00Z" } } ``` Same shape as an element of [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources#response). `scope_metadata.vaults` is discovered during the connect check, so you can show what the connection reaches without a second live call. > **`config` only carries `auth_method` if you sent one.** Omit `auth_method` — the common case, since it defaults to `methods[0]` — and `config` comes back `{}`. The connection still works identically; a later [rotation](/docs/api-reference/wire/update-identity-source) simply falls back to the provider's default method rather than reusing a recorded one. > **You can connect the same provider more than once** — one source per vault, or dev and prod side by side. Display-name collisions aren't blocked either. --- ### Error Responses All errors return JSON of the form `{ "status": "error", "error": { "code": "...", "message": "..." } }`. Provider-specific codes carry the provider's own guidance in `message`. | Code | HTTP | When | |------|------|------| | `INVALID_BODY` | 400 | Body isn't a JSON object, or a field's value isn't a string | | `INVALID_INPUT` | 400 | `provider` or `display_name` missing, or a required provider field is blank | | `UNSUPPORTED_PROVIDER` | 400 | This engine has no source registered for that `provider` | | `UNSUPPORTED_AUTH_METHOD` | 400 | The provider doesn't offer that `auth_method` | | `UNKNOWN_CONNECT_FIELD` | 400 | A field name the chosen method doesn't declare — usually a typo | | `INVALID_TOKEN_FORMAT` | 400 | (1Password) The token doesn't start with `ops_` | | `INVALID_VAULT_URL` | 400 | (Azure) A `vault_urls` entry is malformed, isn't an Azure Key Vault address, or no Key Vault exists at it. Hosts must end in `.vault.azure.net`, `.vault.usgovcloudapi.net` or `.vault.azure.cn` — arbitrary URLs are refused, so Wire can't be pointed at a host of your choosing | | `SOURCE_TOKEN_REJECTED` | 400 | The provider rejected the credentials. Check them and the access they grant | | `SOURCE_FORBIDDEN` | 400 | Authentication succeeded but the credential can't read the requested vault. Grant the role, then retry | | `SOURCE_ITEM_NOT_FOUND` | 400 | The referenced vault was not found | | `SOURCE_THROTTLED` | 429 | The provider is rate-limiting us. Retry shortly | | `SOURCE_VERIFY_FAILED` | 400 | Could not reach the provider to verify. Retry in a moment | | `SOURCE_NOT_AVAILABLE` | 503 | The source resolver isn't configured on this engine | > `SOURCE_TOKEN_REJECTED` and `SOURCE_FORBIDDEN` look alike and mean opposite things. The first says the credential is wrong; the second says the credential is fine but hasn't been granted the vault. Only the second is fixed at the provider without issuing a new secret. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # 1Password curl {API_BASE}/identity-sources \ -X POST \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "provider": "1password", "display_name": "Engineering vault", "token": "ops_your_service_account_token" }' # Azure Key Vault curl {API_BASE}/identity-sources \ -X POST \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "provider": "azure_key_vault", "display_name": "Prod Key Vault", "tenant_id": "00000000-0000-0000-0000-000000000000", "client_id": "00000000-0000-0000-0000-000000000000", "client_secret": "your_client_secret", "vault_urls": "https://my-vault.vault.azure.net" }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( '{API_BASE}/identity-sources', headers={'X-API-Key': 'your_api_key'}, json={ 'provider': '1password', 'display_name': 'Engineering vault', 'token': 'ops_your_service_account_token', }, ) data = response.json() if data['status'] == 'ok': print(f"Connected: {data['identity_source']['id']}") else: print(f"Error: {data['error']['code']} — {data['error']['message']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch(`{API_BASE}/identity-sources`, { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ provider: '1password', display_name: 'Engineering vault', token: 'ops_your_service_account_token', }), }); const data = await response.json(); if (data.status === 'ok') { console.log(`Connected: ${data.identity_source.id}`); } else { console.error(`Error: ${data.error.code} — ${data.error.message}`); } ``` </Tab> </Tabs> --- ### Rate limit 20 requests per minute per user (each call verifies live against the provider). ### Related - [GET /v1/wire/identity-sources/providers](/docs/api-reference/wire/list-providers) — the fields to send, per provider - [GET /v1/wire/identity-sources/\{id\}/containers](/docs/api-reference/wire/list-source-containers) — browse what the new source can read - [PATCH /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/update-identity-source) — rename or rotate the credential - [POST /v1/wire/login](/docs/api-reference/wire/login) — sign in using an entry from this source --- # DELETE Disconnect Source (/docs/api-reference/wire/delete-identity-source) <EndpointBanner method="DELETE" path="/identity-sources/{id}" /> Disconnects a source and deletes its stored credential. What happens to the identities that were sourced from it is your choice, and it is the only decision on this page that can't be undone. Requires an `X-API-Key`; the source must belong to the authenticated user. Check [`identities_using`](/docs/api-reference/wire/get-identity-source) first to see what's at stake. --- ### Two behaviours **Keep the identities (default).** The identities survive with their `source_id` cleared. They keep running on their cached session cookies until those expire, then can't refresh — the dashboard shows them as *source disconnected*. Reconnect and re-bind to restore them. ``` DELETE /identity-sources/{id} ``` **Delete the identities too.** The sourced identities are removed along with the source, in one atomic statement. **This cannot be undone.** ``` DELETE /identity-sources/{id}/identities DELETE /identity-sources/{id}?delete_identities=true ``` <EndpointBanner method="DELETE" path="/identity-sources/{id}/identities" /> Both destructive forms do the same thing. Prefer the `/identities` path: the destructive intent is in the URL, so it can't be reached by a stale link or a copy-pasted query string that lost its flag. The `?delete_identities=true` variant is kept for existing callers. > Any value other than exactly `true` for `delete_identities` is treated as false — the safe reading. `?delete_identities=1` and `?delete_identities=TRUE` both **keep** your identities. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | Identity source ID | ### Query Parameters | Parameter | Type | Description | |-----------|------|-------------| | `delete_identities` | string | `true` deletes the sourced identities along with the source. Anything else keeps them. Ignored on the `/identities` path, which always deletes | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok" } ``` Identical for both behaviours. It does not report how many identities were affected — read [`identities_using`](/docs/api-reference/wire/get-identity-source) before deleting if you need that. --- ### Error Responses | Code | HTTP | When | |------|------|------| | `NOT_FOUND` | 404 | No source with that ID | | `FORBIDDEN` | 403 | The source belongs to another user | | `INTERNAL_ERROR` | 500 | The disconnect failed; nothing was deleted | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # keep the identities — they run on cached cookies until those expire curl {API_BASE}/identity-sources/f1e2d3c4-0000-0000-0000-000000000000 \ -X DELETE \ -H "X-API-Key: your_api_key" # delete the identities too — irreversible curl {API_BASE}/identity-sources/f1e2d3c4-0000-0000-0000-000000000000/identities \ -X DELETE \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests base = '{API_BASE}/identity-sources' source_id = 'f1e2d3c4-0000-0000-0000-000000000000' headers = {'X-API-Key': 'your_api_key'} # Know the blast radius before deciding. detail = requests.get(f'{base}/{source_id}', headers=headers).json() print(f"{detail.get('identities_using', '?')} identities depend on this source") # Keep them: requests.delete(f'{base}/{source_id}', headers=headers) # Or remove them with the source (irreversible): # requests.delete(f'{base}/{source_id}/identities', headers=headers) ``` </Tab> <Tab value="JavaScript"> ```javascript const base = `{API_BASE}/identity-sources`; const sourceId = 'f1e2d3c4-0000-0000-0000-000000000000'; const headers = { 'X-API-Key': 'your_api_key' }; // Know the blast radius before deciding. const detail = await (await fetch(`${base}/${sourceId}`, { headers })).json(); console.log(`${detail.identities_using ?? '?'} identities depend on this source`); // Keep them: await fetch(`${base}/${sourceId}`, { method: 'DELETE', headers }); // Or remove them with the source (irreversible): // await fetch(`${base}/${sourceId}/identities`, { method: 'DELETE', headers }); ``` </Tab> </Tabs> --- ### Rate limit 20 requests per minute per user, for both forms. ### Related - [GET /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/get-identity-source) — `identities_using` before you disconnect - [GET /v1/wire/identity-sources/\{id\}/identities](/docs/api-reference/wire/list-source-identities) — exactly which identities those are - [PATCH /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/update-identity-source) — rotate instead of disconnecting; bindings survive --- # POST Execute Task (/docs/api-reference/wire/execute-task) <EndpointBanner method="POST" path="/task" /> Submit a task to execute a Wire action. Credits are deducted immediately at submission and refunded automatically if the job fails. Poll for results using [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job). --- ### Request Body ```json { "action_id": "li_profile_scrape", "credential_id": "11111111-2222-3333-4444-555555555555", "params": { "profile_url": "https://www.linkedin.com/in/example" }, "webhook_url": "https://your-app.com/hooks/anakin" } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | The action to run. Find action IDs in the [Wire dashboard](/wire) | | `credential_id` | string (UUID) | Required when the action has `auth_mode: "required"`; optional when `auth_mode: "optional"` (supply to run authenticated, omit to run anonymously). Get IDs from [GET /v1/wire/identities](/docs/api-reference/wire/list-identities) | | `params` | object | Action-specific input parameters | | `webhook_url` | string | URL to `POST` a signed `wire.job.completed` / `wire.job.failed` event to when the task settles — delivered within ~10 seconds of completion. See [Webhooks](/docs/api-reference/webhooks) | > **`auth_mode` semantics**: > - `required` — `credential_id` is mandatory; otherwise the request returns `401 AUTH_REQUIRED`. > - `optional` — `credential_id` is honored if provided (authenticated branch). Omit it to run anonymously. > - `none` — `credential_id` is ignored if provided. > > List your credentials with [GET /v1/wire/identities](/docs/api-reference/wire/list-identities), or manage them in the [Identities dashboard](/wire/identities). --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "poll_url": "/v1/wire/jobs/7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c" } ``` | Field | Type | Description | |-------|------|-------------| | `job_id` | string | Use this to poll for results | | `poll_url` | string | Convenience path for the polling endpoint | Use the `job_id` with [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) to retrieve your results. --- ### Error Responses | Code | HTTP | When | |------|------|------| | `BAD_REQUEST` | 400 | Request body isn't valid JSON, or `action_id` is missing/invalid | | `NOT_FOUND` | 404 | No action matches `action_id` | | `ACTION_UNAVAILABLE` | 503 | The action exists but is not active | | `AUTH_REQUIRED` | 401 | The action requires a connected account and no credential was supplied | | `AUTH_EXPIRED` | 401 | The `credential_id` exists but its session is no longer valid | | `FORBIDDEN` | 403 | The `credential_id` doesn't belong to you or belongs to a different catalog | | `INSUFFICIENT_CREDITS` | 402 | Your balance can't cover the action's credit cost | | `EXECUTION_FAILED` | 500 | The engine couldn't enqueue the task (no credits deducted) | **Insufficient credits** — `402 Payment Required` ```json { "status": "error", "error": { "code": "INSUFFICIENT_CREDITS", "message": "You need 5 credits. Current balance: 2", "balance": 2, "required": 5 } } ``` **Authentication required** — `401 Unauthorized` Returned when the action requires you to connect your account for the target website and you have no credential for it. Visit the `connect_url` to authenticate. ```json { "status": "error", "error": { "code": "AUTH_REQUIRED", "message": "This action requires a LinkedIn connection.", "connect_url": "/products/wire/linkedin/connect" } } ``` **Credential expired** — `401 Unauthorized` Returned when the `credential_id` you passed exists but its session is no longer valid (cookies expired, token revoked, password changed, etc.). The user needs to reconnect. List your credentials with [GET /v1/wire/identities](/docs/api-reference/wire/list-identities) — `status` will show `expired`. ```json { "status": "error", "error": { "code": "AUTH_EXPIRED", "message": "Credential is no longer active. Please reconnect." } } ``` **Forbidden** — `403 Forbidden` Returned when the `credential_id` you passed exists but doesn't belong to you, or belongs to a different catalog than the action you're calling. Common cause: copy/pasting a `credential_id` between accounts, or accidentally passing an Amazon credential to a LinkedIn action. ```json { "status": "error", "error": { "code": "FORBIDDEN", "message": "Credential does not belong to this user or catalog" } } ``` **Submission failed** — `500 Internal Server Error` Transient server-side failure when enqueueing the task. Credits are not deducted. Retry the request; if it persists, contact support with the request timestamp. ```json { "status": "error", "error": { "code": "EXECUTION_FAILED", "message": "Failed to submit task. Please try again." } } ``` > Note: `EXECUTION_FAILED` can also appear as a final status on a successfully submitted job (see [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job)). Same code, different meaning — at submission it means the engine couldn't enqueue; at job completion it means the scraper raised an unrecoverable error. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "li_profile_scrape", "credential_id": "11111111-2222-3333-4444-555555555555", "params": { "profile_url": "https://www.linkedin.com/in/example" } }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={ 'action_id': 'li_profile_scrape', 'credential_id': '11111111-2222-3333-4444-555555555555', 'params': { 'profile_url': 'https://www.linkedin.com/in/example' } } ) data = response.json() print(f"Job submitted: {data['job_id']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'li_profile_scrape', credential_id: '11111111-2222-3333-4444-555555555555', params: { profile_url: 'https://www.linkedin.com/in/example' } }) }); const { job_id } = await response.json(); console.log('Job submitted:', job_id); ``` </Tab> </Tabs> > `credential_id` is mandatory for actions with `auth_mode: "required"` (e.g. `li_profile_scrape`). It is honored when supplied for `auth_mode: "optional"` actions and ignored for `auth_mode: "none"`. Get yours from [GET /v1/wire/identities](/docs/api-reference/wire/list-identities). --- ### Rate limit 20 requests per minute per user. --- # GET Catalog Details (/docs/api-reference/wire/get-catalog) <EndpointBanner method="GET" path="/catalog/{slug}" /> Returns the catalog entry for a single website plus every visible action it exposes. This is the canonical way to discover an action's `action_id`, parameter schema, mode (`async` / `sync`), and credit cost before calling [POST /v1/wire/task](/docs/api-reference/wire/execute-task). Requires an `X-API-Key`. Returns public actions plus any private actions you own. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `slug` **required** | string | Catalog slug (e.g. `airbnb`, `linkedin`). Get slugs from [GET /v1/wire/catalog](/docs/api-reference/wire/list-catalogs) | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "catalog": { "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "slug": "airbnb", "name": "Airbnb", "url": "https://www.airbnb.com", "domain": "airbnb.com", "category": "travel", "description": "Search listings, fetch reviews, and pull host details.", "logo_url": "https://cdn.anakin.io/logos/airbnb.png", "auth_required": true, "auth_type": "cookie", "auth_types": ["browser_state", "credentials"], "auth_login_url": "https://www.airbnb.com/login", "supported_sources": ["1password", "azure_key_vault"], "status": "active", "created_at": "2026-02-01T00:00:00Z", "updated_at": "2026-04-20T12:00:00Z", "action_count": 4 }, "actions": [ { "id": "f1e2d3c4-b5a6-7890-1234-56789abcdef0", "action_id": "ab_search_listings", "catalog_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "name": "Search Listings", "description": "Search Airbnb listings by query, dates, and guest count.", "tags": ["search", "listings"], "type": "read", "mode": "async", "auth_mode": "none", "auth_required": false, "parameters": [ { "name": "query", "type": "string", "required": true, "description": "Location query, e.g. 'pacific heights'" }, { "name": "checkin", "type": "string", "required": false, "description": "ISO date" }, { "name": "checkout", "type": "string", "required": false, "description": "ISO date" }, { "name": "adults", "type": "integer", "required": false, "default": 1 }, { "name": "children", "type": "integer", "required": false }, { "name": "infants", "type": "integer", "required": false }, { "name": "pets", "type": "integer", "required": false }, { "name": "cursor", "type": "string", "required": false, "description": "Pagination cursor from previous response" }, { "name": "currency", "type": "string", "required": false }, { "name": "locale", "type": "string", "required": false } ], "credits_per_call": 1, "wheel_version": "0.4.2", "wheel_filename": "ab_search_listings-0.4.2-py3-none-any.whl", "build_request_id": "6964eeb6-b426-4095-a1ab-b443f8311c9d", "status": "active", "created_at": "2026-02-01T00:00:00Z", "updated_at": "2026-04-20T12:00:00Z" } ], "login_input_schema": [ { "name": "email", "type": "string", "required": true, "default": "", "description": "Account email for the catalog login." }, { "name": "password", "type": "string", "required": true, "default": "", "description": "Account password. Sensitive — never logged or persisted." }, { "name": "country", "type": "string", "required": false, "default": "us", "description": "ISO country code for the storefront region." } ] } ``` #### Catalog fields See [GET /v1/wire/catalog](/docs/api-reference/wire/list-catalogs#response) — the `catalog` object on this endpoint matches the same `CatalogEntry` shape. In addition to the fields documented there, the object always includes: | Field | Type | Description | |-------|------|-------------| | `catalog.auth_type` | string \| null | Primary credential type for this catalog (e.g. `browser_state`, `credentials`), or `null` if the catalog has no auth | | `catalog.auth_login_url` | string \| null | Where users should log in to seed a `browser_state` credential | | `catalog.supported_sources` | string[] | Identity-source providers this catalog accepts credentials from (`1password`, `azure_key_vault`); empty when none. Binding a [source-backed login](/docs/api-reference/wire/login) is rejected with `PROVIDER_NOT_SUPPORTED_BY_CATALOG` when the provider isn't listed here | > Optional fields (`logo_url`, `auth_type`, `auth_login_url`) are omitted when not set, not returned as `null`. #### Login input schema | Field | Type | Description | |-------|------|-------------| | `login_input_schema` | array | Top-level array of `{ name, type, required, default, description }` — the login fields for credentials-mode catalogs. Empty for catalogs without a credentials login | #### Action fields | Field | Type | Description | |-------|------|-------------| | `actions[].action_id` | string | **Pass this as `action_id` when calling [POST /v1/wire/task](/docs/api-reference/wire/execute-task)** | | `actions[].name` | string | Display name | | `actions[].description` | string \| null | Short summary | | `actions[].tags` | string[] | Free-form tags for filtering / discovery | | `actions[].type` | string | Action category — `read` or `write` | | `actions[].mode` | string | `async` (returns `job_id`, poll for results) or `sync` (returns data inline) | | `actions[].auth_mode` | string | `"none"` \| `"optional"` \| `"required"`. `required` ⇒ `credential_id` mandatory; `optional` ⇒ `credential_id` honored if supplied; `none` ⇒ credentials are ignored | | `actions[].auth_required` | boolean | **Deprecated** legacy mirror — true iff `auth_mode === "required"`. Prefer reading `auth_mode` | | `actions[].parameters` | array | Array of param descriptors — `{ name, type, required, default?, description? }` — one per input the action accepts via [POST /v1/wire/task](/docs/api-reference/wire/execute-task) | | `actions[].credits_per_call` | integer | Base credit cost. Some actions use a CEL expression in `action_config.credit_expression` to compute final cost from the response — final cost is reflected on the [job record](/docs/api-reference/wire/get-job) | | `actions[].wheel_version` | string \| null | Build version of the underlying scraper | | `actions[].wheel_filename` | string | Filename of the built wheel (e.g. `mb_search-1.1.0-py3-none-any.whl`) | | `actions[].build_request_id` | string (UUID) | The build request that produced this action | | `actions[].status` | string | `active` or `pending_review` | | `actions[].owner_user_id` | string \| null | Set when this is a private action you built via [build-request](/wire) | > Only actions visible to the requesting user are returned. Private actions belonging to other users are filtered out server-side. --- ### Error Responses **Catalog not found** — `404 Not Found` ```json { "status": "error", "error": { "code": "NOT_FOUND", "message": "Catalog entry not found" } } ``` --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/catalog/airbnb \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests response = requests.get( '{API_BASE}/catalog/airbnb', headers={'X-API-Key': 'your_api_key'} ) data = response.json() print(f"Catalog: {data['catalog']['name']}") for action in data['actions']: print(f" {action['action_id']:30} {action['credits_per_call']} cr {action['name']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('{API_BASE}/catalog/airbnb', { headers: { 'X-API-Key': 'your_api_key' } }); const { catalog, actions } = await response.json(); console.log(`Catalog: ${catalog.name}`); for (const action of actions) { console.log(` ${action.action_id.padEnd(30)} ${action.credits_per_call} cr ${action.name}`); } ``` </Tab> </Tabs> --- ### Rate Limit 120 requests per minute per IP. --- # GET Source Details (/docs/api-reference/wire/get-identity-source) <EndpointBanner method="GET" path="/identity-sources/{id}" /> Returns a single connected source. Unlike [the list endpoint](/docs/api-reference/wire/list-identity-sources), this one also carries `identities_using` — how many identities would be affected if you disconnected it. That count is the thing worth checking before a [disconnect](/docs/api-reference/wire/delete-identity-source). The encrypted credential is never returned. Requires an `X-API-Key`; the source must belong to the authenticated user. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | Identity source ID | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "identity_source": { "id": "f1e2d3c4-0000-0000-0000-000000000000", "user_id": "8a7b6c5d-0000-0000-0000-000000000000", "provider": "1password", "display_name": "Engineering vault", "config": { "auth_method": "service_account" }, "scope_metadata": { "vaults": [ { "id": "abcd1234", "name": "Engineering" } ] }, "status": "active", "last_verified_at": "2026-06-08T10:00:00Z", "created_at": "2026-06-01T09:00:00Z", "updated_at": "2026-06-08T10:00:00Z" }, "identities_using": 3 } ``` | Field | Type | Description | |-------|------|-------------| | `identity_source` | object | Same shape as an element of [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources#response) | | `identity_source.config.auth_method` | string | Which auth method the source was connected with — **present only if `auth_method` was sent at connect time**, otherwise `config` is `{}`. A [rotation](/docs/api-reference/wire/update-identity-source) reuses it when present, and falls back to the provider's default when not | | `identities_using` | number | Identities whose credentials come from this source. Use [GET /v1/wire/identity-sources/\{id\}/identities](/docs/api-reference/wire/list-source-identities) for the list itself | > **`identities_using` is omitted — not zeroed — when the count can't be taken.** The source is still returned, so treat an absent key as *unknown* and a `0` as *counted, and nothing depends on it*. The distinction matters before a [cascading disconnect](/docs/api-reference/wire/delete-identity-source): "unknown" read as "zero" is how you delete identities you were told didn't exist. The list endpoint deliberately omits this count: computing it per row would be an N+1 query. --- ### Error Responses | Code | HTTP | When | |------|------|------| | `NOT_FOUND` | 404 | No source with that ID | | `FORBIDDEN` | 403 | The source belongs to another user | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/identity-sources/f1e2d3c4-0000-0000-0000-000000000000 \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests source_id = 'f1e2d3c4-0000-0000-0000-000000000000' response = requests.get( f'{API_BASE}/identity-sources/{source_id}', headers={'X-API-Key': 'your_api_key'}, ) data = response.json() source = data['identity_source'] print(f"{source['display_name']} ({source['status']}) — {data.get('identities_using', '?')} identities") ``` </Tab> <Tab value="JavaScript"> ```javascript const sourceId = 'f1e2d3c4-0000-0000-0000-000000000000'; const response = await fetch(`{API_BASE}/identity-sources/${sourceId}`, { headers: { 'X-API-Key': 'your_api_key' } }); const { identity_source, identities_using } = await response.json(); console.log( `${identity_source.display_name} (${identity_source.status}) — ${identities_using ?? '?'} identities` ); ``` </Tab> </Tabs> --- ### Rate limit 60 requests per minute per user. ### Related - [GET /v1/wire/identity-sources/\{id\}/identities](/docs/api-reference/wire/list-source-identities) — which identities those are - [POST /v1/wire/identity-sources/\{id\}/verify](/docs/api-reference/wire/verify-identity-source) — re-test the stored credential - [DELETE /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/delete-identity-source) — disconnect --- # GET Job Status (/docs/api-reference/wire/get-job) <EndpointBanner method="GET" path="/jobs/{id}" /> Retrieve the status and results of a Wire task. Poll this endpoint until the job reaches `completed` or `failed`. See the [Polling Jobs](/docs/api-reference/polling-jobs) guide for recommended patterns. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The job ID returned from [POST /v1/wire/task](/docs/api-reference/wire/execute-task) | --- ### Response — Processing <StatusBadge code={200} text="OK" /> ```json { "status": "processing", "retry_after_ms": 2000 } ``` ### Response — Completed <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "data": { "name": "Jane Doe", "headline": "Software Engineer at Acme Corp", "location": "San Francisco, CA", "connections": 500 }, "credits_used": 2, "execution_ms": 8420 } ``` ### Response — Completed (file result) <StatusBadge code={200} text="OK" /> Some actions return a **file** (e.g. a PDF) instead of inline JSON. In that case `data` is `null` and a `files` array lists the artifacts. Fetch the bytes from [`GET /jobs/{id}/download`](/docs/api-reference/wire/get-job-download). ```json { "status": "completed", "data": null, "files": [ { "name": "statement-2026-07-15.pdf", "content_type": "application/pdf", "size_bytes": 128623 } ], "credits_used": 1, "execution_ms": 5575 } ``` ### Response — Failed <StatusBadge code={200} text="OK" /> ```json { "status": "failed", "error": { "code": "EXECUTION_FAILED", "message": "The target page could not be loaded." }, "credits_used": 0 } ``` --- ### Response Fields | Field | Type | Description | |-------|------|-------------| | `status` | string | `processing`, `completed`, or `failed` | | `retry_after_ms` | number | Milliseconds to wait before polling again (present when `processing`) | | `data` | object | Structured result data (present when `completed`; `null` for a file result) | | `files` | array | File artifacts, present when the result is a file (then `data` is `null`). Each entry is `{ name, content_type, size_bytes }`. Download the bytes via [`GET /jobs/{id}/download`](/docs/api-reference/wire/get-job-download) | | `credits_used` | number | Credits charged for this job. `0` on failure (credits are refunded) | | `execution_ms` | number | Total execution time in milliseconds | | `error` | object | Error details (present when `failed`) | ### Job Statuses | Status | Description | |--------|-------------| | `processing` | Job is queued or actively running | | `completed` | Job finished successfully — `data` is populated | | `failed` | Job encountered an error — credits were refunded | --- ### Error Responses | Code | HTTP | When | |------|------|------| | `NOT_FOUND` | 404 | The job doesn't exist, or belongs to another user | ```json { "status": "error", "error": { "code": "NOT_FOUND", "message": "Job not found" } } ``` --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/jobs/7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests import time job_id = '7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c' while True: response = requests.get( f'{API_BASE}/jobs/{job_id}', headers={'X-API-Key': 'your_api_key'} ) data = response.json() if data['status'] == 'completed': print(data['data']) break elif data['status'] == 'failed': print(f"Job failed: {data['error']['message']}") break time.sleep(3) ``` </Tab> <Tab value="JavaScript"> ```javascript const jobId = '7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c'; const poll = async () => { while (true) { const res = await fetch(`{API_BASE}/jobs/${jobId}`, { headers: { 'X-API-Key': 'your_api_key' } }); const data = await res.json(); if (data.status === 'completed') { console.log(data.data); break; } else if (data.status === 'failed') { console.error(data.error.message); break; } await new Promise(r => setTimeout(r, 3000)); } }; poll(); ``` </Tab> </Tabs> For polling patterns, see the [Polling Jobs](/docs/api-reference/polling-jobs) reference. --- # GET Download Job File (/docs/api-reference/wire/get-job-download) <EndpointBanner method="GET" path="/jobs/{id}/download" /> Download a file produced by a completed Wire task, for actions that return a **file** (e.g. a PDF) rather than inline JSON. When a result is a file, [`GET /jobs/{id}`](/docs/api-reference/wire/get-job) returns `data: null` with a `files` manifest; this endpoint streams the actual bytes. The response body **is the file** — not JSON. It sets `Content-Type` and `Content-Disposition: attachment; filename="…"`. Write it to disk; do **not** JSON-parse a successful response. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string | The job ID returned from [POST /v1/wire/task](/docs/api-reference/wire/execute-task) | ### Query Parameters | Parameter | Type | Description | |-----------|------|-------------| | `file` | string | For a multi-file result, the `name` of the file to download (from the `files` manifest in [`GET /jobs/{id}`](/docs/api-reference/wire/get-job)). Omit to stream the primary file. | --- ### Response — Success <StatusBadge code={200} text="OK" /> The raw file bytes, with headers describing the file: ``` Content-Type: application/pdf Content-Disposition: attachment; filename="statement-2026-07-15.pdf" Content-Length: 128623 ``` ### Response — Error <StatusBadge code={400} text="Bad Request" /> Errors are returned as JSON (only the success body is binary), so branch on the status code before treating the body as a file: ```json { "status": "error", "error": { "code": "NO_DOWNLOAD", "message": "Result is not available for download." } } ``` --- ### Error Responses | Code | HTTP | When | |------|------|------| | `NO_DOWNLOAD` | 400 | The job has no downloadable result, or `?file=<name>` doesn't match any file in the manifest | | `NOT_FOUND` | 404 | The job doesn't exist, or belongs to another user | --- ### Multi-file results A job can return more than one file. List them from [`GET /jobs/{id}`](/docs/api-reference/wire/get-job) (`files`), then download each by name: ```bash curl "{API_BASE}/jobs/{id}/download?file=statement.pdf" -H "X-API-Key: your_api_key" -o statement.pdf curl "{API_BASE}/jobs/{id}/download?file=usage.csv" -H "X-API-Key: your_api_key" -o usage.csv ``` --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/jobs/7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c/download \ -H "X-API-Key: your_api_key" \ -o result.pdf ``` </Tab> <Tab value="Python"> ```python import requests job_id = '7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c' resp = requests.get( f'{API_BASE}/jobs/{job_id}/download', headers={'X-API-Key': 'your_api_key'}, ) resp.raise_for_status() with open('result.pdf', 'wb') as f: f.write(resp.content) # raw bytes — do not resp.json() ``` </Tab> <Tab value="JavaScript"> ```javascript const jobId = '7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c'; const res = await fetch(`{API_BASE}/jobs/${jobId}/download`, { headers: { 'X-API-Key': 'your_api_key' }, }); const blob = await res.blob(); // raw bytes — not res.json() // In a browser, trigger a download: const url = URL.createObjectURL(blob); const a = document.createElement('a'); a.href = url; a.download = 'result.pdf'; a.click(); URL.revokeObjectURL(url); ``` </Tab> </Tabs> --- # GET List Catalogs (/docs/api-reference/wire/list-catalogs) <EndpointBanner method="GET" path="/catalog" /> Returns every website in the catalog that has at least one visible action. Use this to discover what's available before drilling into a specific catalog with [GET /v1/wire/catalog/\{slug\}](/docs/api-reference/wire/get-catalog). Requires an `X-API-Key`. Returns all visible catalogs — public catalogs plus any private actions you own. --- ### Query Parameters | Parameter | Type | Description | |-----------|------|-------------| | `scope` | string | Set to `my` to return only catalogs that contain actions you own, or `global` for a public-only view (excludes your private actions). Default: all visible catalogs | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "catalog": [ { "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "slug": "airbnb", "name": "Airbnb", "url": "https://www.airbnb.com", "domain": "airbnb.com", "category": "travel", "description": "Search listings, fetch reviews, and pull host details.", "logo_url": "https://cdn.anakin.io/logos/airbnb.png", "auth_required": false, "auth_types": [], "status": "active", "created_at": "2026-02-01T00:00:00Z", "updated_at": "2026-04-20T12:00:00Z", "action_count": 4 } ] } ``` | Field | Type | Description | |-------|------|-------------| | `catalog[].id` | string (UUID) | Internal catalog ID — pass this when creating identities | | `catalog[].slug` | string | URL-safe identifier (e.g. `airbnb`, `linkedin`) — use this in path-based endpoints | | `catalog[].name` | string | Display name | | `catalog[].domain` | string | Primary domain the catalog targets | | `catalog[].category` | string | High-level grouping (e.g. `travel`, `commerce`, `social`) | | `catalog[].auth_required` | boolean | `true` if any action in this catalog requires a connected account | | `catalog[].auth_types` | string[] | Allowed credential types when `auth_required` is true. Subset of `browser_state`, `credentials`, `api_key`, `token` | | `catalog[].auth_login_url` | string \| null | Where users should log in to seed a `browser_state` credential | | `catalog[].status` | string | `active` or `pending_review` | | `catalog[].action_count` | integer | Number of visible actions in this catalog (for the requesting user) | > Optional fields (`logo_url`, `auth_type`, `auth_login_url`) are omitted when not set, not returned as `null`. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/catalog \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests response = requests.get( '{API_BASE}/catalog', headers={'X-API-Key': 'your_api_key'} ) for entry in response.json()['catalog']: print(f"{entry['slug']:20} {entry['action_count']:3} actions {entry['name']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('{API_BASE}/catalog', { headers: { 'X-API-Key': 'your_api_key' } }); const { catalog } = await response.json(); for (const entry of catalog) { console.log(`${entry.slug.padEnd(20)} ${entry.action_count} actions ${entry.name}`); } ``` </Tab> </Tabs> --- ### Rate Limit 120 requests per minute per IP. --- # GET List Identities (/docs/api-reference/wire/list-identities) <EndpointBanner method="GET" path="/identities" /> <EndpointBanner method="GET" path="/identities/{id}" /> Returns the authenticated user's identities and their attached credentials. Use the returned `credentials[].id` as the `credential_id` when submitting authenticated tasks via [POST /v1/wire/task](/docs/api-reference/wire/execute-task). An **identity** is a named container tied to one website (e.g. "Personal Amazon"). Each identity can have multiple **credentials** of different types: `browser_state`, `credentials`, `api_key`, or `token`. This page covers two endpoints: - **`GET /v1/wire/identities`** — list all your identities (optionally filtered by `catalog_id`) - **`GET /v1/wire/identities/\{id\}`** — fetch a single identity by ID (returns the same identity object, just one of them) --- ### Query Parameters (list endpoint only) | Parameter | Type | Description | |-----------|------|-------------| | `catalog_id` | string | Filter identities to a single website catalog (optional) | ### Path Parameters (single endpoint only) | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | Identity ID | --- ### Response (list) <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "identities": [ { "id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "user_id": "9f8e7d6c-5b4a-3210-9876-543210fedcba", "catalog_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "name": "Personal Amazon", "is_default": true, "created_at": "2026-04-15T10:00:00Z", "updated_at": "2026-04-20T12:34:56Z", "credentials": [ { "id": "11111111-2222-3333-4444-555555555555", "identity_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "credential_type": "browser_state", "status": "active", "expires_at": "2026-05-15T10:00:00Z", "last_used_at": "2026-04-20T12:34:56Z", "metadata": {}, "created_at": "2026-04-15T10:00:00Z", "updated_at": "2026-04-20T12:34:56Z" } ] } ] } ``` | Field | Type | Description | |-------|------|-------------| | `identities[].id` | string (UUID) | Identity ID | | `identities[].user_id` | string (UUID) | The user who owns this identity (always returned) | | `identities[].catalog_id` | string (UUID) | The catalog (website) this identity belongs to | | `identities[].name` | string | User-given name (unique per user + catalog) | | `identities[].is_default` | boolean | Marked as the user's default for this catalog | | `identities[].created_at` | string (RFC3339) | When the identity was created | | `identities[].updated_at` | string (RFC3339) | When the identity was last updated (always returned) | | `credentials[].id` | string (UUID) | **Use this as `credential_id` in task requests** | | `credentials[].credential_type` | string | One of: `browser_state`, `credentials`, `api_key`, `token` | | `credentials[].status` | string | `active` or `expired` — only `active` credentials can be used | | `credentials[].expires_at` | string (RFC3339) | When the credential's session expires (present on `credentials`-type credentials) | | `credentials[].last_used_at` | string (RFC3339) | When the credential was last used | | `credentials[].metadata` | object | Free-form metadata for the credential | | `credentials[].created_at` | string (RFC3339) | When the credential was created | | `credentials[].updated_at` | string (RFC3339) | When the credential was last updated | | `credentials[].source_id` | string (UUID) | The bound identity source. **Present only on source-backed credentials.** | | `credentials[].source_ref` | object | The bound vault locator — see [`source_ref`](/docs/api-reference/wire/login#the-source_ref-locator). **Present only on source-backed credentials.** | The encrypted credential data (`encrypted_data`) is never returned in the response. A **source-backed** credential additionally carries `source_id` and `source_ref`: ```json { "id": "11111111-2222-3333-4444-555555555555", "identity_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "credential_type": "credentials", "status": "active", "source_id": "f1e2d3c4-0000-0000-0000-000000000000", "source_ref": { "entry": { "container": "abcd1234", "key": "wxyz5678" } }, "expires_at": "2026-06-09T10:00:00Z", "last_used_at": "2026-06-08T12:34:56Z", "metadata": {}, "created_at": "2026-06-08T10:00:00Z", "updated_at": "2026-06-08T10:00:00Z" } ``` ### Response (single — `GET /identities/\{id\}`) Same shape as a single element of the `identities[]` array above, returned at the top level: ```json { "status": "ok", "identity": { "id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "user_id": "9f8e7d6c-5b4a-3210-9876-543210fedcba", "catalog_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "name": "Personal Amazon", "is_default": true, "created_at": "2026-04-15T10:00:00Z", "updated_at": "2026-04-20T12:34:56Z", "credentials": [ /* same shape */ ] } } ``` Returns `404 Not Found` (code `NOT_FOUND`) if the identity genuinely doesn't exist. If the identity exists but belongs to another user, it returns `403 Forbidden` (code `FORBIDDEN`, "Identity does not belong to user") rather than `404`. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/identities \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests response = requests.get( '{API_BASE}/identities', headers={'X-API-Key': 'your_api_key'} ) for identity in response.json()['identities']: print(f"{identity['name']}:") for cred in identity.get('credentials', []): print(f" {cred['credential_type']} → {cred['id']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('{API_BASE}/identities', { headers: { 'X-API-Key': 'your_api_key' } }); const { identities } = await response.json(); for (const identity of identities) { console.log(`${identity.name}:`); for (const cred of identity.credentials || []) { console.log(` ${cred.credential_type} → ${cred.id}`); } } ``` </Tab> </Tabs> --- ### Managing identities via UI You can also view and copy credential IDs from the [Identities dashboard](/wire/identities) — hover over a credential pill to reveal a copy button. --- # GET List Sources (/docs/api-reference/wire/list-identity-sources) <EndpointBanner method="GET" path="/identity-sources" /> Returns the authenticated user's connected **identity sources**. An identity source is a connection to a vault you own — 1Password or Azure Key Vault — that Wire reads secrets from when running authenticated actions. The secret itself never enters Wire's database: it is read into memory for a single login run and dropped. What Wire stores is a pointer to it (see [`source_ref`](/docs/api-reference/wire/login#the-source_ref-locator)). The encrypted access token is **never** returned. Each source carries a `status` of `pending`, `active`, `revoked`, or `expired`. To see which identities source their credentials from a connection, use [GET /v1/wire/identity-sources/\{id\}/identities](/docs/api-reference/wire/list-source-identities). --- ### Query Parameters Both parameters are optional and may be combined. | Parameter | Type | Description | |-----------|------|-------------| | `provider` | string | Filter by provider name — `1password` or `azure_key_vault` | | `status` | string | Filter by status — one of `pending`, `active`, `revoked`, `expired` | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "identity_sources": [ { "id": "f1e2d3c4-0000-0000-0000-000000000000", "user_id": "8a7b6c5d-0000-0000-0000-000000000000", "provider": "1password", "display_name": "Engineering vault", "config": {}, "scope_metadata": { "vaults": [ { "id": "abcd1234", "name": "Engineering" } ] }, "status": "active", "last_verified_at": "2026-06-08T10:00:00Z", "created_at": "2026-06-01T09:00:00Z", "updated_at": "2026-06-08T10:00:00Z" } ] } ``` If you have no connected sources (or none match the filters), `identity_sources` is an empty array. | Field | Type | Description | |-------|------|-------------| | `identity_sources[].id` | string (UUID) | Identity source ID | | `identity_sources[].user_id` | string (UUID) | Owner of the connection | | `identity_sources[].provider` | string | Provider name — `1password` or `azure_key_vault` | | `identity_sources[].display_name` | string | User-given label for the connection | | `identity_sources[].config` | object | Provider-specific configuration (defaults to `{}`) | | `identity_sources[].scope_metadata` | object | Scope detail — the containers this source can read, as `{ "vaults": [{ id, name }] }` | | `identity_sources[].status` | string | One of `pending`, `active`, `revoked`, `expired` | | `identity_sources[].last_verified_at` | string \| null | RFC3339 UTC — when the connection was last verified | | `identity_sources[].credentials_expires_at` | string \| null | RFC3339 UTC — when the stored token expires, or `null` if it does not | | `identity_sources[].created_at` | string | RFC3339 UTC — when the source was connected | | `identity_sources[].updated_at` | string | RFC3339 UTC — last modification | The encrypted access token is never returned in the response. > Optional fields (`last_verified_at`, `credentials_expires_at`, `scope_metadata`) are omitted when not set, rather than returned as `null`. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl "{API_BASE}/identity-sources?provider=1password&status=active" \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests response = requests.get( '{API_BASE}/identity-sources', headers={'X-API-Key': 'your_api_key'}, params={'provider': '1password', 'status': 'active'}, ) for source in response.json()['identity_sources']: print(f"{source['display_name']} ({source['provider']}) → {source['status']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const params = new URLSearchParams({ provider: '1password', status: 'active' }); const response = await fetch(`{API_BASE}/identity-sources?${params}`, { headers: { 'X-API-Key': 'your_api_key' } }); const { identity_sources } = await response.json(); for (const source of identity_sources) { console.log(`${source.display_name} (${source.provider}) → ${source.status}`); } ``` </Tab> </Tabs> --- ### Rate limit 60 requests per minute per user. ### Related - [GET /v1/wire/identity-sources/\{id\}/identities](/docs/api-reference/wire/list-source-identities) — identities using a source - [GET /v1/wire/identity-sources/\{id\}/containers](/docs/api-reference/wire/list-source-containers) — browse a source's containers - [POST /v1/wire/identity-sources](/docs/api-reference/wire/create-identity-source) — connect another vault - [Identity sources overview](/docs/api-reference/wire#identity-sources) — the whole flow, connect through bind --- # GET List Providers (/docs/api-reference/wire/list-providers) <EndpointBanner method="GET" path="/identity-sources/providers" /> Lists every vault provider this engine can connect, each describing **itself**: the fields its connect form needs, the ways it can authenticate, and the nouns its hierarchy uses. Read this before [connecting a source](/docs/api-reference/wire/create-identity-source) rather than hardcoding a provider's fields — a provider added later shows up here with no client change. Not user-specific, and it makes no upstream calls. Requires an `X-API-Key`. > **This is a static index route.** It is matched before `/identity-sources/{id}`, so `providers` is a reserved source ID that can never collide with a real one. --- ### Response <StatusBadge code={200} text="OK" /> Providers are sorted by `type`. ```json { "status": "ok", "providers": [ { "type": "1password", "display_name": "1Password", "auth_type": "credentials", "browsable": true, "summary": "Sign in using a saved vault item — no password stored in Wire.", "container_label": "Vault", "entry_label": "Login item", "methods": [ { "id": "service_account", "label": "Service Account", "instructions": "Create a Service Account in 1Password (Developer Tools → Service Accounts) with read access to the vaults you want Anakin to use, then paste its token below.", "docs_url": "https://developer.1password.com/docs/service-accounts/get-started", "fields": [ { "name": "token", "label": "Service Account Token", "type": "password", "secret": true, "placeholder": "ops_...", "pattern": "^ops_", "help": "Starts with “ops_”. Encrypted at rest; never displayed again.", "required": true } ] } ], "connect": { "fields": ["…same as methods[0].fields…"] } }, { "type": "azure_key_vault", "display_name": "Azure Key Vault", "auth_type": "credentials", "browsable": true, "summary": "Sign in using a secret from your Key Vault — no password stored in Wire.", "container_label": "Vault", "entry_label": "Secret", "methods": [ { "id": "service_principal", "label": "Service principal (client secret)", "caution": "On each Key Vault, assign the app the \"Key Vault Secrets User\" role (or a get+list access policy). This is the step most often missed — without it the connection authenticates but can't read anything.", "docs_url": "https://learn.microsoft.com/en-us/azure/key-vault/general/rbac-guide", "fields": ["…tenant_id, client_id, client_secret, vault_urls…"] } ], "connect": { "fields": ["…same as methods[0].fields…"] } } ] } ``` | Field | Type | Description | |-------|------|-------------| | `providers[].type` | string | Registry key. **Send this as `provider`** when [connecting](/docs/api-reference/wire/create-identity-source), and it's what `provider` filters on in [list sources](/docs/api-reference/wire/list-identity-sources) | | `providers[].display_name` | string | Human label, e.g. `1Password` | | `providers[].auth_type` | string | The catalog `auth_type` this provider's credentials feed. A catalog must list this in its `auth_types` for a binding to be accepted | | `providers[].browsable` | boolean | Whether [containers](/docs/api-reference/wire/list-source-containers) / [entries](/docs/api-reference/wire/list-source-entries) can be browsed. When `false`, bind by reference instead of picking | | `providers[].summary` | string | One line on what connecting this provider does. Optional | | `providers[].container_label` | string | The provider's noun for a container — `Vault` for both current providers. Defaults to `Container` | | `providers[].entry_label` | string | The provider's noun for an entry — `Login item` for 1Password, `Secret` for Azure Key Vault. Defaults to `Entry` | | `providers[].methods` | array | The ways to authenticate to this provider, each with its own field set. Providers with one way return exactly one | | `providers[].connect` | object | Mirrors `methods[0]` — present so single-method clients keep working. Prefer `methods` | **`methods[]`** | Field | Type | Description | |-------|------|-------------| | `id` | string | Stable key. **Send this as `auth_method`** when connecting. Omit `auth_method` to get `methods[0]` | | `label` | string | Shown when a provider offers a choice | | `instructions` | string | Setup guidance to render above the fields. Optional | | `caution` | string | The one step that most often goes wrong for this provider, kept out of `instructions` so a client can give it its own weight. Render it prominently — for Azure Key Vault it names the per-vault role assignment, whose omission fails *silently*: the credential authenticates and then reads nothing. Optional | | `docs_url` | string | Link to the provider's own setup guide. Optional | | `fields` | array | The inputs this method needs — see below | **`methods[].fields[]`** | Field | Type | Description | |-------|------|-------------| | `name` | string | The key to send in the connect body. Values are top-level, not nested | | `label` | string | Shown to the user | | `type` | string | `text`, `password` or `textarea` | | `secret` | boolean | Mask the input; never echoed back by any endpoint | | `placeholder` | string | Example value. Optional | | `help` | string | Hint text under the field. Optional | | `pattern` | string | Validation regex, e.g. `^ops_`. Optional | | `required` | boolean | Whether connect fails without it | > **Unknown field names are rejected, not ignored.** Sending a field the chosen method doesn't declare returns `UNKNOWN_CONNECT_FIELD` — a typo'd name fails as a typo rather than surfacing later as a confusing upstream auth error. If the engine has no providers registered, `providers` is an empty array. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/identity-sources/providers \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests response = requests.get( '{API_BASE}/identity-sources/providers', headers={'X-API-Key': 'your_api_key'}, ) for provider in response.json()['providers']: required = [f['name'] for f in provider['methods'][0]['fields'] if f['required']] print(f"{provider['type']}: needs {', '.join(required)}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch(`{API_BASE}/identity-sources/providers`, { headers: { 'X-API-Key': 'your_api_key' } }); const { providers } = await response.json(); for (const provider of providers) { const required = provider.methods[0].fields.filter((f) => f.required).map((f) => f.name); console.log(`${provider.type}: needs ${required.join(', ')}`); } ``` </Tab> </Tabs> --- ### Rate limit 60 requests per minute per user. ### Related - [POST /v1/wire/identity-sources](/docs/api-reference/wire/create-identity-source) — connect a provider using the schema returned here - [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources) — list what's already connected --- # GET List Containers (/docs/api-reference/wire/list-source-containers) <EndpointBanner method="GET" path="/identity-sources/{id}/containers" /> Lists the **containers** a connected identity source can read. A container is a provider's top-level grouping of secrets: | Provider | A container is | An entry is | |----------|----------------|-------------| | `1password` | a vault | a login item | | `azure_key_vault` | a Key Vault | a secret | This endpoint is provider-agnostic — the source's `provider` decides what gets listed, and the response shape is the same either way. Use a container's `id` as the `\{container_id\}` path segment in [GET /v1/wire/identity-sources/\{id\}/containers/\{container_id\}/entries](/docs/api-reference/wire/list-source-entries) to browse what's inside. This is a **live call** to the provider — the engine uses the source's stored credential to enumerate containers at request time, so the result always reflects the current state of the connected account. Requires an `X-API-Key`. The source must belong to the authenticated user. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | The connected source's ID. Get it from [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources) | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "containers": [ { "id": "abcd1234", "name": "Engineering" } ] } ``` | Field | Type | Description | |-------|------|-------------| | `containers[].id` | string | Container ID. **Use this as `\{container_id\}` in the [entries endpoint](/docs/api-reference/wire/list-source-entries)**, and as `container` in a `source_ref` | | `containers[].name` | string | Display name | | `containers[].item_count` | number | How many entries the container holds. Optional — omitted when the provider doesn't report it | > **Container IDs are not always opaque.** A 1Password vault ID is a short alphanumeric string, but an Azure Key Vault container ID is the vault's URL — `https://my-vault.vault.azure.net`. Percent-encode it when you put it in the entries path. --- ### Error Responses All errors return JSON of the form `{ "status": "error", "error": { "code": "...", "message": "..." } }`. | Code | HTTP | When | |------|------|------| | `SOURCE_AUTH_FAILED` | 400 | The provider rejected the stored credential — rotate or reconnect the source (it is now marked revoked) | | `SOURCE_FORBIDDEN` | 400 | The credential is valid but has no access to the container. Grant it access at the provider, then retry | | `SOURCE_INACTIVE` | 400 | The source is not active | | `SOURCE_THROTTLED` | 429 | The provider is rate-limiting us. Retry shortly | | `SOURCE_UNREACHABLE` | 502 | Could not reach the provider. If the vault restricts network access, allow our egress addresses | | `SOURCE_UPSTREAM_ERROR` | 502 | The provider failed for an unclassified reason | | `PROVIDER_NOT_SUPPORTED` | 400 | No source is registered for this provider in this engine version | | `PROVIDER_NOT_BROWSABLE` | 400 | The provider doesn't support browsing | | `SOURCE_NOT_AVAILABLE` | 503 | The source resolver isn't configured on this engine | | `NOT_FOUND` | 404 | Source not found | | `FORBIDDEN` | 403 | The source belongs to another user | > `SOURCE_AUTH_FAILED` is the only failure that revokes the source. A permission failure against one container leaves the connection healthy — every other container on it still works. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/identity-sources/f1e2d3c4-b5a6-7890-1234-56789abcdef0/containers \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests source_id = 'f1e2d3c4-b5a6-7890-1234-56789abcdef0' response = requests.get( f'{API_BASE}/identity-sources/{source_id}/containers', headers={'X-API-Key': 'your_api_key'} ) data = response.json() if data['status'] == 'ok': for container in data['containers']: print(f"{container['id']} {container['name']}") else: print(f"Error: {data['error']['code']} — {data['error']['message']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const sourceId = 'f1e2d3c4-b5a6-7890-1234-56789abcdef0'; const response = await fetch( `{API_BASE}/identity-sources/${sourceId}/containers`, { headers: { 'X-API-Key': 'your_api_key' } } ); const data = await response.json(); if (data.status === 'ok') { for (const container of data.containers) { console.log(`${container.id} ${container.name}`); } } else { console.error(`Error: ${data.error.code} — ${data.error.message}`); } ``` </Tab> </Tabs> --- ### Rate limit 30 requests per minute per user (each call makes a live request to the provider). ### Related - [GET /v1/wire/identity-sources/\{id\}/containers/\{container_id\}/entries](/docs/api-reference/wire/list-source-entries) — browse entries in a container - [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources) — list connected sources - [POST /v1/wire/login](/docs/api-reference/wire/login) — sign in using an entry from a source --- # GET List Entries (/docs/api-reference/wire/list-source-entries) <EndpointBanner method="GET" path="/identity-sources/{id}/containers/{container_id}/entries" /> Lists the **entries** inside a single container — the selectable secrets you can bind a login to. A 1Password container yields login items; an Azure Key Vault container yields secrets. The response shape is the same for every provider. This is a **live call** to the provider — the engine uses the source's stored credential to enumerate entries at request time. Use an entry's `id` as `key` in the `source_ref` when creating a source-backed login via [POST /v1/wire/login](/docs/api-reference/wire/login). Requires an `X-API-Key`. The source must belong to the authenticated user. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | The connected source's ID | | `container_id` **required** | string | A container ID from [GET /v1/wire/identity-sources/\{id\}/containers](/docs/api-reference/wire/list-source-containers) | > **Percent-encode `container_id`.** An Azure Key Vault container ID is a URL (`https://my-vault.vault.azure.net`), which has to survive as a single path segment — send it as `https%3A%2F%2Fmy-vault.vault.azure.net`. 1Password's alphanumeric vault IDs are unaffected, but encoding them is harmless, so encode unconditionally. --- ### Query Parameters | Parameter | Type | Description | |-----------|------|-------------| | `domain` | string | Case-insensitive substring. Filters to entries whose title **or** any associated URL contains it. This is what powers the catalog-domain pre-filter in the identity-creation UI. | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "entries": [ { "id": "wxyz5678", "title": "Acme Prod", "category": "Login", "urls": [ { "href": "https://acme.com/login", "label": "website" } ] } ] } ``` | Field | Type | Description | |-------|------|-------------| | `entries[].id` | string | Entry ID. **Use this as `key` in `source_ref`** when creating a [source-backed login](/docs/api-reference/wire/login) | | `entries[].title` | string | Entry title as set at the provider | | `entries[].category` | string | Provider's own classification — a 1Password category such as `Login`, or an Azure secret's content type. Optional — omitted when unknown | | `entries[].urls` | array | Websites associated with the entry. Each element is `{ href, label }`. For 1Password these are the item's own website fields; for Azure Key Vault it is the first `domain` / `url` / `site` / `host` tag found on the secret. Optional | > **Results are capped, and the cap is per provider** — 200 entries for 1Password, 500 for Azure Key Vault. A larger container is truncated silently; the response carries no "there are more" marker. Pass `domain` to narrow the list to the site you're targeting rather than relying on the cap. What `domain` matches differs by provider. 1Password matches the item title and the item's own website URLs. Azure Key Vault matches the secret name, **any** tag key or value, and — because a dotted domain can't appear in a secret name — the bare first label too, so `acme.com` finds a secret called `acme-portal`. Entry listings never contain secret values — only the metadata needed to pick one. --- ### Error Responses All errors return JSON of the form `{ "status": "error", "error": { "code": "...", "message": "..." } }`. | Code | HTTP | When | |------|------|------| | `SOURCE_AUTH_FAILED` | 400 | The provider rejected the stored credential — rotate or reconnect the source (it is now marked revoked) | | `SOURCE_FORBIDDEN` | 400 | The credential is valid but has no access to this container. Grant it access at the provider, then retry | | `SOURCE_ITEM_NOT_FOUND` | 400 | The container was not found. Check that `container_id` is percent-encoded | | `SOURCE_INACTIVE` | 400 | The source is not active | | `SOURCE_THROTTLED` | 429 | The provider is rate-limiting us. Retry shortly | | `SOURCE_UNREACHABLE` | 502 | Could not reach the provider. If the vault restricts network access, allow our egress addresses | | `SOURCE_UPSTREAM_ERROR` | 502 | The provider failed for an unclassified reason | | `PROVIDER_NOT_SUPPORTED` | 400 | No source is registered for this provider in this engine version | | `PROVIDER_NOT_BROWSABLE` | 400 | The provider doesn't support browsing | | `SOURCE_NOT_AVAILABLE` | 503 | The source resolver isn't configured on this engine | | `NOT_FOUND` | 404 | Source not found | | `FORBIDDEN` | 403 | The source belongs to another user | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # 1Password — opaque container id curl "{API_BASE}/identity-sources/f1e2d3c4-b5a6-7890-1234-56789abcdef0/containers/abcd1234/entries?domain=acme.com" \ -H "X-API-Key: your_api_key" # Azure Key Vault — container id is a URL, so it must be percent-encoded curl "{API_BASE}/identity-sources/f1e2d3c4-b5a6-7890-1234-56789abcdef0/containers/https%3A%2F%2Fmy-vault.vault.azure.net/entries?domain=acme.com" \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests from urllib.parse import quote source_id = 'f1e2d3c4-b5a6-7890-1234-56789abcdef0' container_id = 'https://my-vault.vault.azure.net' # or a 1Password vault id response = requests.get( f'{API_BASE}/identity-sources/{source_id}/containers/{quote(container_id, safe="")}/entries', headers={'X-API-Key': 'your_api_key'}, params={'domain': 'acme.com'}, ) data = response.json() if data['status'] == 'ok': for entry in data['entries']: print(f"{entry['id']} {entry['title']}") else: print(f"Error: {data['error']['code']} — {data['error']['message']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const sourceId = 'f1e2d3c4-b5a6-7890-1234-56789abcdef0'; const containerId = 'https://my-vault.vault.azure.net'; // or a 1Password vault id const params = new URLSearchParams({ domain: 'acme.com' }); const response = await fetch( `{API_BASE}/identity-sources/${sourceId}/containers/${encodeURIComponent(containerId)}/entries?${params}`, { headers: { 'X-API-Key': 'your_api_key' } } ); const data = await response.json(); if (data.status === 'ok') { for (const entry of data.entries) { console.log(`${entry.id} ${entry.title}`); } } else { console.error(`Error: ${data.error.code} — ${data.error.message}`); } ``` </Tab> </Tabs> --- ### Rate limit 30 requests per minute per user (each call makes a live request to the provider). ### Related - [GET /v1/wire/identity-sources/\{id\}/containers](/docs/api-reference/wire/list-source-containers) — list containers first - [POST /v1/wire/login](/docs/api-reference/wire/login) — sign in using the entry you picked - [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources) — list connected sources --- # GET Identities Using a Source (/docs/api-reference/wire/list-source-identities) <EndpointBanner method="GET" path="/identity-sources/{id}/identities" /> Lists the identities that currently source their credentials from a given connection, joined with the catalog (website) each identity belongs to. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | Identity source ID | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "identities": [ { "id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "name": "Prod login", "is_default": false, "catalog_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "catalog_slug": "neb", "catalog_name": "Nebula", "has_active_credential": true, "created_at": "2026-06-01T09:00:00Z" } ] } ``` | Field | Type | Description | |-------|------|-------------| | `identities[].id` | string (UUID) | Identity ID | | `identities[].name` | string | User-given name for the identity | | `identities[].is_default` | boolean | Marked as the user's default for its catalog | | `identities[].catalog_id` | string (UUID) | The catalog (website) this identity belongs to | | `identities[].catalog_slug` | string | Catalog slug, e.g. `neb` | | `identities[].catalog_name` | string | Human-readable catalog name | | `identities[].has_active_credential` | boolean | `true` = the identity still has non-expired session cookies (ready to use); `false` = it needs re-verify / login | | `identities[].created_at` | string | RFC3339 UTC — when the identity was created | If no identities source credentials from this connection, `identities` is an empty array. --- ### Error Responses All errors return JSON of the form `{ "status": "error", "error": { "code": "...", "message": "..." } }`. | Code | HTTP | When | |------|------|------| | `NOT_FOUND` | 404 | The source doesn't exist (message "Integration not found") | | `FORBIDDEN` | 403 | The source belongs to another user (message "Integration does not belong to user") | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/identity-sources/f1e2d3c4-0000-0000-0000-000000000000/identities \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests source_id = 'f1e2d3c4-0000-0000-0000-000000000000' response = requests.get( f'{API_BASE}/identity-sources/{source_id}/identities', headers={'X-API-Key': 'your_api_key'}, ) for identity in response.json()['identities']: ready = 'ready' if identity['has_active_credential'] else 'needs re-verify' print(f"{identity['name']} ({identity['catalog_name']}) → {ready}") ``` </Tab> <Tab value="JavaScript"> ```javascript const sourceId = 'f1e2d3c4-0000-0000-0000-000000000000'; const response = await fetch(`{API_BASE}/identity-sources/${sourceId}/identities`, { headers: { 'X-API-Key': 'your_api_key' } }); const { identities } = await response.json(); for (const identity of identities) { const ready = identity.has_active_credential ? 'ready' : 'needs re-verify'; console.log(`${identity.name} (${identity.catalog_name}) → ${ready}`); } ``` </Tab> </Tabs> --- ### Rate limit 60 requests per minute per user. ### Related - [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources) — list connected sources - [GET /v1/wire/identities](/docs/api-reference/wire/list-identities) — list all your identities and their credentials - [Identity sources overview](/docs/api-reference/wire#identity-sources) — the whole flow, connect through bind --- # POST Create Identity / Login (/docs/api-reference/wire/login) <EndpointBanner method="POST" path="/login" /> Create-and-sign-in in a single call: this endpoint **finds-or-creates** an identity for the catalog and signs it in, returning a `credential_id` you can immediately use with [POST /v1/wire/task](/docs/api-reference/wire/execute-task). There are two ways to supply the login: - **Classic `params`** — pass the catalog's username/password (and any other login fields) directly. - **Vault locator** — pass a `source_id` + `source_ref` pointing at an entry in a connected [identity source](/docs/api-reference/wire/list-identity-sources), and Wire reads the login fields from your own vault at sign-in time. No password is typed into Wire. The locator is provider-agnostic: 1Password and Azure Key Vault take the same shape. The endpoint: 1. Resolves the catalog by slug 2. Finds-or-creates an identity for you (auto-named from `params` if you don't supply `identity_name`; an explicit `identity_name` is **required** when using a locator) 3. Resolves the login fields — from `params`, or from the vault entry referenced by `source_id` + `source_ref` 4. Runs the catalog's login wheel against those fields 5. Persists a cookies-only credential (the password is never stored) 6. Returns everything needed to call authenticated actions > **The password is discarded after a successful sign-in.** Only the issued cookies are stored, encrypted. When the cookies expire (see `expires_at`), call `/login` again to mint fresh ones. For browser-based catalogs (`auth_types` includes `browser_state` but **not** `credentials`), use the [interactive connect flow](/wire/identities) instead. --- ### Request Body ```json { "catalog_slug": "neb", "identity_name": "work-account", "params": { "email": "alice@example.com", "password": "..." } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `catalog_slug` **required** | string | Catalog slug, e.g. `neb`. Find it in [GET /v1/wire/catalog](/docs/api-reference/wire/list-catalogs). | | `identity_name` | string | Optional label for the identity. If omitted, derived from `params` (priority: `email` > `username` > `user` > `login` > `account_id` > `account`, falling back to `"default"`). Same name re-uses the same identity row across calls. | | `params` | object | Wheel-defined login fields. Shape varies per catalog — discover it by reading `login_input_schema` from [GET /v1/wire/catalog/\{slug\}](/docs/api-reference/wire/get-catalog). Required unless you supply a `source_id` + `source_ref` locator. | | `source_id` | string (UUID) | Optional. A connected identity source from [GET /v1/wire/identity-sources](/docs/api-reference/wire/list-identity-sources). When paired with `source_ref`, the engine resolves the login fields from your vault instead of `params`. | | `source_ref` | object | Optional. Where in the vault the login fields live — see [The `source_ref` locator](#the-source_ref-locator) below. Used together with `source_id` as an alternative to `params`. | > **`identity_name` is required when using the locator.** With a locator there are no `params` to auto-derive a name from, so you must pass an explicit `identity_name`. You also cannot send both `params` and a locator — pick one. A locator request body looks like: ```json { "catalog_slug": "neb", "identity_name": "work-account", "source_id": "f1e2d3c4-0000-0000-0000-000000000000", "source_ref": { "entry": { "container": "abcd1234", "key": "wxyz5678" } } } ``` --- ### The `source_ref` locator `source_ref` says **where each login field lives**; the catalog's login action schema says **what the wheel needs**. Wire joins the two — you never state which vault field maps to which login parameter. **(a) One entry supplies every field.** The common case, and what the dashboard's identity picker emits: ```json { "entry": { "container": "abcd1234", "key": "wxyz5678" } } ``` `container` is a container ID from [list containers](/docs/api-reference/wire/list-source-containers); `key` is an entry ID from [list entries](/docs/api-reference/wire/list-source-entries). **(b) Per-field refs.** For secrets kept apart — a username and password stored as two separate Key Vault secrets: ```json { "fields": { "username": { "container": "https://my-vault.vault.azure.net", "key": "acme-user" }, "password": { "container": "https://my-vault.vault.azure.net", "key": "acme-pass" } } } ``` **(c) Both.** `fields` overrides specific fields and inherits `container` / `key` / `version` from `entry` wherever they're omitted — e.g. an Azure secret holding the password, with the username carried in a tag: ```json { "entry": { "container": "https://my-vault.vault.azure.net", "key": "acme" }, "fields": { "username": { "property": "tag:username" } } } ``` | Field | Type | Description | |-------|------|-------------| | `container` | string | The container holding the entry | | `key` | string | The entry within the container. **Required on `entry`**, and on a `fields` ref when there is no `entry` to inherit one from | | `property` | string | A named field inside the entry — a 1Password field name, or `tag:<name>` for an Azure secret tag. Defaults to the whole entry | | `version` | string | A specific version of the entry. Omit for "latest" — which is what makes a password rotated in your vault take effect on the next `/login` with no change here | > **Wire matches the vault's field names to the login action's parameter names**, falling back to a small alias table (`username` ↔ `user` / `login` / `email`, `password` ↔ `pass` / `secret`, and so on). If a required login field can't be resolved, the call fails with `SOURCE_FIELD_MISSING` and names the fields, so you fix them in one pass. Each parameter the action declares is resolved in this order: 1. An explicit per-field ref in `fields` — inheriting `container`, `key` and `version` from `entry` wherever they're omitted 2. An exact name match inside the entry 3. An alias match, per the table above 4. The entry's own `property`, applied last so it can never shadow a real name match A distinct entry is read **once** per sign-in however many parameters it supplies, so pointing four fields at one item costs one round trip to your vault. > **`username` and `email` travel together.** When one of the pair resolves and the other doesn't, Wire fills in the missing one with the same value — so a login action declaring only `email` still receives `username`, and vice versa. Passwords are never duplicated, and this is the only pair. > > It exists for wheels whose published schema is narrower than what they actually read: before this endpoint resolved fields from a schema, every sourced login was sent `username`, `email` and `password` regardless, and an action declaring `email` while indexing `username` worked precisely because both were always present. Expect to see the extra key when inspecting what a login received. The pre-canonical 1Password shape — `{ "vault_id": "...", "item_id": "...", "fields": ["username","password"] }` — is still accepted, and credentials bound with it keep working. New integrations should use the shape above. --- ### Discovering the params shape Different catalogs require different fields. Always read the schema first: ```bash curl {API_BASE}/catalog/neb \ -H "X-API-Key: your_api_key" ``` The response includes `login_input_schema` — an array of `{name, type, required, description}` field descriptors. Send exactly those fields in `params`. --- ### Response <StatusBadge code={201} text="Created" /> ```json { "status": "verified", "identity_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "identity_name": "alice@example.com", "credential_id": "11111111-2222-3333-4444-555555555555", "catalog_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "catalog_slug": "neb", "expires_at": "2026-05-29T08:09:23Z" } ``` | Field | Type | Description | |-------|------|-------------| | `status` | string | Always `"verified"` on success | | `identity_id` | string (UUID) | The identity this credential is attached to. Stable across re-runs with the same name. | | `identity_name` | string | The name used (explicit or derived) | | `credential_id` | string (UUID) | **Pass this as `credential_id` in subsequent [POST /v1/wire/task](/docs/api-reference/wire/execute-task) calls.** | | `catalog_id` | string (UUID) | The catalog's UUID — saves you another lookup | | `catalog_slug` | string | Echoed back | | `expires_at` | string | RFC3339 UTC. When the session expires. Capped at 30 days; defaults to 24h if the wheel doesn't supply one. Re-call `/login` to refresh. | --- ### Idempotent re-runs Calling `/login` again with the same `identity_name` (or the same derived name) re-uses the existing identity row and refreshes its credential in place — no duplicate rows accumulate. Pass an explicit `identity_name` if you want multiple identities for the same catalog under one account (e.g. work + personal). --- ### Error Responses All errors return JSON of the form `{ "status": "error", "error": { "code": "...", "message": "...", "missing_fields"?: [...] } }`. The wheel's raw error text is never echoed; only the typed `code` is surfaced. **Validation / setup** — request rejected before the wheel runs: | Code | HTTP | When | |------|------|------| | `INVALID_BODY` | 400 | Request body isn't valid JSON | | `INVALID_INPUT` | 400 | `catalog_slug` missing; OR neither `params` nor `source_id`+`source_ref` supplied; OR both supplied; OR locator used without `identity_name` | | `INVALID_PARAMS` | 400 | One or more `required: true` fields from `login_input_schema` are missing, null, or blank. Response includes `missing_fields: [...]`. The wheel is **not** invoked — failures here cost nothing. | | `CATALOG_NOT_FOUND` | 400 | No catalog matches `catalog_slug` | | `IDENTITY_BINDING_MISMATCH` | 409 | The named identity already exists bound to a different source/credential than the request | | `LOGIN_NOT_SUPPORTED` | 400 | The catalog's `auth_types` doesn't include `credentials`, OR the catalog has no published login action | | `LOGIN_NOT_AVAILABLE` | 503 | Credentials-mode login isn't configured on this engine (rare; infrastructure issue) | | `LOGIN_TRANSPORT_ERROR` | 502 | Could not reach the wheel runner | **Locator-only** — the binding was rejected before the wheel ran: | Code | HTTP | When | |------|------|------| | `INVALID_INTEGRATION_REF` | 400 | `source_ref` isn't a JSON object. (Supplying only one half of the locator is reported as `INVALID_INPUT` above, not this) | | `SOURCE_NOT_FOUND` | 400 | The referenced `source_id` doesn't exist | | `FORBIDDEN` | 403 | The referenced source belongs to another user | | `SOURCE_INACTIVE` | 400 | The referenced source is not active — reconnect it | | `PROVIDER_NOT_SUPPORTED_BY_CATALOG` | 400 | The catalog doesn't list this source's provider in `supported_sources`. See [GET /v1/wire/catalog/\{slug\}](/docs/api-reference/wire/get-catalog) | | `CATALOG_MISSING_REQUIRED_AUTH_TYPE` | 400 | The catalog has no login wheel that can consume this provider's credentials | | `PROVIDER_CREDENTIAL_TYPE_MISMATCH` | 400 | The provider resolves to a different credential type than this request creates | | `SOURCE_FIELD_MISSING` | 400 | The entry can't supply a required login field. Response includes `missing_fields: [...]` | | `SOURCE_AUTH_FAILED` | 400 | The provider rejected the stored credential — rotate or reconnect the source | | `SOURCE_FORBIDDEN` | 400 | The source has no access to that container. Grant it access at the provider, then retry | | `SOURCE_ITEM_NOT_FOUND` | 400 | The referenced container or entry no longer exists | | `SOURCE_THROTTLED` | 429 | The provider is rate-limiting us. Retry shortly | | `SOURCE_UNREACHABLE` | 502 | Could not reach the provider | | `SOURCE_NOT_AVAILABLE` | 503 | Source resolver not configured on this engine | **Wheel ran** (HTTP 200, `status: "error"`) — the request was well-formed and the wheel ran, but sign-in didn't complete: | Code | HTTP | When | |------|------|------| | `BAD_PASSWORD` | 200 | Sign-in rejected by the site — wrong credentials | | `MFA_REQUIRED` | 200 | Account requires multi-factor auth (not yet supported) | | `CAPTCHA_REQUIRED` | 200 | Site is showing a captcha challenge | | `ACCOUNT_LOCKED` | 200 | Site reports the account is locked | | `LOGIN_TIMEOUT` | 200 | Sign-in took too long | | `LOGIN_PAGE_CHANGED` | 200 | Site's login flow may have changed | | `LOGIN_NO_COOKIES` | 200 | Sign-in completed but no session cookie was issued | | `LOGIN_INFRASTRUCTURE_ERROR` | 200 | Transient Browser Connect / upstream issue. Retry. | | `LOGIN_FAILED` | 200 | Generic fallback — sign-in failed for an unclassified reason | > **HTTP 200 with `status: "error"`** is used for "the request was well-formed and the wheel ran, but the sign-in itself didn't complete." Treat any `status != "verified"` as failure regardless of HTTP code. Example `INVALID_PARAMS` response: ```json { "status": "error", "error": { "code": "INVALID_PARAMS", "message": "Missing required login fields: password", "missing_fields": ["password"] } } ``` --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # (a) classic — username/password in params curl {API_BASE}/login \ -X POST \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "catalog_slug": "neb", "params": { "email": "alice@example.com", "password": "your_password" } }' # (b) vault locator — fields resolved from your own vault; identity_name required curl {API_BASE}/login \ -X POST \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "catalog_slug": "neb", "identity_name": "work-account", "source_id": "f1e2d3c4-0000-0000-0000-000000000000", "source_ref": { "entry": { "container": "abcd1234", "key": "wxyz5678" } } }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( '{API_BASE}/login', headers={'X-API-Key': 'your_api_key'}, json={ 'catalog_slug': 'neb', 'params': { 'email': 'alice@example.com', 'password': 'your_password', }, }, ) data = response.json() if data['status'] == 'verified': credential_id = data['credential_id'] # Use credential_id with /v1/wire/task else: print(f"Login failed: {data['error']['code']} — {data['error']['message']}") if data['error']['code'] == 'INVALID_PARAMS': print(f"Missing: {data['error']['missing_fields']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('{API_BASE}/login', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ catalog_slug: 'neb', params: { email: 'alice@example.com', password: 'your_password', }, }), }); const data = await response.json(); if (data.status === 'verified') { const credentialId = data.credential_id; // Use credentialId with /v1/wire/task } else { console.error(`Login failed: ${data.error.code} — ${data.error.message}`); if (data.error.code === 'INVALID_PARAMS') { console.error('Missing:', data.error.missing_fields); } } ``` </Tab> </Tabs> --- ### Rate limit 10 requests per minute per user (each call triggers a real sign-in run against the target site). ### Related - [GET /v1/wire/catalog/\{slug\}](/docs/api-reference/wire/get-catalog) — discover `login_input_schema` - [GET /v1/wire/identities](/docs/api-reference/wire/list-identities) — list identities + credentials - [GET /v1/wire/identity-sources/\{id\}/containers](/docs/api-reference/wire/list-source-containers) — list containers for the locator - [GET /v1/wire/identity-sources/\{id\}/containers/\{container_id\}/entries](/docs/api-reference/wire/list-source-entries) — list entries for the locator - [POST /v1/wire/task](/docs/api-reference/wire/execute-task) — use the returned `credential_id` to run an authenticated action - [POST /v1/wire/identity-sources](/docs/api-reference/wire/create-identity-source) — connect a vault to bind against --- # Provider Setup (/docs/api-reference/wire/provider-setup) Before you can [connect a source](/docs/api-reference/wire/create-identity-source), you have to create a credential **at the provider** that grants Wire read access to the vaults you choose. This page is the walkthrough for each supported provider. Neither provider requires giving Wire your account password. Both use a scoped, revocable machine credential that you can withdraw at any time without touching your own login. You never send a vault secret to Wire. You send the credential that lets Wire *read* one, and Wire reads it at sign-in time only, into memory, for one login run. --- ## 1Password 1Password is zero-knowledge, so there is no "give us your password" path. Access is granted through a **Service Account token**, which begins with `ops_`. 1. In 1Password, go to **Developer Tools → Service Accounts** and create one. 2. Grant it **read** access to the vaults you want Wire to use — and only those. 3. Copy the token. 1Password shows it once. 4. [Connect it](/docs/api-reference/wire/create-identity-source) as `provider: "1password"` with the token in the `token` field. That's the whole setup. Wire lists the vaults the token can read during the connect check, so a successful connect immediately tells you what it reaches. To revoke Wire's access later, delete the Service Account in 1Password — or [disconnect the source](/docs/api-reference/wire/delete-identity-source), which deletes the stored token on our side. **Reference:** [1Password Service Accounts](https://developer.1password.com/docs/service-accounts/get-started) --- ## Azure Key Vault Azure needs more setup than 1Password, because access is granted per vault rather than per token. ### 1. Create an app registration In the Azure portal, go to **Microsoft Entra ID → App registrations → New registration**. Give it a name; the defaults are fine for everything else. From the app's **Overview** page, note two values you'll need: - **Application (client) ID** → send as `client_id` - **Directory (tenant) ID** → send as `tenant_id` ### 2. Create a client secret Under **Certificates & secrets → Client secrets → New client secret**, create one and copy its **Value** — not its Secret ID. Azure shows the value once. This is the `client_secret` field. > **Secrets expire.** Azure client secrets carry an expiry (commonly 6, 12 or 24 months) and Entra doesn't report it through the token exchange, so Wire can't warn you in advance — `credentials_expires_at` stays empty for Azure sources. Diary the expiry yourself, and when it comes, [rotate](/docs/api-reference/wire/update-identity-source) rather than reconnecting: rotation keeps the source ID, so every identity bound to it keeps working. ### 3. Grant the app access to each vault **This is the step most setups miss.** Without it the connection authenticates successfully and reads nothing. For each Key Vault you want Wire to use, open it and go to **Access control (IAM) → Add role assignment**, then assign your app registration the **Key Vault Secrets User** role. If the vault still uses the older access-policy model instead of RBAC, add an access policy granting **Get** and **List** on secrets. Role assignments take a few moments to propagate. A connect that fails immediately after granting the role usually succeeds on a retry a minute later. ### 4. Collect the vault URLs Azure has no API for "which vaults may this principal read", so Wire can't discover them — you list them explicitly. Each vault's URL is on its Overview page, in the form: ``` https://<vault-name>.vault.azure.net ``` Send them in `vault_urls`, one per line. Wire verifies **every** one during connect, so a missing role assignment fails the connect naming the offending vault rather than surfacing later as a mysterious login failure. Only Key Vault addresses are accepted — the host must end in `.vault.azure.net`, `.vault.usgovcloudapi.net` or `.vault.azure.cn`. Anything else is rejected with `INVALID_VAULT_URL`. ### 5. Connect ```bash curl {API_BASE}/identity-sources \ -X POST \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "provider": "azure_key_vault", "display_name": "Prod Key Vault", "tenant_id": "00000000-0000-0000-0000-000000000000", "client_id": "00000000-0000-0000-0000-000000000000", "client_secret": "your_client_secret_value", "vault_urls": "https://my-vault.vault.azure.net\nhttps://other-vault.vault.azure.net" }' ``` **Reference:** [Azure Key Vault RBAC guide](https://learn.microsoft.com/en-us/azure/key-vault/general/rbac-guide) --- ### Storing login fields in Key Vault A Key Vault secret holds a single opaque value, whereas a login usually needs at least a username and a password. There are two ways to bridge that, and both are expressed in the [`source_ref` locator](/docs/api-reference/wire/login#the-source_ref-locator) when you sign in. **Two secrets.** Store the username and password as separate secrets and point at each: ```json { "fields": { "username": { "container": "https://my-vault.vault.azure.net", "key": "acme-user" }, "password": { "container": "https://my-vault.vault.azure.net", "key": "acme-pass" } } } ``` **One secret, username in a tag.** Store the password as the secret's value and add a tag named `username` to it. Reference the tag with the `tag:` prefix, inheriting the rest from `entry`: ```json { "entry": { "container": "https://my-vault.vault.azure.net", "key": "acme" }, "fields": { "username": { "property": "tag:username" } } } ``` Omit `version` and Wire always reads the current version — so rotating a password in Key Vault takes effect on the next login with no change on our side. --- ### Troubleshooting | Symptom | Cause | Fix | |---------|-------|-----| | Connect fails with `SOURCE_FORBIDDEN` naming a vault | The app authenticated but has no role on that vault | Assign **Key Vault Secrets User** on the named vault, then retry | | Connect fails with `SOURCE_TOKEN_REJECTED` | Wrong `tenant_id` / `client_id`, or the client secret's **ID** was sent instead of its **Value** | Re-copy the secret Value from **Certificates & secrets** | | Connect fails with `INVALID_VAULT_URL` | Malformed URL, or no Key Vault exists at it | Copy the URL from the vault's Overview page | | Connected fine, but browsing a container returns "not found" | The container ID is a URL and wasn't percent-encoded into the path | See [List Entries](/docs/api-reference/wire/list-source-entries) — encode it | | Worked before, now `SOURCE_AUTH_FAILED` everywhere | The client secret expired or was deleted | [Rotate](/docs/api-reference/wire/update-identity-source) with a new secret; bindings survive | | Login fails with `SOURCE_FIELD_MISSING` | The secret can't supply a field the login needs | Check `missing_fields` in the response, then add the field or use per-field refs above | A working source that later starts failing is worth a [verify](/docs/api-reference/wire/verify-identity-source) first — it distinguishes a dead credential from a vault whose role assignment was removed, which look identical from a failed login. --- ### Related - [GET /v1/wire/identity-sources/providers](/docs/api-reference/wire/list-providers) — the exact fields each provider expects, straight from the engine - [POST /v1/wire/identity-sources](/docs/api-reference/wire/create-identity-source) — connect once setup is done - [POST /v1/wire/login](/docs/api-reference/wire/login) — sign in using an entry from the connected vault --- # GET Resolve Actions (/docs/api-reference/wire/search-actions) <EndpointBanner method="GET" path="/resolve" /> Find Wire actions across every catalog in a single request. Useful when you know what you want to do (e.g. "search listings", "fetch profile") but don't yet know which catalog or `action_id` to call. > `GET /v1/wire/resolve` is the canonical name for this endpoint; `GET /v1/wire/search` is an equivalent, still-live alias. Both accept the same parameters and return the same shape. Discovery is public — read-only lookups work **with no key** (see [Zero Touch](/docs/integrations/ai-agents/zero-touch)); the `connected` flag and account-scoped results require an `X-API-Key`. --- ### Query Parameters | Parameter | Type | Description | |-----------|------|-------------| | `q` | string | Free-text query — matches against action name, description, and tags | | `catalog` | string | Restrict results to a single catalog slug (e.g. `airbnb`) | | `category` | string | Restrict results to a catalog category (e.g. `travel`, `commerce`) | | `auth` | string | **Legacy.** Set to `false` to exclude actions that require authentication. Superseded by `auth_mode` | | `auth_mode` | string | Filter by auth mode: `none`, `optional`, or `required`. Takes precedence over the legacy `auth` filter when both are supplied | All filters are optional and combine with AND semantics. Omitting every filter returns all visible actions. --- ### Response <StatusBadge code={200} text="OK" /> ```json { "results": [ { "action_id": "ab_search_listings", "catalog_name": "Airbnb", "catalog_slug": "airbnb", "name": "Search Listings", "description": "Search Airbnb listings by query, dates, and guest count.", "mode": "async", "auth_mode": "none", "auth_required": false, "connected": false, "params": [ { "name": "query", "type": "string", "required": true }, { "name": "checkin", "type": "string", "required": false }, { "name": "checkout", "type": "string", "required": false }, { "name": "adults", "type": "integer", "required": false, "default": 1 } ], "credits": 1 } ] } ``` | Field | Type | Description | |-------|------|-------------| | `results[].action_id` | string | Pass this as `action_id` to [POST /v1/wire/task](/docs/api-reference/wire/execute-task) | | `results[].catalog_name` | string | Display name of the parent catalog | | `results[].catalog_slug` | string | Slug of the parent catalog — use with [GET /v1/wire/catalog/\{slug\}](/docs/api-reference/wire/get-catalog) for full details | | `results[].mode` | string | `async` or `sync` | | `results[].auth_mode` | string | `"none"` \| `"optional"` \| `"required"`. See [execute-task](/docs/api-reference/wire/execute-task) for semantics | | `results[].auth_required` | boolean | **Deprecated** legacy mirror — true iff `auth_mode === "required"` | | `results[].connected` | boolean | `true` if you already have an active credential for this catalog | | `results[].params` | array | Array of param descriptors — `{ name, type, required, default?, description? }` — one per input the action accepts | | `results[].credits` | integer | Base credit cost per call | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # Search across all catalogs curl "{API_BASE}/resolve?q=listings" \ -H "X-API-Key: your_api_key" # Restrict to one catalog and exclude auth-required actions curl "{API_BASE}/resolve?catalog=airbnb&auth=false" \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests response = requests.get( '{API_BASE}/resolve', params={'q': 'listings', 'auth': 'false'}, headers={'X-API-Key': 'your_api_key'}, ) for r in response.json()['results']: print(f"{r['catalog_slug']:15} {r['action_id']:30} {r['credits']} cr {r['name']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const url = new URL('{API_BASE}/resolve'); url.searchParams.set('q', 'listings'); url.searchParams.set('auth', 'false'); const response = await fetch(url, { headers: { 'X-API-Key': 'your_api_key' } }); const { results } = await response.json(); for (const r of results) { console.log(`${r.catalog_slug.padEnd(15)} ${r.action_id.padEnd(30)} ${r.credits} cr ${r.name}`); } ``` </Tab> </Tabs> --- ### Rate Limit 120 requests per minute per IP. --- # PATCH Rename or Rotate Source (/docs/api-reference/wire/update-identity-source) <EndpointBanner method="PATCH" path="/identity-sources/{id}" /> Renames a connected source, rotates its stored credential, or both in one call. **Rotation preserves the row.** The source ID never changes, so every identity bound to it keeps working. Disconnecting and reconnecting instead would null those bindings and force you to re-bind each identity by hand — this endpoint exists so a routine secret rotation isn't that. A replacement credential is re-verified by exactly the code that admitted the original, and only replaces the working one on success. **A rejected rotation leaves the existing credential untouched** — a fat-fingered token can't brick a live connection. Requires an `X-API-Key`; the source must belong to the authenticated user. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | Identity source ID | --- ### Request Body Send `display_name`, provider credential fields, or both. Sending neither is an error. | Parameter | Type | Description | |-----------|------|-------------| | `display_name` | string | New label. Cannot be blank — omit the key entirely to leave the name alone | | `auth_method` | string | Which method to rotate with. Defaults to the method the source was connected with (`config.auth_method`) | | *provider fields* | string | The full field set for the chosen method, same as [connect](/docs/api-reference/wire/create-identity-source). Present fields trigger a rotation | Rename only: ```json { "display_name": "Engineering vault (prod)" } ``` Rotate a 1Password token: ```json { "token": "ops_new_service_account_token" } ``` Rotate an Azure client secret — the whole method field set, not just the changed value: ```json { "tenant_id": "00000000-0000-0000-0000-000000000000", "client_id": "00000000-0000-0000-0000-000000000000", "client_secret": "the_new_secret", "vault_urls": "https://my-vault.vault.azure.net" } ``` > **Rotation re-activates a revoked source.** If the source flipped to `revoked` because the provider rejected the old credential, a successful rotation returns it to `active` with its identity bindings intact. --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "identity_source": { "id": "f1e2d3c4-0000-0000-0000-000000000000", "user_id": "8a7b6c5d-0000-0000-0000-000000000000", "provider": "1password", "display_name": "Engineering vault (prod)", "config": { "auth_method": "service_account" }, "scope_metadata": { "vaults": [ { "id": "abcd1234", "name": "Engineering" } ] }, "status": "active", "last_verified_at": "2026-06-09T11:00:00Z", "created_at": "2026-06-01T09:00:00Z", "updated_at": "2026-06-09T11:00:00Z" } } ``` The refreshed row, same shape as [GET /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/get-identity-source). A rotation also refreshes `scope_metadata` and bumps `last_verified_at`, since the new credential was just proven against the provider. --- ### Error Responses | Code | HTTP | When | |------|------|------| | `INVALID_BODY` | 400 | Body isn't a JSON object, or a field's value isn't a string | | `INVALID_INPUT` | 400 | Neither `display_name` nor credential fields supplied, or `display_name` is blank | | `NOT_FOUND` | 404 | No source with that ID | | `FORBIDDEN` | 403 | The source belongs to another user | | `UNSUPPORTED_PROVIDER` | 400 | This engine has no source registered for the row's provider | | `UNSUPPORTED_AUTH_METHOD` | 400 | The provider doesn't offer that `auth_method` | | `UNKNOWN_CONNECT_FIELD` | 400 | A field name the chosen method doesn't declare | | `INVALID_TOKEN_FORMAT` | 400 | (1Password) The replacement token doesn't start with `ops_` | | `INVALID_VAULT_URL` | 400 | (Azure) A `vault_urls` entry is malformed, or no Key Vault exists at it | | `SOURCE_TOKEN_REJECTED` | 400 | The provider rejected the replacement credential | | `SOURCE_FORBIDDEN` | 400 | The replacement authenticates but can't read a requested vault | | `SOURCE_THROTTLED` | 429 | The provider is rate-limiting us. Retry shortly | | `SOURCE_VERIFY_FAILED` | 400 | Could not reach the provider to verify. Retry in a moment | | `SOURCE_NOT_AVAILABLE` | 503 | The source resolver isn't configured on this engine | Every rotation failure above is non-destructive — the stored credential is unchanged and the source keeps its previous status. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # rename only curl {API_BASE}/identity-sources/f1e2d3c4-0000-0000-0000-000000000000 \ -X PATCH \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "display_name": "Engineering vault (prod)" }' # rotate the credential curl {API_BASE}/identity-sources/f1e2d3c4-0000-0000-0000-000000000000 \ -X PATCH \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "token": "ops_new_service_account_token" }' ``` </Tab> <Tab value="Python"> ```python import requests source_id = 'f1e2d3c4-0000-0000-0000-000000000000' response = requests.patch( f'{API_BASE}/identity-sources/{source_id}', headers={'X-API-Key': 'your_api_key'}, json={'token': 'ops_new_service_account_token'}, ) data = response.json() if data['status'] == 'ok': print(f"Rotated — status is now {data['identity_source']['status']}") else: # The old credential is still in place. print(f"Rotation rejected: {data['error']['code']} — {data['error']['message']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const sourceId = 'f1e2d3c4-0000-0000-0000-000000000000'; const response = await fetch(`{API_BASE}/identity-sources/${sourceId}`, { method: 'PATCH', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ token: 'ops_new_service_account_token' }), }); const data = await response.json(); if (data.status === 'ok') { console.log(`Rotated — status is now ${data.identity_source.status}`); } else { // The old credential is still in place. console.error(`Rotation rejected: ${data.error.code} — ${data.error.message}`); } ``` </Tab> </Tabs> --- ### Rate limit 20 requests per minute per user. ### Related - [GET /v1/wire/identity-sources/providers](/docs/api-reference/wire/list-providers) — the field set for each auth method - [POST /v1/wire/identity-sources/\{id\}/verify](/docs/api-reference/wire/verify-identity-source) — check the current credential without replacing it - [DELETE /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/delete-identity-source) — disconnect instead --- # POST Verify Identity / Re-Login (/docs/api-reference/wire/verify-credential) <EndpointBanner method="POST" path="/credentials/verify" /> Re-verify and refresh the saved login for an **existing** identity. Wire re-runs the catalog's login wheel and refreshes the stored session cookies in place. Works for all identity types: - **Vault-sourced credential** — omit `params` (or send `{}`). The engine re-reads the login fields from the bound vault entry. - **Manually-entered credential** (Plaid-style, no bound source) — send the catalog's login `params`. There is nothing to re-resolve, so the fields are required. This is the same endpoint the dashboard **Re-verify** action uses, for both kinds of credential. > **The password is never stored.** Only the issued session cookies are persisted, encrypted. When the cookies expire (see `expires_at`), call this endpoint again to mint fresh ones. It behaves like [POST /v1/wire/login](/docs/api-reference/wire/login), but targets a known `identity_id` instead of finding-or-creating an identity. --- ### Request Body ```json { "identity_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "params": { "username": "alice@example.com", "password": "..." } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `identity_id` **required** | string (UUID) | The identity whose credential should be re-verified. | | `params` | object | The catalog's login fields (same shape as [POST /login](/docs/api-reference/wire/login)). **Omit or send `{}`** for a vault-sourced credential — the engine re-reads the fields from the bound source. **Required** when the credential has no bound source (manual credentials must send `params`). | --- ### Response <StatusBadge code={201} text="Created" /> ```json { "status": "verified", "credential": { "id": "11111111-2222-3333-4444-555555555555", "identity_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "credential_type": "credentials", "status": "active", "source_id": "f1e2d3c4-0000-0000-0000-000000000000", "source_ref": { "entry": { "container": "abcd1234", "key": "wxyz5678" } }, "expires_at": "2026-06-09T10:00:00Z", "metadata": {}, "created_at": "2026-06-08T10:00:00Z", "updated_at": "2026-06-08T10:00:00Z" } } ``` | Field | Type | Description | |-------|------|-------------| | `status` | string | Always `"verified"` on success | | `credential.id` | string (UUID) | **Pass this as `credential_id` in subsequent [POST /v1/wire/task](/docs/api-reference/wire/execute-task) calls.** | | `credential.identity_id` | string (UUID) | The identity this credential belongs to | | `credential.credential_type` | string | Credential kind, e.g. `credentials` | | `credential.status` | string | Credential status — `active` after a successful re-verify | | `credential.source_id` | string (UUID) | The bound identity source. **Present only for sourced credentials** (omitted for manual). | | `credential.source_ref` | object | The bound vault locator — see [`source_ref`](/docs/api-reference/wire/login#the-source_ref-locator). **Present only for sourced credentials.** | | `credential.expires_at` | string | RFC3339 UTC. When the refreshed session expires. | | `credential.metadata` | object | Free-form metadata for the credential (always present). | | `credential.created_at` | string | RFC3339 UTC. When the credential was created. | | `credential.updated_at` | string | RFC3339 UTC. When the credential was last updated (always present). | > `source_id` and `source_ref` appear only for vault-sourced credentials; they are omitted for manually-entered ones. --- ### Error Responses All errors return JSON of the form `{ "status": "error", "error": { "code": "...", "message": "..." } }`. The wheel's raw error text is never echoed; only the typed `code` is surfaced. **Validation / setup** — request rejected before the wheel runs: | Code | HTTP | When | |------|------|------| | `INVALID_BODY` | 400 | Request body isn't valid JSON | | `INVALID_INPUT` | 400 | `identity_id` missing; OR `params` empty **and** no source is bound to this credential (manual credentials must send `params`) | | `NOT_FOUND` | 404 | Identity not found | | `FORBIDDEN` | 403 | Identity belongs to another user | | `LOGIN_NOT_SUPPORTED` | 400 | The catalog doesn't support credentials login, or has no published login action | | `LOGIN_NOT_AVAILABLE` | 503 | Credentials-mode login isn't configured on this engine | | `LOGIN_TRANSPORT_ERROR` | 502 | Could not reach the wheel runner | **Source resolution** — only when re-reading a vault-bound credential, before the wheel runs: | Code | HTTP | When | |------|------|------| | `SOURCE_NOT_FOUND` | 400 | The credential's bound source was deleted — reconnect it | | `SOURCE_AUTH_FAILED` | 400 | The provider rejected the stored credential — rotate or reconnect the source in the dashboard | | `SOURCE_FORBIDDEN` | 400 | The source has no access to that container. Grant it access at the provider, then retry | | `SOURCE_ITEM_NOT_FOUND` | 400 | The bound container or entry no longer exists | | `SOURCE_FIELD_MISSING` | 400 | The entry can't supply a required login field. Response includes `missing_fields: [...]` | | `SOURCE_INACTIVE` | 400 | The bound source is not active | | `SOURCE_THROTTLED` | 429 | The provider is rate-limiting us. Retry shortly | | `SOURCE_UNREACHABLE` | 502 | Could not reach the provider | | `SOURCE_NOT_AVAILABLE` | 503 | Source resolver not configured on this engine | **Wheel ran (HTTP 200, `status: "error"`)** — the request was well-formed and the wheel ran, but sign-in didn't complete. Treat any `status != "verified"` as failure. | Code | HTTP | When | |------|------|------| | `BAD_PASSWORD` | 200 | Sign-in rejected by the site — wrong credentials | | `MFA_REQUIRED` | 200 | Account requires multi-factor auth (not yet supported) | | `CAPTCHA_REQUIRED` | 200 | Site is showing a captcha challenge | | `ACCOUNT_LOCKED` | 200 | Site reports the account is locked | | `LOGIN_TIMEOUT` | 200 | Sign-in took too long | | `LOGIN_PAGE_CHANGED` | 200 | Site's login flow may have changed | | `LOGIN_NO_COOKIES` | 200 | Sign-in completed but no session cookie was issued | | `LOGIN_INFRASTRUCTURE_ERROR` | 200 | Transient Browser Connect / upstream issue — retry (same as `/login`) | | `LOGIN_FAILED` | 200 | Generic fallback — sign-in failed for an unclassified reason | > **HTTP 200 with `status: "error"`** is used for "the request was well-formed and the wheel ran, but the sign-in itself didn't complete." Treat any `status != "verified"` as failure regardless of HTTP code. --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash # (a) vault-sourced credential — no params; fields re-read from the bound entry curl {API_BASE}/credentials/verify \ -X POST \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "identity_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c" }' # (b) manual credential — send the catalog's login params curl {API_BASE}/credentials/verify \ -X POST \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "identity_id": "7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c", "params": { "username": "alice@example.com", "password": "your_password" } }' ``` </Tab> <Tab value="Python"> ```python import requests response = requests.post( '{API_BASE}/credentials/verify', headers={'X-API-Key': 'your_api_key'}, json={ 'identity_id': '7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c', 'params': { 'username': 'alice@example.com', 'password': 'your_password', }, }, ) data = response.json() if data['status'] == 'verified': credential_id = data['credential']['id'] # Use credential_id with /v1/wire/task else: print(f"Verify failed: {data['error']['code']} — {data['error']['message']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const response = await fetch('{API_BASE}/credentials/verify', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json', }, body: JSON.stringify({ identity_id: '7c3f1a2b-4d5e-6f7a-8b9c-0d1e2f3a4b5c', params: { username: 'alice@example.com', password: 'your_password', }, }), }); const data = await response.json(); if (data.status === 'verified') { const credentialId = data.credential.id; // Use credentialId with /v1/wire/task } else { console.error(`Verify failed: ${data.error.code} — ${data.error.message}`); } ``` </Tab> </Tabs> --- ### Rate limit 10 requests per minute per user (each call runs a real sign-in against the target site). ### Related - [GET /v1/wire/identities](/docs/api-reference/wire/list-identities) — list identities + credentials - [POST /v1/wire/login](/docs/api-reference/wire/login) — single-call find-or-create credentials login - [POST /v1/wire/login](/docs/api-reference/wire/login) — Create Identity / Login (password or vault locator) - [POST /v1/wire/task](/docs/api-reference/wire/execute-task) — use the returned `credential.id` to run an authenticated action - [Identity sources overview](/docs/api-reference/wire#identity-sources) — the whole flow, connect through bind --- # POST Verify Source (/docs/api-reference/wire/verify-identity-source) <EndpointBanner method="POST" path="/identity-sources/{id}/verify" /> Re-runs the provider's connection check against the **stored** credential — a health check, not a rotation. Nothing is replaced; use [PATCH](/docs/api-reference/wire/update-identity-source) for that. Use it to confirm a source still works before relying on it, or to clear a `revoked` status after fixing the problem at the provider. No request body. Requires an `X-API-Key`; the source must belong to the authenticated user. --- ### What the result does to the source | Outcome | Effect on the row | |---------|-------------------| | Success | `last_verified_at` is bumped; a previously `revoked` source returns to `active` | | Credential rejected (`SOURCE_AUTH_FAILED`) | Status flips to `revoked` — the credential itself is dead | | Permission failure (`SOURCE_FORBIDDEN`) | Status is **unchanged** — the credential is healthy, it just lacks a grant on one vault | > That distinction is the whole point of this endpoint. Only a rejected credential revokes a source; a missing role assignment on one vault leaves every other vault on the connection working, and is fixed at the provider without issuing a new secret. For Azure Key Vault, verification succeeds if **any** scoped vault is readable. One revoked role assignment shouldn't mark a connection that still works for its other vaults as broken — per-vault failures surface at bind time instead, where they can name the vault. --- ### Path Parameters | Parameter | Type | Description | |-----------|------|-------------| | `id` **required** | string (UUID) | Identity source ID | --- ### Response <StatusBadge code={200} text="OK" /> ```json { "status": "ok", "identity_source": { "id": "f1e2d3c4-0000-0000-0000-000000000000", "user_id": "8a7b6c5d-0000-0000-0000-000000000000", "provider": "azure_key_vault", "display_name": "Prod Key Vault", "config": { "auth_method": "service_principal" }, "scope_metadata": { "vaults": [ { "id": "https://my-vault.vault.azure.net", "name": "my-vault" } ] }, "status": "active", "last_verified_at": "2026-06-09T11:30:00Z", "created_at": "2026-06-01T09:00:00Z", "updated_at": "2026-06-09T11:30:00Z" } } ``` The refreshed row, with `last_verified_at` freshly stamped. If that bump fails to persist, the response is still `ok` — the check itself passed, which is what you asked. --- ### Error Responses | Code | HTTP | When | |------|------|------| | `NOT_FOUND` | 404 | No source with that ID | | `FORBIDDEN` | 403 | The source belongs to another user | | `SOURCE_AUTH_FAILED` | 400 | The provider rejected the stored credential. **The source is now `revoked`** — rotate or reconnect | | `SOURCE_FORBIDDEN` | 400 | The credential is valid but lacks access. Grant it at the provider, then verify again | | `SOURCE_ITEM_NOT_FOUND` | 400 | A scoped vault no longer exists | | `SOURCE_INACTIVE` | 400 | The source is not active | | `SOURCE_THROTTLED` | 429 | The provider is rate-limiting us. Retry shortly | | `SOURCE_UNREACHABLE` | 502 | Could not reach the provider. If the vault restricts network access, allow our egress addresses | | `SOURCE_UPSTREAM_ERROR` | 502 | The provider failed for an unclassified reason | | `SOURCE_NOT_AVAILABLE` | 503 | The source resolver isn't configured on this engine | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl {API_BASE}/identity-sources/f1e2d3c4-0000-0000-0000-000000000000/verify \ -X POST \ -H "X-API-Key: your_api_key" ``` </Tab> <Tab value="Python"> ```python import requests source_id = 'f1e2d3c4-0000-0000-0000-000000000000' response = requests.post( f'{API_BASE}/identity-sources/{source_id}/verify', headers={'X-API-Key': 'your_api_key'}, ) data = response.json() if data['status'] == 'ok': print(f"Healthy — verified at {data['identity_source']['last_verified_at']}") elif data['error']['code'] == 'SOURCE_FORBIDDEN': print('Credential is fine; grant it access to the vault at the provider.') else: print(f"Unhealthy: {data['error']['code']} — {data['error']['message']}") ``` </Tab> <Tab value="JavaScript"> ```javascript const sourceId = 'f1e2d3c4-0000-0000-0000-000000000000'; const response = await fetch(`{API_BASE}/identity-sources/${sourceId}/verify`, { method: 'POST', headers: { 'X-API-Key': 'your_api_key' }, }); const data = await response.json(); if (data.status === 'ok') { console.log(`Healthy — verified at ${data.identity_source.last_verified_at}`); } else if (data.error.code === 'SOURCE_FORBIDDEN') { console.warn('Credential is fine; grant it access to the vault at the provider.'); } else { console.error(`Unhealthy: ${data.error.code} — ${data.error.message}`); } ``` </Tab> </Tabs> --- ### Rate limit 10 requests per minute per user (each call makes a live request to the provider). ### Related - [PATCH /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/update-identity-source) — rotate the credential a failed verify points at - [GET /v1/wire/identity-sources/\{id\}](/docs/api-reference/wire/get-identity-source) — current status and dependent-identity count - [POST /v1/wire/credentials/verify](/docs/api-reference/wire/verify-credential) — re-verify a *login*, not the source behind it --- # Overview (/docs/documentation) Scrape any website, extract structured data with AI, and search the web — all through a simple REST API. ### Products <CardGrid> <ProductCard title="Wire" description="Pre-built actions for popular websites" href="/docs/api-reference/wire" /> <ProductCard title="URL Scraper" description="Scrape single or batch URLs with caching and AI extraction" href="/docs/api-reference/url-scraper" /> <ProductCard title="Search API" description="AI-powered web search with citations (synchronous)" href="/docs/api-reference/search" /> <ProductCard title="Agentic Search" description="Multi-stage automated research pipeline" href="/docs/api-reference/agentic-search" /> </CardGrid> ### Key features - **Zero blocks** — anti-detection and proxy routing across [207 countries and territories](/docs/api-reference/supported-countries) - **Async job pattern** — submit a job, poll for results when ready - **AI extraction** — structured JSON from any page with `generateJson: true` - **Headless browser** — JS-heavy sites and SPAs with `useBrowser: true` - **Intelligent caching** — 30x faster on repeat requests ### Use cases <CardGrid> <ProductCard title="AI & RAG Pipelines" description="Feed web data into LLMs and AI assistants" href="/docs/documentation/use-cases/ai-rag" /> <ProductCard title="Deep Research" description="Multi-source automated research pipelines" href="/docs/documentation/use-cases/deep-research" /> <ProductCard title="Price Monitoring" description="Track pricing and product changes" href="/docs/documentation/use-cases/price-monitoring" /> <ProductCard title="Lead Generation" description="Extract leads from company websites" href="/docs/documentation/use-cases/lead-generation" /> </CardGrid> See all [use cases](/docs/documentation/use-cases) for more examples. Get started with the [Quick Start](/docs/documentation/getting-started) guide. --- # AI Visibility (/docs/documentation/ai-visibility) AI Visibility answers a simple question: **what do the major AI engines say when asked the same thing?** Send one prompt and get every engine's answer back — ChatGPT, Gemini, Google AI Overview — side by side, with measured latency and credit cost per source. Typical uses: - **Brand & content teams** — see how AI engines describe your product, your category, or your competitors - **Research** — triangulate a question across engines instead of trusting one - **SEO / AEO** — understand what Google's AI Overview surfaces for the queries you care about ### How it works 1. **Submit a prompt** from the dashboard (AI Visibility under Tools) or via the [API](/docs/api-reference/ai-visibility). Pick which sources to query — all enabled engines by default — and the **country** to search from (default US), just like the URL scraper's geo targeting. 2. **Answers arrive live.** Each engine is executed through Wire's catalogue; results land independently, so a fast engine's answer shows while slower ones are still running. 3. **Compare.** Each source card shows the extracted answer, the raw response, measured latency, and credit cost — downloadable per source or as a zip. 4. **Keep or ship the results.** Runs are retained for 7 days in Recents, exportable per source (markdown) or as a zip, and every settled run can fire a signed [webhook](/docs/api-reference/webhooks) (`ai.search.completed` / `ai.search.failed`). Webhook deliveries carry short per-source previews, not full answers — receivers fetch the complete results from the event's `result_url`. ### Credits Each source that completes bills its Wire action's per-call rate; failed sources cost nothing. Costs are shown per source and per run. Results are never cached — every run queries the engines fresh, so repeated prompts bill each time. ### Good to know - Searches run server-side — closing the tab doesn't stop a run; it finishes and waits in Recents. - A failed source can be retried individually without re-running the others. - Wire runs made by AI Visibility are tagged with an **AI Visibility** badge in Wire's Run History. - Answers are region-dependent — Google may not serve an AI Overview at all for some query/country combinations. Results reflect what a user in the selected country would see. Ready to integrate? Start with the [API reference](/docs/api-reference/ai-visibility). --- # Overview (/docs/documentation/getting-started) Get your API key and scrape your first page — choose the path that fits your workflow. --- ## 1. Get your API key Sign up at the [Dashboard](/dashboard) — it's free, no credit card required. You start with **300 credits**. Copy your API key from the dashboard. It starts with `ak-`. --- ## 2. Choose your path <CardGrid> <ProductCard title="Quick Start: CLI" description="Install the CLI and scrape from your terminal in 2 minutes. Best for trying things out fast." href="/docs/documentation/getting-started/cli" /> </CardGrid> ### Pick your language Native HTTP client examples — minimal or zero dependencies, copy-pasteable. <CardGrid> <ProductCard title="cURL" description="Talk to the API directly from a shell script or CI pipeline. No language runtime needed." href="/docs/documentation/getting-started/curl" /> <ProductCard title="Python" description="The requests library. Drop-in for Django, FastAPI, Celery, and notebook workflows." href="/docs/documentation/getting-started/python" /> <ProductCard title="Node.js" description="Built-in fetch, no npm install. Works with Next.js, Express, NestJS, and TypeScript." href="/docs/documentation/getting-started/node" /> <ProductCard title="Go" description="Standard library net/http. Ideal for backends and CLI tools." href="/docs/documentation/getting-started/go" /> <ProductCard title="Ruby" description="Stdlib net/http + json. Drop-in for Rails service objects and Sidekiq jobs." href="/docs/documentation/getting-started/ruby" /> <ProductCard title="PHP" description="Built-in cURL extension. Works in Laravel, Symfony, and plain PHP 8+." href="/docs/documentation/getting-started/php" /> <ProductCard title="Java" description="Java 11+ HttpClient with Jackson. Plugs into Spring Boot, Quarkus, Micronaut." href="/docs/documentation/getting-started/java" /> <ProductCard title="Rust" description="Lightweight ureq crate, no async runtime needed." href="/docs/documentation/getting-started/rust" /> <ProductCard title="Elixir" description="Req — modern Elixir HTTP client, bundled with Phoenix 1.7+." href="/docs/documentation/getting-started/elixir" /> <ProductCard title=".NET" description="Built-in HttpClient and System.Text.Json. Works with .NET 6+ and ASP.NET Core." href="/docs/documentation/getting-started/dotnet" /> </CardGrid> --- ## Products | I want to... | Product | |---|---| | Run pre-built website actions | [Wire](/docs/api-reference/wire) | | Extract content from a URL | [URL Scraper](/docs/api-reference/url-scraper) | | Scrape multiple URLs at once | [URL Scraper (batch)](/docs/api-reference/url-scraper/batch-url-scraping) | | Search the web with AI | [Search API](/docs/api-reference/search) | | Deep multi-source research | [Agentic Search](/docs/api-reference/agentic-search) | | Scrape login-protected pages | [Browser Sessions](/docs/api-reference/browser-sessions) | See [Pricing & Credits](/docs/documentation/pricing) for costs per operation. --- ## Quick reference | | | |---|---| | **Base URL** | `https://api.anakin.io/v1` | | **Auth header** | `X-API-Key: ak-your-key-here` | | **Free credits** | 300 on signup | | **Rate limits** | 60/min per scraping/search endpoint, 20/min Wire tasks, 10/min AI eval — [details](/docs/documentation/rate-limits) | | **Failed jobs** | Not charged — credits deducted only on success | {/* sync-flow validation 2026-04-27 — safe to remove */} --- # Pricing & Credits (/docs/documentation/pricing) ## Per-Request Pricing AnakinScraper uses a simple credit-based system. Each API request consumes credits based on the type of operation performed. | Request Type | Credits | Description | |--------------|---------|-------------| | Wire action | Varies | Credit cost shown per action in the catalog — deducted upfront, refunded on failure | | URL scrape | 1 credit | HTML extraction with optional JS rendering | | + AI Summary | +1 credit | AI-generated summary of the page content | | + AI JSON extraction | +2 credits | AI-structured data extraction | | Batch scraping | 1 credit × URLs | 1 credit per URL in the batch (add-ons apply per URL) | | Crawl | 1 credit × pages | Multi-page crawl, charged upfront based on max pages | | Map (URL discovery) | 1 credit | Discover all URLs on a domain | | Search API | 3 credits | Web search with full content extraction | | Agentic Search | 10 + 1/URL | 10 base + 1 credit per URL scraped during research | | Browser API | 1 credit / 2 min | Live CDP browser session, billed per 2-minute interval (rounded up) | ## Plans ### Starter (Free) - 300 credits - Basic scraping - API access ### Pro ($9/month) - 3,000 credits/month - Priority support - Advanced features - 99.9% uptime SLA ### Scale ($29/month) - 12,000 credits/month - Priority support - All API endpoints - 99.9% uptime SLA ### Enterprise (Custom) - Unlimited credits - Dedicated support - Custom integrations - SLA guarantees [Contact sales](https://calendly.com/d/ctqw-64s-rgt/let-s-talk-about-your-use-case-anakin-io) for Enterprise pricing. ## Credit Usage Examples ### Basic HTML Scrape ```bash curl -X POST https://api.anakin.io/v1/scrape \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` **Cost: 1 credit** ### URL Scrape with JS Rendering ```bash curl -X POST https://api.anakin.io/v1/scrape \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com", "useBrowser": true}' ``` **Cost: 1 credit** (same as basic — JS rendering has no additional cost) ### Batch Scraping (10 URLs) ```bash curl -X POST https://api.anakin.io/v1/scrape/batch \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"urls": ["url1", "url2", ...], "useBrowser": true}' ``` **Cost: 10 credits** (1 credit × 10 URLs) ### URL Scrape with AI Summary ```bash curl -X POST https://api.anakin.io/v1/scrape \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com", "formats": ["markdown", "summary"]}' ``` **Cost: 2 credits** (1 base + 1 for AI summary) ### URL Scrape with AI JSON Extraction ```bash curl -X POST https://api.anakin.io/v1/scrape \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com", "formats": ["markdown", "json"]}' ``` **Cost: 3 credits** (1 base + 2 for AI JSON extraction) ### Crawl (10 pages) ```bash curl -X POST https://api.anakin.io/v1/crawl \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com", "maxPages": 10}' ``` **Cost: 10 credits** (1 credit × 10 pages) ### Map (URL Discovery) ```bash curl -X POST https://api.anakin.io/v1/map \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` **Cost: 1 credit** --- # Rate Limits (/docs/documentation/rate-limits) AnakinScraper applies rate limits per API key to ensure reliable performance for all users. Limits are enforced using a sliding window algorithm — the window counts requests within the last N seconds and rejects once the limit is reached. --- ## Limits by endpoint Each row below represents an **independent bucket** — hitting the limit on one endpoint does not affect your quota on another. Limits are scoped either per API key (per-user) or per IP, as noted in the **Bucket** column. #### Wire — discovery (no auth required) | Endpoint | Rate limit | Bucket | |----------|-----------|--------| | `GET /v1/wire/catalog` | 60 requests/min | per-IP | | `GET /v1/wire/catalog/{slug}` | 60 requests/min | per-IP | | `GET /v1/wire/search` | 30 requests/min | per-IP | #### Wire — authenticated | Endpoint | Rate limit | Bucket | |----------|-----------|--------| | `POST /v1/wire/task` | 20 requests/min | per-user | | `GET /v1/wire/jobs` | 60 requests/min | per-user | | `GET /v1/wire/jobs/{id}` | 60 requests/min | per-user | | `GET /v1/wire/jobs/{id}/download` | 30 requests/min | per-user | > **Wire polling is rate-limited.** Unlike URL Scraper / Search polling — which have no rate limit — `GET /v1/wire/jobs/{id}` is capped at **60 requests/min per user**. Treat it as roughly one poll per second per job and use exponential backoff if you're polling many jobs in parallel. #### Scraping | Endpoint | Rate limit | Bucket | |----------|-----------|--------| | `POST /v1/url-scraper` | 60 requests/min | per-user | | `POST /v1/url-scraper/batch` | 60 requests/min | per-user | | `POST /v1/map` | 60 requests/min | per-user | | `POST /v1/crawl` | 60 requests/min | per-user | #### Search | Endpoint | Rate limit | Bucket | |----------|-----------|--------| | `POST /v1/search` | 60 requests/min | per-user | | `POST /v1/agentic-search` | 60 requests/min | per-user | #### Browser Sessions | Endpoint | Rate limit | Bucket | |----------|-----------|--------| | `POST /v1/sessions/manual-start` | 60 requests/min | per-user | | `POST /v1/sessions/manual-save` | 60 requests/min | per-user | | `PATCH /v1/sessions/{id}` | 60 requests/min | per-user | | `DELETE /v1/sessions/{id}` | 60 requests/min | per-user | #### AI Evaluation | Endpoint | Rate limit | Bucket | |----------|-----------|--------| | `POST /v1/ai/evaluate` | 10 requests/min | per-user | | `POST /v1/ai/evaluate/stream` | 10 requests/min | per-user | #### Endpoints with no rate limit The following GET endpoints are not rate-limited — you can poll them as often as needed: - `GET /v1/url-scraper/{id}` - `GET /v1/agentic-search/{id}` - `GET /v1/map/{id}` - `GET /v1/crawl/{id}` --- ## Rate limit response When you exceed a rate limit, the API returns a `429 Too Many Requests` response: ```json { "error": "rate_limit_exceeded", "message": "Too many requests. Please try again later." } ``` See [Error Responses](/docs/api-reference/error-responses) for the full error format and the canonical retry pattern with exponential backoff and jitter. --- ## Handling rate limits ### Retry with exponential backoff The recommended approach is to wait and retry with exponential backoff. Start with a short delay and double it on each retry. <Tabs items={["Python", "JavaScript"]}> <Tab value="Python"> ```python import requests import time def scrape_with_retry(url, api_key, max_retries=3): """Submit a scrape job with automatic retry on rate limit.""" delay = 2 for attempt in range(max_retries + 1): response = requests.post( "https://api.anakin.io/v1/url-scraper", headers={"X-API-Key": api_key}, json={"url": url} ) if response.status_code == 429: if attempt == max_retries: raise Exception("Rate limit exceeded after retries") print(f"Rate limited, retrying in {delay}s...") time.sleep(delay) delay *= 2 continue response.raise_for_status() return response.json() result = scrape_with_retry("https://example.com", "ak-your-key-here") print(result["jobId"]) ``` </Tab> <Tab value="JavaScript"> ```javascript async function scrapeWithRetry(url, apiKey, maxRetries = 3) { let delay = 2000; for (let attempt = 0; attempt <= maxRetries; attempt++) { const response = await fetch("https://api.anakin.io/v1/url-scraper", { method: "POST", headers: { "X-API-Key": apiKey, "Content-Type": "application/json" }, body: JSON.stringify({ url }) }); if (response.status === 429) { if (attempt === maxRetries) { throw new Error("Rate limit exceeded after retries"); } console.log(`Rate limited, retrying in ${delay / 1000}s...`); await new Promise(r => setTimeout(r, delay)); delay *= 2; continue; } if (!response.ok) throw new Error(`HTTP ${response.status}`); return await response.json(); } } const result = await scrapeWithRetry("https://example.com", "ak-your-key-here"); console.log(result.jobId); ``` </Tab> </Tabs> ### Use batch endpoints If you're scraping multiple URLs, use the [batch endpoint](/docs/api-reference/url-scraper/batch-url-scraping) instead of submitting individual requests. A single batch request can include up to 10 URLs and only counts as one request against the rate limit. ```bash # Bad: 10 requests, 10 against rate limit for url in url1 url2 ... url10; do curl -X POST .../v1/url-scraper -d "{\"url\": \"$url\"}" done # Good: 1 request, 1 against rate limit curl -X POST https://api.anakin.io/v1/url-scraper/batch \ -H "X-API-Key: ak-your-key-here" \ -H "Content-Type: application/json" \ -d '{"urls": ["url1", "url2", "...", "url10"]}' ``` ### Spread requests over time If you have a large list of URLs, pace your submissions rather than sending them all at once. A simple approach is to add a short delay between requests: ```python import time urls = ["https://example.com/1", "https://example.com/2", ...] for url in urls: result = scrape_with_retry(url, api_key) job_ids.append(result["jobId"]) time.sleep(1) # ~60 requests/min stays within the limit ``` --- ## Tips - **Rate limits apply to submit endpoints only.** Poll as often as you like — GET endpoints for checking job status are not rate-limited. - **Batch when possible.** A single batch request with 10 URLs uses 1 rate-limit slot, not 10. - **Cache results.** AnakinScraper caches responses for 24 hours. Repeat requests for the same URL return instantly and cost zero credits, but they still count against rate limits. - **Use the CLI for simple workloads.** The [Anakin CLI](/docs/sdks/cli) handles rate limiting and retries automatically. --- ## Increasing your limits If you need higher rate limits for your use case, contact us: - **Email** — support@anakin.io - **Enterprise plan** — includes custom rate limits. [Talk to sales](https://calendly.com/d/ctqw-64s-rgt/let-s-talk-about-your-use-case-anakin-io). --- # Use Cases (/docs/documentation/use-cases) Explore how different teams leverage AnakinScraper to power their AI applications, data pipelines, and business workflows. <CardGrid> <ProductCard title="AI & Agent Data Ingestion" description="Feed web data into LLMs, AI agents, and retrieval pipelines." href="/docs/documentation/use-cases/ai-agent-data-ingestion" /> <ProductCard title="Competitive Intelligence & Pricing" description="Monitor competitor websites, pricing strategies, and detect changes in real-time." href="/docs/documentation/use-cases/competitive-intel" /> <ProductCard title="Structured Market & Web Research" description="Run structured multi-source research and extract insights from the web." href="/docs/documentation/use-cases/market-research" /> <ProductCard title="Financial & Corporate Filings" description="Extract structured data from financial reports, SEC filings, and corporate documents." href="/docs/documentation/use-cases/financial-filings" /> <ProductCard title="Lead Generation" description="Extract and enrich leads from company websites and directories." href="/docs/documentation/use-cases/lead-generation" /> <ProductCard title="SEO & Search Intelligence" description="Monitor search rankings, audit content, and track online presence." href="/docs/documentation/use-cases/seo-marketing" /> <ProductCard title="Review & Sentiment Data Extraction" description="Collect reviews, ratings, and sentiment signals from across the web." href="/docs/documentation/use-cases/review-sentiment" /> <ProductCard title="ML & Training Data Collection" description="Build large-scale datasets from the web for model training and fine-tuning." href="/docs/documentation/use-cases/ml-training-data" /> <ProductCard title="Browser Automation & Web Workflows" description="Automate complex browser interactions and multi-step web workflows." href="/docs/documentation/use-cases/browser-automation" /> <ProductCard title="Content Aggregation" description="Collect articles, news, and content from multiple sources." href="/docs/documentation/use-cases/content-aggregation" /> </CardGrid> --- # CLI (/docs/documentation/getting-started/cli) The CLI is the fastest way to use AnakinScraper. It handles job submission, polling, and output formatting for you. --- ## Install the CLI Requires Python 3.10+. <Tabs items={["pip", "pipx"]}> <Tab value="pip"> ```bash pip install anakin-cli ``` </Tab> <Tab value="pipx"> ```bash pipx install anakin-cli ``` </Tab> </Tabs> --- ## Authenticate Save your API key (you only need to do this once): ```bash anakin login --api-key "ak-your-key-here" ``` Verify it worked: ```bash anakin status ``` --- ## Scrape your first page ```bash anakin scrape "https://example.com" ``` The CLI submits the job, polls until it's done, and prints the markdown output to your terminal. That's it. **Save to a file:** ```bash anakin scrape "https://example.com" -o page.md ``` --- ## Go further ### Extract structured JSON with AI ```bash anakin scrape "https://news.ycombinator.com" --format json ``` AI automatically extracts structured data from the page — no schema needed. ### Scrape JavaScript-heavy sites ```bash anakin scrape "https://example.com/spa" --browser ``` Enables full browser rendering for SPAs and dynamic content. Only use when needed — standard scraping is faster. ### Batch scrape multiple URLs ```bash anakin scrape-batch "https://a.com" "https://b.com" "https://c.com" ``` Scrape up to 10 URLs in parallel with a single command. ### Search the web with AI ```bash anakin search "best web scraping libraries 2025" ``` Returns AI-powered search results instantly. ### Deep research ```bash anakin research "comparison of web frameworks 2025" -o report.json ``` Runs a multi-stage AI research pipeline across 20+ sources. Takes 1–5 minutes. --- ## Next steps <CardGrid> <ProductCard title="CLI Commands" description="Full reference for all CLI commands, flags, and options" href="/docs/sdks/cli/commands" /> <ProductCard title="CLI Examples" description="Real-world recipes: piping, scripting, and batch workflows" href="/docs/sdks/cli/examples" /> <ProductCard title="API Reference" description="Full endpoint docs if you want to use the API directly" href="/docs/api-reference" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # cURL (/docs/documentation/getting-started/curl) Hit the AnakinScraper REST API directly with `curl`. Useful for quick tests, shell scripts, CI pipelines, or any environment without a language runtime. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install `curl` is preinstalled on macOS, Linux, and most CI runners. The polling script below also uses `jq` for JSON parsing — install it once if you don't have it: ```bash # macOS brew install jq # Debian / Ubuntu sudo apt-get install jq ``` --- ## Submit a single request Fire-and-forget submit, useful when you'll poll separately or check results later in the dashboard: ```bash curl -X POST https://api.anakin.io/v1/url-scraper \ -H "X-API-Key: $ANAKIN_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` Response: ```json { "jobId": "job_abc123xyz", "status": "pending" } ``` Poll the job: ```bash curl https://api.anakin.io/v1/url-scraper/job_abc123xyz \ -H "X-API-Key: $ANAKIN_API_KEY" ``` --- ## Submit + poll in one script Save as `scrape.sh` (`chmod +x scrape.sh`): ```bash #!/bin/bash set -e : "${ANAKIN_API_KEY:?ANAKIN_API_KEY is not set}" BASE="https://api.anakin.io/v1" URL="${1:-https://example.com}" submitted=$(curl -sS -X POST "$BASE/url-scraper" \ -H "X-API-Key: $ANAKIN_API_KEY" \ -H "Content-Type: application/json" \ -d "{\"url\": \"$URL\"}") job_id=$(echo "$submitted" | jq -r '.jobId') for _ in $(seq 1 60); do job=$(curl -sS "$BASE/url-scraper/$job_id" -H "X-API-Key: $ANAKIN_API_KEY") \ || { sleep 3; continue; } # retry transient errors status=$(echo "$job" | jq -r '.status') case "$status" in completed) echo "$job" | jq -r '.markdown'; exit 0 ;; failed) echo "scrape failed: $(echo "$job" | jq -r '.error')" >&2; exit 1 ;; esac sleep 3 done echo "timed out after 3 minutes" >&2 exit 1 ``` Run it: ```bash ./scrape.sh https://example.com ``` --- ## What this does 1. Submits the URL to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient `curl` errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```bash curl -sS -X POST "$BASE/url-scraper" \ -H "X-API-Key: $ANAKIN_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://news.ycombinator.com", "generateJson": true}' ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```bash curl -sS -X POST "$BASE/url-scraper" \ -H "X-API-Key: $ANAKIN_API_KEY" \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com/spa", "useBrowser": true}' ``` > Only use browser mode when needed — standard scraping is faster and cheaper. ### Search the web with AI The Search API is **synchronous** — no polling needed: ```bash curl -sS -X POST https://api.anakin.io/v1/search \ -H "X-API-Key: $ANAKIN_API_KEY" \ -H "Content-Type: application/json" \ -d '{"prompt": "best web scraping libraries 2025"}' ``` --- ## Use it from CI Drop the polling script into a GitHub Actions step or a cron job. The `ANAKIN_API_KEY` should come from a secret store (GitHub Secrets, Vault, Doppler, etc.) — never hard-code it: ```yaml # .github/workflows/scrape.yml - name: Scrape pricing page env: ANAKIN_API_KEY: ${{ secrets.ANAKIN_API_KEY }} run: ./scrape.sh https://example.com/pricing > pricing.md ``` --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # .NET (/docs/documentation/getting-started/dotnet) Submit a scrape, poll for the result, and handle transient errors — using the standard library's `HttpClient` and `System.Text.Json`. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install No NuGet packages needed — `HttpClient`, `System.Net.Http.Json`, and `System.Text.Json` are all in the .NET 6+ standard library. Create a console project: ```bash dotnet new console -o quickstart cd quickstart ``` --- ## Scrape a page Replace `Program.cs` with: ```csharp using System.Net.Http.Json; using System.Text.Json.Nodes; const string Base = "https://api.anakin.io/v1"; var apiKey = Environment.GetEnvironmentVariable("ANAKIN_API_KEY") ?? throw new InvalidOperationException("ANAKIN_API_KEY is not set"); using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(30) }; http.DefaultRequestHeaders.Add("X-API-Key", apiKey); async Task<JsonNode?> Request(HttpMethod method, string path, object? body = null) { var req = new HttpRequestMessage(method, Base + path); if (body != null) req.Content = JsonContent.Create(body); try { var resp = await http.SendAsync(req); return await resp.Content.ReadFromJsonAsync<JsonNode>(); } catch (HttpRequestException) { return null; } // caller retries on null } var submitted = await Request(HttpMethod.Post, "/url-scraper", new { url = "https://example.com" }); var jobId = submitted!["jobId"]!.ToString(); for (int i = 0; i < 60; i++) { var job = await Request(HttpMethod.Get, $"/url-scraper/{jobId}"); if (job == null) { await Task.Delay(3000); // retry transient errors continue; } var status = job["status"]!.ToString(); if (status == "completed") { Console.WriteLine(job["markdown"]); return; } if (status == "failed") { throw new Exception($"scrape failed: {job["error"]}"); } await Task.Delay(3000); } throw new TimeoutException("timed out after 3 minutes"); ``` Run it: ```bash dotnet run ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient `HttpRequestException` errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```csharp var submitted = await Request(HttpMethod.Post, "/url-scraper", new { url = "https://news.ycombinator.com", generateJson = true }); ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```csharp var submitted = await Request(HttpMethod.Post, "/url-scraper", new { url = "https://example.com/spa", useBrowser = true }); ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Use it from ASP.NET Core Register a typed `HttpClient` via `IHttpClientFactory` to get connection pooling and DI for free, then call from a hosted background service (`IHostedService`) or a Hangfire/Quartz job — the polling loop awaits up to 3 minutes per URL, so background execution is the natural fit. ```csharp // Program.cs builder.Services.AddHttpClient("anakin", c => { c.BaseAddress = new Uri("https://api.anakin.io/v1"); c.DefaultRequestHeaders.Add("X-API-Key", builder.Configuration["ANAKIN_API_KEY"]); c.Timeout = TimeSpan.FromSeconds(30); }); ``` --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # Elixir (/docs/documentation/getting-started/elixir) Submit a scrape, poll for the result, and handle transient errors — using `Req`, the modern Elixir HTTP client (already bundled with Phoenix 1.7+). --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `x-api-key` header. --- ## Install `Req` is the de facto modern HTTP client in Elixir — JSON encode/decode, retry, and connection pooling all built in. Phoenix 1.7+ already bundles it; standalone projects need one dep: Add to `mix.exs`: ```elixir defp deps do [ {:req, "~> 0.5"} ] end ``` Then: ```bash mix deps.get ``` --- ## Scrape a page Save as `lib/quickstart.ex`: ```elixir defmodule Quickstart do @base "https://api.anakin.io/v1" defp api_key do System.get_env("ANAKIN_API_KEY") || raise "ANAKIN_API_KEY is not set" end defp request(method, path, body \\ nil) do Req.request( method: method, url: @base <> path, headers: [{"x-api-key", api_key()}, {"content-type", "application/json"}], json: body, receive_timeout: 30_000 ) end def scrape(url) do {:ok, %{body: submitted}} = request(:post, "/url-scraper", %{url: url}) job_id = submitted["jobId"] Enum.reduce_while(1..60, nil, fn _, _ -> case request(:get, "/url-scraper/#{job_id}") do {:ok, %{body: %{"status" => "completed"} = job}} -> {:halt, job} {:ok, %{body: %{"status" => "failed", "error" => err}}} -> raise "scrape failed: #{err}" _ -> Process.sleep(3_000) # retry transient errors {:cont, nil} end end) |> case do nil -> raise "timed out after 3 minutes" job -> job end end end job = Quickstart.scrape("https://example.com") IO.puts(job["markdown"]) ``` Run it: ```bash mix run lib/quickstart.ex ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient network errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```elixir {:ok, %{body: submitted}} = request(:post, "/url-scraper", %{ url: "https://news.ycombinator.com", generateJson: true }) ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```elixir {:ok, %{body: submitted}} = request(:post, "/url-scraper", %{ url: "https://example.com/spa", useBrowser: true }) ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Use it from Phoenix Wrap `scrape/1` in a `GenServer` or schedule via `Oban` — the polling loop blocks for up to 3 minutes per URL. For Phoenix LiveView, kick off scraping in a `Task.async/1` and update the LiveView when the result arrives. --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # Go (/docs/documentation/getting-started/go) Submit a scrape, poll for the result, and handle transient errors — all with `net/http` from Go's standard library. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install No third-party packages needed — everything is in the standard library. Just initialize a module: ```bash go mod init quickstart ``` --- ## Scrape a page Save as `main.go`: ```go package main import ( "bytes" "encoding/json" "fmt" "net/http" "os" "time" ) func main() { apiKey := os.Getenv("ANAKIN_API_KEY") if apiKey == "" { panic("ANAKIN_API_KEY is not set") } base := "https://api.anakin.io/v1" client := &http.Client{Timeout: 30 * time.Second} do := func(method, path string, body any) map[string]any { var buf bytes.Buffer if body != nil { json.NewEncoder(&buf).Encode(body) } req, _ := http.NewRequest(method, base+path, &buf) req.Header.Set("X-API-Key", apiKey) req.Header.Set("Content-Type", "application/json") resp, err := client.Do(req) if err != nil { return nil // caller retries on nil } defer resp.Body.Close() var out map[string]any json.NewDecoder(resp.Body).Decode(&out) return out } submitted := do("POST", "/url-scraper", map[string]string{"url": "https://example.com"}) jobID := submitted["jobId"].(string) for i := 0; i < 60; i++ { job := do("GET", "/url-scraper/"+jobID, nil) if job == nil { time.Sleep(3 * time.Second) // retry transient errors continue } switch job["status"] { case "completed": fmt.Println(job["markdown"]) return case "failed": panic(fmt.Sprintf("scrape failed: %v", job["error"])) } time.Sleep(3 * time.Second) } panic("timed out after 3 minutes") } ``` Run it: ```bash go run main.go ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient network errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```go submitted := do("POST", "/url-scraper", map[string]any{ "url": "https://news.ycombinator.com", "generateJson": true, }) ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```go submitted := do("POST", "/url-scraper", map[string]any{ "url": "https://example.com/spa", "useBrowser": true, }) ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # Java (/docs/documentation/getting-started/java) Submit a scrape, poll for the result, and handle transient errors — using `java.net.http.HttpClient` from the Java 11+ standard library plus Jackson for JSON parsing. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install `HttpClient` is in the Java 11+ standard library. Jackson is the de facto JSON library and is already on the classpath in nearly every Spring Boot, Quarkus, or Micronaut project. **Maven**: ```xml <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.17.0</version> </dependency> ``` **Gradle**: ```groovy implementation 'com.fasterxml.jackson.core:jackson-databind:2.17.0' ``` --- ## Scrape a page Save as `Quickstart.java`: ```java import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import java.time.Duration; import java.util.Map; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.ObjectMapper; public class Quickstart { static final String BASE = "https://api.anakin.io/v1"; static final String API_KEY = System.getenv("ANAKIN_API_KEY"); static final HttpClient HTTP = HttpClient.newBuilder() .connectTimeout(Duration.ofSeconds(30)).build(); static final ObjectMapper JSON = new ObjectMapper(); static JsonNode request(String method, String path, Object body) throws Exception { var publisher = body == null ? HttpRequest.BodyPublishers.noBody() : HttpRequest.BodyPublishers.ofString(JSON.writeValueAsString(body)); var req = HttpRequest.newBuilder(URI.create(BASE + path)) .header("X-API-Key", API_KEY) .header("Content-Type", "application/json") .method(method, publisher).build(); var resp = HTTP.send(req, HttpResponse.BodyHandlers.ofString()); return JSON.readTree(resp.body()); } static JsonNode scrape(String url) throws Exception { var submitted = request("POST", "/url-scraper", Map.of("url", url)); var jobId = submitted.get("jobId").asText(); for (int i = 0; i < 60; i++) { JsonNode job; try { job = request("GET", "/url-scraper/" + jobId, null); } catch (Exception e) { Thread.sleep(3000); continue; } // retry transient errors switch (job.get("status").asText()) { case "completed": return job; case "failed": throw new RuntimeException("scrape failed: " + job.path("error").asText("")); } Thread.sleep(3000); } throw new RuntimeException("timed out after 3 minutes"); } public static void main(String[] args) throws Exception { if (API_KEY == null) throw new RuntimeException("ANAKIN_API_KEY is not set"); var job = scrape("https://example.com"); System.out.println(job.get("markdown").asText()); } } ``` Run it (with Maven/Gradle handling Jackson on the classpath): ```bash mvn compile exec:java -Dexec.mainClass=Quickstart # or with Gradle: ./gradlew run ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient I/O errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```java var submitted = request("POST", "/url-scraper", Map.of( "url", "https://news.ycombinator.com", "generateJson", true )); ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```java var submitted = request("POST", "/url-scraper", Map.of( "url", "https://example.com/spa", "useBrowser", true )); ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Use it from Spring Boot Wrap the `request` and `scrape` methods in a `@Service` and call from a `@Async` method or a Spring Batch job — the polling loop blocks for up to 3 minutes per URL, so background execution is the natural fit. For non-blocking use, switch to `HTTP.sendAsync()` and chain the polling with `CompletableFuture`. --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # Node.js (/docs/documentation/getting-started/node) Submit a scrape, poll for the result, and handle transient errors — using `fetch`, which is built into Node.js 18+ and every modern browser. No `axios`, no `node-fetch`, no dependencies. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install `fetch` is built into Node.js 18+ and Bun and Deno. Confirm: ```bash node --version # v18.0.0 or later ``` No npm packages needed for the basic flow. Just an `.mjs` (or `"type": "module"` in `package.json`) for top-level `await`. --- ## Scrape a page Save as `quickstart.mjs`: ```javascript const BASE = "https://api.anakin.io/v1" const API_KEY = process.env.ANAKIN_API_KEY if (!API_KEY) throw new Error("ANAKIN_API_KEY is not set") async function request(method, path, body) { try { const resp = await fetch(BASE + path, { method, headers: { "X-API-Key": API_KEY, "Content-Type": "application/json" }, body: body ? JSON.stringify(body) : undefined, signal: AbortSignal.timeout(30_000), }) return await resp.json() } catch { return null // caller retries on null } } async function scrape(url) { const submitted = await request("POST", "/url-scraper", { url }) const jobId = submitted.jobId for (let i = 0; i < 60; i++) { const job = await request("GET", `/url-scraper/${jobId}`) if (!job) { await new Promise(r => setTimeout(r, 3000)) // retry transient errors continue } if (job.status === "completed") return job if (job.status === "failed") { throw new Error(`scrape failed: ${job.error}`) } await new Promise(r => setTimeout(r, 3000)) } throw new Error("timed out after 3 minutes") } const job = await scrape("https://example.com") console.log(job.markdown) ``` Run it: ```bash node quickstart.mjs ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient fetch errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```javascript const submitted = await request("POST", "/url-scraper", { url: "https://news.ycombinator.com", generateJson: true, }) ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```javascript const submitted = await request("POST", "/url-scraper", { url: "https://example.com/spa", useBrowser: true, }) ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Use it from Next.js / Express / NestJS Drop the `request` and `scrape` functions into a service module and call from a queued background job (BullMQ, Inngest, Trigger.dev) — the polling loop awaits up to 3 minutes per URL, so background execution is the natural fit. For Next.js specifically, run from a route handler with `export const maxDuration = 300` if your platform supports it, or push to a queue: ```javascript // app/api/scrape/route.js import { Queue } from "bullmq" const scrapeQueue = new Queue("scrape") export async function POST(req) { const { url } = await req.json() await scrapeQueue.add("scrape", { url }) return Response.json({ queued: true }) } ``` --- ## TypeScript For typed responses, define a minimal interface and cast — no SDK needed: ```typescript type Job = { id: string status: "pending" | "completed" | "failed" markdown?: string generatedJson?: unknown error?: string } const job = await scrape("https://example.com") as Job ``` --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # PHP (/docs/documentation/getting-started/php) Submit a scrape, poll for the result, and handle transient errors — all with PHP's bundled `curl` extension. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install No Composer packages needed — `curl` and `json` are PHP extensions enabled by default. Requires **PHP 8.0+** for the `throw` expression in the example below. ```bash php --version # confirm 8.0+ php -m | grep -E "curl|json" # confirm extensions present ``` --- ## Scrape a page Save as `quickstart.php`: ```php <?php $apiKey = getenv("ANAKIN_API_KEY") ?: throw new Exception("ANAKIN_API_KEY is not set"); $base = "https://api.anakin.io/v1"; function request(string $method, string $path, ?array $body = null): ?array { global $apiKey, $base; $ch = curl_init($base . $path); curl_setopt_array($ch, [ CURLOPT_RETURNTRANSFER => true, CURLOPT_CUSTOMREQUEST => $method, CURLOPT_HTTPHEADER => ["X-API-Key: $apiKey", "Content-Type: application/json"], CURLOPT_TIMEOUT => 30, ]); if ($body !== null) { curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($body)); } $response = curl_exec($ch); $err = curl_error($ch); curl_close($ch); return $err ? null : json_decode($response, true); } function scrape(string $url): array { $submitted = request("POST", "/url-scraper", ["url" => $url]); $jobId = $submitted["jobId"]; for ($i = 0; $i < 60; $i++) { $job = request("GET", "/url-scraper/" . $jobId); if ($job === null) { sleep(3); // retry transient errors continue; } if ($job["status"] === "completed") return $job; if ($job["status"] === "failed") { throw new Exception("scrape failed: " . ($job["error"] ?? "")); } sleep(3); } throw new Exception("timed out after 3 minutes"); } $job = scrape("https://example.com"); echo $job["markdown"]; ``` Run it: ```bash php quickstart.php ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient cURL errors silently — only surfaces real failures. 4. Echoes the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```php $submitted = request("POST", "/url-scraper", [ "url" => "https://news.ycombinator.com", "generateJson" => true, ]); ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```php $submitted = request("POST", "/url-scraper", [ "url" => "https://example.com/spa", "useBrowser" => true, ]); ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Use it from Laravel Drop the `request` and `scrape` functions into a service class and call from a queued job — the polling loop blocks for up to 3 minutes per URL, so background jobs are the natural fit: ```php // app/Jobs/ScrapeUrlJob.php class ScrapeUrlJob implements ShouldQueue { public function __construct(public string $url) {} public function handle(AnakinScraper $scraper): void { $job = $scraper->scrape($this->url); Page::create(["url" => $this->url, "markdown" => $job["markdown"]]); } } ``` --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # Python (/docs/documentation/getting-started/python) Submit a scrape, poll for the result, and handle transient errors — using `requests`, the standard Python HTTP library. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install `requests` is the de facto Python HTTP library. Install once: ```bash pip install requests ``` If you'd rather avoid any dependency, the same logic works with `urllib.request` from the standard library — but `requests` is cleaner and present in nearly every Python project. --- ## Scrape a page Save as `quickstart.py`: ```python import os import time import requests BASE = "https://api.anakin.io/v1" API_KEY = os.environ.get("ANAKIN_API_KEY") if not API_KEY: raise SystemExit("ANAKIN_API_KEY is not set") session = requests.Session() session.headers.update({"X-API-Key": API_KEY, "Content-Type": "application/json"}) def request(method: str, path: str, json=None): try: resp = session.request(method, BASE + path, json=json, timeout=30) return resp.json() except requests.RequestException: return None # caller retries on None def scrape(url: str) -> dict: submitted = request("POST", "/url-scraper", {"url": url}) job_id = submitted["jobId"] for _ in range(60): job = request("GET", f"/url-scraper/{job_id}") if job is None: time.sleep(3) # retry transient errors continue if job["status"] == "completed": return job if job["status"] == "failed": raise RuntimeError(f"scrape failed: {job.get('error')}") time.sleep(3) raise TimeoutError("timed out after 3 minutes") if __name__ == "__main__": job = scrape("https://example.com") print(job["markdown"]) ``` Run it: ```bash python quickstart.py ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient `RequestException` errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: True` to have AI return structured data: ```python submitted = request("POST", "/url-scraper", { "url": "https://news.ycombinator.com", "generateJson": True, }) ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: True`: ```python submitted = request("POST", "/url-scraper", { "url": "https://example.com/spa", "useBrowser": True, }) ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Use it from Django / FastAPI Wrap `scrape()` in a Celery / RQ / Dramatiq task — the polling loop blocks for up to 3 minutes per URL, so background execution is the natural fit. For FastAPI specifically, swap `requests` for `httpx.AsyncClient` and `await asyncio.sleep(3)` to keep the event loop free: ```python # app/tasks/scrape.py from celery import shared_task @shared_task def scrape_url(url: str): job = scrape(url) Page.objects.create(url=url, markdown=job["markdown"]) ``` --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # Ruby (/docs/documentation/getting-started/ruby) Submit a scrape, poll for the result, and handle transient errors — all with `net/http` and `json` from Ruby's standard library. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install No gems needed — `net/http` and `json` ship with Ruby. This works in any Ruby 2.7+ project, including Rails. --- ## Scrape a page Save as `quickstart.rb`: ```ruby require "net/http" require "json" require "uri" BASE_URL = "https://api.anakin.io/v1" API_KEY = ENV.fetch("ANAKIN_API_KEY") { raise "ANAKIN_API_KEY is not set" } def request(method, path, body = nil) uri = URI(BASE_URL + path) http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.read_timeout = 30 req = Net::HTTPGenericRequest.new(method, !body.nil?, true, uri.request_uri) req["X-API-Key"] = API_KEY req["Content-Type"] = "application/json" req.body = body.to_json if body JSON.parse(http.request(req).body) end def scrape(url) submitted = request("POST", "/url-scraper", { url: url }) job_id = submitted["jobId"] 60.times do begin job = request("GET", "/url-scraper/#{job_id}") rescue StandardError sleep 3 # retry transient errors next end case job["status"] when "completed" then return job when "failed" then raise "scrape failed: #{job['error']}" end sleep 3 end raise "timed out after 3 minutes" end job = scrape("https://example.com") puts job["markdown"] ``` Run it: ```bash ruby quickstart.rb ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient network errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Pass `generateJson: true` to have AI return structured data: ```ruby submitted = request("POST", "/url-scraper", { url: "https://news.ycombinator.com", generateJson: true }) ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```ruby submitted = request("POST", "/url-scraper", { url: "https://example.com/spa", useBrowser: true }) ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Use it from Rails Drop the `request` and `scrape` methods into a service object (e.g. `app/services/anakin_scraper.rb`) and call from a job: ```ruby # app/jobs/scrape_url_job.rb class ScrapeUrlJob < ApplicationJob queue_as :default def perform(url) job = AnakinScraper.scrape(url) Page.create!(url: url, markdown: job["markdown"]) end end ``` Background jobs are the natural fit — the polling loop blocks for up to 3 minutes per URL. --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # Rust (/docs/documentation/getting-started/rust) Submit a scrape, poll for the result, and handle transient errors — using `ureq` (a minimal sync HTTP client) and `serde_json`. --- ## Authentication Set your API key as an environment variable. Get a key from the [Dashboard](/dashboard). ```bash export ANAKIN_API_KEY=ak-your-key-here ``` The base URL is `https://api.anakin.io/v1`. Every request authenticates via the `X-API-Key` header. --- ## Install Rust's stdlib doesn't ship an HTTP client. `ureq` is the lightest sensible choice — synchronous, no async runtime, single small crate. For async or `reqwest` users, the same logic translates trivially. Add to `Cargo.toml`: ```toml [dependencies] ureq = { version = "2", features = ["json"] } serde_json = "1" ``` --- ## Scrape a page Save as `src/main.rs`: ```rust use std::env; use std::thread::sleep; use std::time::Duration; use serde_json::{json, Value}; fn main() -> Result<(), Box<dyn std::error::Error>> { let api_key = env::var("ANAKIN_API_KEY") .map_err(|_| "ANAKIN_API_KEY is not set")?; let base = "https://api.anakin.io/v1"; let agent = ureq::AgentBuilder::new() .timeout(Duration::from_secs(30)) .build(); let submitted: Value = agent.post(&format!("{base}/url-scraper")) .set("X-API-Key", &api_key) .send_json(json!({ "url": "https://example.com" }))? .into_json()?; let job_id = submitted["jobId"].as_str().ok_or("missing jobId")?; for _ in 0..60 { let resp = agent.get(&format!("{base}/url-scraper/{job_id}")) .set("X-API-Key", &api_key) .call(); let job: Value = match resp { Ok(r) => r.into_json()?, Err(_) => { sleep(Duration::from_secs(3)); continue; } // retry transient errors }; match job["status"].as_str() { Some("completed") => { println!("{}", job["markdown"].as_str().unwrap_or("")); return Ok(()); } Some("failed") => { return Err(format!("scrape failed: {}", job["error"]).into()); } _ => {} } sleep(Duration::from_secs(3)); } Err("timed out after 3 minutes".into()) } ``` Run it: ```bash cargo run ``` --- ## What this does 1. Submits `https://example.com` to `/url-scraper` and gets back a `jobId`. 2. Polls `/url-scraper/{jobId}` every 3 seconds (up to 60 attempts = 3 minutes). 3. Retries transient network errors silently — only surfaces real failures. 4. Prints the final `markdown` when the job completes. Most jobs finish in 3–15 seconds. --- ## Go further ### Extract structured JSON with AI Replace the submit body with `generateJson: true` to have AI return structured data: ```rust let submitted: Value = agent.post(&format!("{base}/url-scraper")) .set("X-API-Key", &api_key) .send_json(json!({ "url": "https://news.ycombinator.com", "generateJson": true }))? .into_json()?; ``` The completed response includes a `generatedJson` field with structured data inferred from the page. ### Scrape JavaScript-heavy sites For SPAs and dynamically-loaded pages, add `useBrowser: true`: ```rust .send_json(json!({ "url": "https://example.com/spa", "useBrowser": true }))? ``` > Only use browser mode when needed — standard scraping is faster and cheaper. --- ## Async with reqwest If you're already on Tokio, swap `ureq` for `reqwest` and `.await` the requests — the loop structure is identical. The polling sleep becomes `tokio::time::sleep(Duration::from_secs(3)).await`. --- ## Next steps <CardGrid> <ProductCard title="API Reference" description="Full endpoint docs with parameters and response schemas" href="/docs/api-reference" /> <ProductCard title="Polling Jobs" description="Async job patterns, intervals, and production polling code" href="/docs/api-reference/polling-jobs" /> <ProductCard title="Pricing & Credits" description="Credit costs per operation and plan details" href="/docs/documentation/pricing" /> <ProductCard title="Use Cases" description="RAG pipelines, lead gen, price monitoring, and more" href="/docs/documentation/use-cases" /> </CardGrid> --- # AI & Agent Data Ingestion (/docs/documentation/use-cases/ai-agent-data-ingestion) Use Anakin's scraping API to turn web pages into **structured JSON** (or clean text/markdown) for **RAG pipelines**, **AI agents**, and **support copilots**. Typical workflows include crawling docs, help centers, product pages, and forum threads, then chunking and indexing for retrieval. --- ### Common sources * Documentation sites (MDX/Docs frameworks, API references) * Help centers / knowledge bases (Zendesk-style, custom) * Product pages + changelogs * Forums / community threads --- ### What to extract * Title, headings hierarchy (H1/H2/H3) * Main content blocks (exclude nav/footer) * Code blocks (language + code) * Tables, lists, callouts * Canonical URL, publish/updated timestamps * Outbound links for crawl expansion --- ### Implementation notes * Prefer **browser rendering** for JS docs sites and SPAs. * Use **structured extraction** to keep stable fields: `title`, `sections[]`, `code_blocks[]`, `tables[]`. * Use **dedupe keys** (canonical URL + content hash) to avoid re-indexing. * For RAG: chunk by heading boundaries; store metadata (source URL, section path). --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Do you generate embeddings or manage vector databases?</AccordionTrigger> <AccordionContent>No. The API extracts structured content from webpages. Embeddings, chunking, and indexing are handled in your own AI pipeline.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can I extract clean content from JS-heavy documentation sites?</AccordionTrigger> <AccordionContent>Yes. Use browser rendering to extract the fully rendered DOM and then structure headings, paragraphs, code blocks, and tables.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>How do I avoid reprocessing unchanged content?</AccordionTrigger> <AccordionContent>Store a content hash of the structured output and compare it across runs. Only re-embed if the hash changes.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>Can I scrape private or logged-in knowledge bases?</AccordionTrigger> <AccordionContent>Yes, if you provide valid session credentials or cookies using authenticated sessions.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Does the API clean or rewrite content?</AccordionTrigger> <AccordionContent>No. It extracts what is present on the page. Any cleaning or transformation happens in your pipeline.</AccordionContent> </AccordionItem> </Accordion> --- # Browser Automation & Web Workflows (/docs/documentation/use-cases/browser-automation) Automate JS-heavy flows to reach data that isn't available via static HTML alone. This includes **authenticated dashboards**, multi-step navigation, and extracting data after interactions. --- ### Common sources * Authenticated portals (member-only pages, dashboards) * Multi-step flows (filters, tabs, pagination controls) * JS-rendered apps with client-side routing --- ### What to extract * Post-login content (tables, lists, metrics) * UI state-specific data (selected filters, tabs, date ranges) * Export links / report links (when available) --- ### Implementation notes * Use authenticated sessions (cookies/tokens) and browser rendering. * Extract stateful metadata: `filters_applied`, `date_range`, `view_name`. * Build idempotent workflows: re-runable steps, clear failure modes, retries. --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Is this full RPA?</AccordionTrigger> <AccordionContent>No. It enables browser-rendered extraction and authenticated flows. Workflow orchestration is built on your side.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can I scrape dashboards behind login?</AccordionTrigger> <AccordionContent>Yes, using authenticated sessions with valid credentials or cookies.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>Can I interact with filters or tabs?</AccordionTrigger> <AccordionContent>Yes, if your workflow triggers those states before extraction.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>What if the page uses client-side routing?</AccordionTrigger> <AccordionContent>Browser rendering resolves the final DOM state for extraction.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Does the API maintain persistent sessions automatically?</AccordionTrigger> <AccordionContent>Session management logic should be handled in your integration.</AccordionContent> </AccordionItem> </Accordion> --- # Competitive Intelligence & Pricing (/docs/documentation/use-cases/competitive-intel) Extract competitor site data (pricing pages, feature pages, release notes, product catalogs) into structured fields for **competitive analysis**, **pricing intelligence**, and **feature comparison**. --- ### Common sources * Pricing pages (plan tiers, add-ons, limits, feature matrices) * Feature pages, integrations pages * Changelogs / release notes * Public product catalogs and category pages --- ### What to extract * Plans and tiers: name, price, billing cadence, currency * Feature matrices (rows/columns normalized) * Usage limits: seats, API calls, storage, rate limits * Add-ons and overage pricing * Promotions, coupons, seasonal offers (where present) * Change metadata: last updated date, page version id, content hash --- ### Implementation notes * Rendering often required for pricing widgets and tabs. * Normalize plan data into a consistent schema across vendors. * Use `content_hash` for change detection and diffing (your system can compute diffs). * Handle geo-based pricing by varying locale headers / region routing (if needed). --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Does this track competitor changes automatically?</AccordionTrigger> <AccordionContent>No. You must schedule recurring scrapes and compare structured outputs to detect changes.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can pricing tables that require interaction be extracted?</AccordionTrigger> <AccordionContent>Yes, if they are rendered in the DOM after interaction. Browser rendering is required for most dynamic pricing widgets.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>Can I extract feature comparison tables?</AccordionTrigger> <AccordionContent>Yes. Tables and structured lists can be normalized into JSON fields.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>Does the API interpret pricing logic (discounts, bundles)?</AccordionTrigger> <AccordionContent>No. It extracts visible pricing data. Interpretation is handled downstream.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>How do I detect real pricing changes?</AccordionTrigger> <AccordionContent>Compare structured pricing JSON across runs instead of raw HTML to avoid layout noise.</AccordionContent> </AccordionItem> </Accordion> --- # Content Aggregation (/docs/documentation/use-cases/content-aggregation) Aggregate content from multiple webpages into normalized outputs for **news aggregation**, **research feeds**, **internal dashboards**, and **content pipelines**. --- ### Common sources * Blogs and news archives * Company update pages * Release notes/changelog pages * Documentation announcement pages --- ### What to extract * Article list entries: title, URL, excerpt, publish date, author * Full article content (main body + headings) * Tags/categories * Media: featured image URL, embeds (if needed) * Canonical URL + source attribution --- ### Implementation notes * Two-stage approach: scrape index pages → collect article URLs → scrape article pages. * Normalize into a single schema across sources. * Use content hashing to detect updates without storing huge diffs. --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Can I scrape blog archives?</AccordionTrigger> <AccordionContent>Yes. Extract article listings and then scrape individual article pages.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Does the API provide RSS feeds?</AccordionTrigger> <AccordionContent>No. You can generate RSS downstream using extracted content.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>How do I detect new articles?</AccordionTrigger> <AccordionContent>Re-scrape index pages and compare extracted URLs or content hashes.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>Can I extract publication dates and authors?</AccordionTrigger> <AccordionContent>Yes, if they are visible in the page content.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Does the API summarize content?</AccordionTrigger> <AccordionContent>No. It extracts content; summarization must be done using your own AI pipeline.</AccordionContent> </AccordionItem> </Accordion> --- # Financial & Corporate Filings (/docs/documentation/use-cases/financial-filings) Extract structured data from corporate websites for **investor relations**, **press releases**, **earnings pages**, and **public disclosures**. Useful for analysis pipelines and research workflows. --- ### Common sources * Investor Relations (IR) pages (quarterly results, presentations) * Press release archives * Leadership pages and governance pages * Public filings portals (where web accessible) --- ### What to extract * Press release entries: title, date, category, URL * Earnings artifacts: PDF links, webcast links, transcripts (if present) * Company metadata: legal name, HQ location, leadership roster * Document metadata: file type, published date, version identifiers --- ### Implementation notes * Many IR sites load content via JS; enable rendering. * Extract document links and store them (download/processing happens downstream). * Preserve timestamps and original source URLs for traceability. --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Can I extract press release archives?</AccordionTrigger> <AccordionContent>Yes. You can scrape index pages and extract titles, dates, and URLs for each entry.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can the API parse PDFs?</AccordionTrigger> <AccordionContent>No. It extracts webpage content. PDF parsing must be handled separately.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>How do I identify the latest earnings release?</AccordionTrigger> <AccordionContent>Scrape the archive page and sort entries by extracted publish date.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>Can I extract document download links?</AccordionTrigger> <AccordionContent>Yes. If links are visible in the rendered page, they can be captured.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Does the API validate financial data?</AccordionTrigger> <AccordionContent>No. It only extracts what is publicly displayed on the webpage.</AccordionContent> </AccordionItem> </Accordion> --- # Lead Generation (/docs/documentation/use-cases/lead-generation) Extract structured business data from the public web (directories, company pages, partner lists) for **prospecting**, **account research**, and **CRM enrichment**. --- ### Common sources * B2B directories, association listings * Partner / reseller / agency directories * Conference sponsor/exhibitor lists * Company "Contact" and "About" pages --- ### What to extract * Company name, domain, category/industry * Location, service areas * Contact channels: emails (if public), phone, contact forms * Social links, team pages (if public) * Signals: certifications, partner badges, technologies listed --- ### Implementation notes * Be strict about "public web only" fields; store provenance. * Use extraction schema that separates `contact_methods[]` from `people[]`. * Avoid brittle scraping of obfuscated emails—prefer consistent selectors. --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Does the API provide private contact data?</AccordionTrigger> <AccordionContent>No. It extracts only what is publicly available on the scraped pages.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can I scrape company directories?</AccordionTrigger> <AccordionContent>Yes, including listing pages and company profile pages.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>Does the API enrich company data?</AccordionTrigger> <AccordionContent>No. It extracts visible fields. Enrichment requires external systems.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>How do I prevent duplicate companies?</AccordionTrigger> <AccordionContent>Use domain normalization and entity deduplication in your own database.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Can I scrape contact forms?</AccordionTrigger> <AccordionContent>You can extract form structure, but submission workflows require browser automation logic on your side.</AccordionContent> </AccordionItem> </Accordion> --- # Structured Market & Web Research (/docs/documentation/use-cases/market-research) Collect structured facts from many web sources for **market research**, **category research**, and **competitive landscape mapping**. This includes aggregating lists, extracting entities, and building a dataset you can query. --- ### Common sources * Directories (companies, tools, marketplaces) * Public listings pages (partners, agencies, vendor ecosystems) * Industry reports pages and statistics pages * Public datasets pages and data portals --- ### What to extract * Entities: name, description, category, website, location * Pricing/positioning summaries (when publicly listed) * Metadata: tags, industries, integrations, target audience * Tables and structured lists * Links to "detail pages" for deeper extraction --- ### Implementation notes * Start with list pages → collect detail URLs → scrape detail pages for full schema. * Use dedupe by domain + entity name normalization. * Keep provenance: every extracted entity should retain its source URL and scrape timestamp. --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Can I scrape entire directories?</AccordionTrigger> <AccordionContent>Yes. Start from listing pages, collect detail page URLs, then scrape each detail page.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Does the API deduplicate entities across sources?</AccordionTrigger> <AccordionContent>No. Deduplication must be implemented in your data pipeline.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>Can I extract structured attributes consistently across different sites?</AccordionTrigger> <AccordionContent>Yes, but you define the schema. Each source may require slightly different extraction logic.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>Is rendering required for directory sites?</AccordionTrigger> <AccordionContent>Often yes. Many modern directories load content via client-side JavaScript.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Can I crawl multiple levels deep?</AccordionTrigger> <AccordionContent>Yes, as long as you manage crawl scope and URL constraints in your workflow.</AccordionContent> </AccordionItem> </Accordion> --- # ML & Training Data Collection (/docs/documentation/use-cases/ml-training-data) Create datasets from the web for **machine learning**, including **NLP**, **information extraction**, **classification**, and **entity recognition** pipelines. --- ### Common sources * Public articles, blogs, documentation, forums * Public product catalogs and listings * Tables and structured lists useful as labeled sources --- ### What to extract * Clean text with provenance (URL, timestamp) * Structured fields suitable for supervised labels (title/category/price/attributes) * Tables as normalized rows * Image URLs / media metadata (if needed for CV pipelines) --- ### Implementation notes * Keep dataset rows deterministic: same URL → same schema fields. * Store raw + cleaned versions (raw HTML optional). * Use stable identifiers: `source_url`, `content_hash`, `extraction_version`. --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Does the API label training data?</AccordionTrigger> <AccordionContent>No. It extracts raw structured data; labeling is your responsibility.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can I collect large volumes of data?</AccordionTrigger> <AccordionContent>Yes, subject to your usage limits and rate management.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>Is the extracted output deterministic?</AccordionTrigger> <AccordionContent>Yes, if the source page content is unchanged.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>Can I extract multilingual content?</AccordionTrigger> <AccordionContent>Yes. The API returns content as rendered on the page.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Should I store raw HTML for training datasets?</AccordionTrigger> <AccordionContent>That depends on your model objective. Structured JSON is typically easier to manage.</AccordionContent> </AccordionItem> </Accordion> --- # Review & Sentiment Data Extraction (/docs/documentation/use-cases/review-sentiment) Collect structured text from reviews, forums, and community posts for **sentiment analysis**, **topic modeling**, and **voice-of-customer** pipelines. --- ### Common sources * Review pages (product/service reviews) * Community threads and Q&A forums * Public social-like pages (web accessible) --- ### What to extract * Review/post text, rating (if present), date * Author handle (if public), verified flags (if present) * Helpful votes / reactions (if present) * Thread structure: parent/child relationships * Product/entity identifiers (SKU, product name, URL) --- ### Implementation notes * Prefer structured fields: `rating`, `body`, `timestamp`, `thread_id`, `reply_to`. * Handle pagination carefully; store page cursors. * Keep raw text clean (strip UI noise like "Read more", "Translate"). --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Does the API calculate sentiment?</AccordionTrigger> <AccordionContent>No. It extracts review text and metadata. Sentiment scoring must be done separately.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can I scrape paginated reviews?</AccordionTrigger> <AccordionContent>Yes. You must manage pagination logic in your workflow.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>Can I extract ratings and timestamps?</AccordionTrigger> <AccordionContent>Yes, if those elements are visible in the rendered DOM.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>How do I avoid duplicate reviews?</AccordionTrigger> <AccordionContent>Use review IDs or compute a hash of stable fields like author, date, and content.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Can I extract nested replies?</AccordionTrigger> <AccordionContent>Yes, as long as reply hierarchy is present in the DOM.</AccordionContent> </AccordionItem> </Accordion> --- # SEO & Search Intelligence (/docs/documentation/use-cases/seo-marketing) Extract data from search and content surfaces for **SERP research**, **keyword intelligence**, **content auditing**, and **AEO/answer engine optimization** workflows. --- ### Common sources * Search results pages (where accessible) * Competitor content hubs and blog archives * Programmatic landing pages * FAQ-rich pages / schema-heavy pages --- ### What to extract * Page title, meta description, canonical URL * Headings, FAQs, structured data markers * Internal links and topic clusters * Publish date and author (if present) * Content layout: lists, tables, definitions --- ### Implementation notes * Rendering helps when SERP pages are JS-heavy. * Extract "main content" + metadata; ignore navigation elements. * Store structured blocks for downstream scoring (readability, topical coverage, schema presence). --- ### FAQs <Accordion type="single" collapsible className="w-full"> <AccordionItem value="q1"> <AccordionTrigger>Is this a rank tracking tool?</AccordionTrigger> <AccordionContent>No. You can extract SERP or page content, but tracking and comparison must be built separately.</AccordionContent> </AccordionItem> <AccordionItem value="q2"> <AccordionTrigger>Can I extract meta tags and structured data?</AccordionTrigger> <AccordionContent>Yes. Titles, meta descriptions, canonical tags, and schema markup can be captured.</AccordionContent> </AccordionItem> <AccordionItem value="q3"> <AccordionTrigger>Does the API analyze keyword performance?</AccordionTrigger> <AccordionContent>No. It extracts page content and metadata. Analysis happens downstream.</AccordionContent> </AccordionItem> <AccordionItem value="q4"> <AccordionTrigger>Can I extract FAQ sections from pages?</AccordionTrigger> <AccordionContent>Yes, if they are present in the DOM or marked up with structured data.</AccordionContent> </AccordionItem> <AccordionItem value="q5"> <AccordionTrigger>Is rendering required for search result pages?</AccordionTrigger> <AccordionContent>Often yes, as many are dynamically generated.</AccordionContent> </AccordionItem> </Accordion> --- # SDKs & CLI (/docs/sdks) Official client libraries, command-line tools, and an MCP server for the Anakin API. All open source under Apache 2.0. --- ### SDKs | SDK | Status | Install | Description | |-----|--------|---------|-------------| | [Python SDK](/docs/sdks/python) | Alpha (v0.1.x) | `pip install anakin-sdk` | Native Python client with `pydantic` v2 typed models | | [Node.js SDK](/docs/sdks/node) | Alpha (v0.1.x) | `npm install @anakin-io/sdk` | Native TypeScript client, ESM + CJS, built on `fetch` | | [Go SDK](/docs/sdks/go) | Alpha (v0.1.x) | `go get github.com/Anakin-Inc/anakin-go` | Pure stdlib Go module, functional options, zero dependencies | | [Rust SDK](/docs/sdks/rust) | Alpha (v0.1.x) | `anakin-sdk = "0.1"` in `Cargo.toml` | Async Rust crate built on reqwest + tokio | | [PHP SDK](/docs/sdks/php) | Alpha (v0.1.x) | `composer require anakin/sdk` | PHP 8.1+ client built on Guzzle 7 | | [Ruby SDK](/docs/sdks/ruby) | Alpha (v0.1.x) | `gem install anakin-sdk` | Pure-stdlib Ruby gem, no runtime dependencies | | [Elixir SDK](/docs/sdks/elixir) | Alpha (v0.1.x) | `{:anakin, "~> 0.1"}` in `mix.exs` | Elixir Hex package built on Req + Jason | | [Java SDK](/docs/sdks/java) | Alpha (v0.1.x) | `io.github.anakin-inc:anakin-sdk:0.1.0` (Maven / Gradle) | Java 17+ client, zero external HTTP dependencies | | [.NET SDK](/docs/sdks/dotnet) | Coming soon | NuGet: `Anakin` (pending) | C# async client for .NET 6+ — publishing in progress | ### CLI | Tool | Status | Description | |------|--------|-------------| | [Anakin CLI](/docs/sdks/cli) | Available | Scrape, search, and research from your terminal | ### MCP server (for AI agents) | Server | Status | Install | Description | |--------|--------|---------|-------------| | [`@anakin-io/mcp`](/docs/integrations/ai-agents/mcp-server) | Alpha (v0.1.x) | `npx -y @anakin-io/mcp init --all` | Plug Anakin into Claude Desktop, Claude Code, Cursor, Cline, Continue, Zed, Windsurf, and VS Code via MCP — no glue code required | --- ### Why use the SDKs? - **Three-line scrape**, no boilerplate. Submit-then-poll is hidden behind a single sync call. - **Identical surface across every language** so switching is mechanical: `scrape`, `map`, `crawl`, `search`, `agentic_search`, `wire`, plus browser-session CRUD. - **Built-in retries** with exponential backoff and `Retry-After` honour on 429s. - **Typed errors** — every SDK exposes the same hierarchy (`AuthenticationError`, `InsufficientCreditsError` with balance/required, `RateLimitError`, `JobFailedError`, etc.) named idiomatically per language. - **Hidden polling** — `client.crawl(...)` returns the final result. No job IDs to manage. --- # Anakin CLI (/docs/sdks/cli) Scrape websites, search the web, and run deep research — all from your terminal. | | | |---|---| | **Latest version** | 0.1.0 | | **License** | MIT | | **Python** | 3.10+ | | **PyPI** | [anakin-cli](https://pypi.org/project/anakin-cli/) | | **Source** | [GitHub](https://github.com/Anakin-Inc/anakin-cli) | ```bash pip install anakin-cli ``` --- ### Prerequisites Before installing, make sure you have: 1. **Python 3.10 or higher** — check with `python --version` 2. **pip** — Python's package manager (included with Python 3.10+) 3. **An API key** — get one from the [Dashboard](/dashboard). If you don't have an account, [sign up here](/signup) --- ### Installation <Tabs items={["pip", "pipx"]}> <Tab value="pip"> ```bash pip install anakin-cli ``` </Tab> <Tab value="pipx"> ```bash # Install in an isolated environment (recommended for CLI tools) pipx install anakin-cli ``` </Tab> </Tabs> Verify the installation: ```bash anakin status ``` To upgrade to the latest version: ```bash pip install --upgrade anakin-cli ``` --- ### Authentication Set up your API key so the CLI can make requests on your behalf. **Login command (recommended)** ```bash anakin login --api-key "ak-your-key-here" ``` This saves your key locally. You only need to do this once. **Environment variable** ```bash export ANAKIN_API_KEY="ak-your-key-here" ``` **Interactive prompt** If no key is configured, the CLI will prompt you to enter one. --- ### Quick start ```bash # Scrape a page to markdown anakin scrape "https://example.com" # Extract structured JSON with AI anakin scrape "https://example.com/product" --format json # Scrape a JS-heavy site with headless browser anakin scrape "https://example.com/spa" --browser # Batch scrape multiple URLs anakin scrape-batch "https://a.com" "https://b.com" "https://c.com" # AI-powered web search anakin search "python async best practices" # Deep research (takes 1–5 min) anakin research "comparison of web frameworks 2025" -o report.json ``` --- ### Commands overview | Command | Description | |---------|-------------| | [`anakin login`](/docs/sdks/cli/commands#login) | Save your API key locally | | [`anakin status`](/docs/sdks/cli/commands#status) | Check version and authentication status | | [`anakin scrape`](/docs/sdks/cli/commands#scrape) | Scrape a single URL to markdown, JSON, or raw | | [`anakin scrape-batch`](/docs/sdks/cli/commands#scrape-batch) | Scrape up to 10 URLs in parallel | | [`anakin search`](/docs/sdks/cli/commands#search) | AI-powered web search (instant results) | | [`anakin research`](/docs/sdks/cli/commands#research) | Deep multi-stage agentic research | See the full [Commands Reference](/docs/sdks/cli/commands) for all flags and options, or check out [Examples & Recipes](/docs/sdks/cli/examples) for real-world usage patterns. --- ### Output modes Every command that returns data supports the `-o` flag to write to a file. Without it, output goes to stdout. ```bash # Print to terminal anakin scrape "https://example.com" # Save to file anakin scrape "https://example.com" -o page.md ``` The `scrape` command also supports three output formats: | Format | Flag | What you get | |--------|------|-------------| | Markdown | `--format markdown` | Clean readable text (default) | | JSON | `--format json` | AI-extracted structured data | | Raw | `--format raw` | Full API response with HTML and metadata | --- ### Tips **Always quote URLs** containing `?`, `&`, or `#` — shells interpret these as special characters: ```bash # Wrong — zsh will fail anakin scrape https://example.com/page?id=123 # Correct anakin scrape "https://example.com/page?id=123" ``` **Piping works cleanly** because all progress messages go to stderr: ```bash anakin scrape "https://example.com" --format json | jq '.title' ``` **Use `--browser`** for JavaScript-heavy sites, SPAs, and dynamically loaded content. **Use `--country`** to route requests through a specific country's proxy. See all [207 supported countries](/docs/api-reference/supported-countries). --- ### Support - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io - **PyPI** — [pypi.org/project/anakin-cli](https://pypi.org/project/anakin-cli/) --- # .NET SDK (/docs/sdks/dotnet) The official .NET / C# SDK for the Anakin API is currently in development and will be published to NuGet shortly. | | | |---|---| | **Status** | Coming soon | | **Target** | `net6.0`, `net8.0` | | **Package** | `Anakin` on NuGet (pending) | | **Source** | [github.com/Anakin-Inc/anakin-dotnet](https://github.com/Anakin-Inc/anakin-dotnet) | | **License** | Apache 2.0 | --- The source code is complete and tested. We are currently working through the NuGet account setup required for publishing. ### Preview Once published, installation will be: ```bash dotnet add package Anakin ``` And usage: ```csharp using Anakin; using var client = AnakinClient.Create(opts => { opts.ApiKey = "ak-..."; // or set ANAKIN_API_KEY }); var doc = await client.ScrapeAsync("https://example.com"); Console.WriteLine(doc.Markdown); ``` All the same methods as the other SDKs — `ScrapeAsync`, `MapAsync`, `CrawlAsync`, `SearchAsync`, `AgenticSearchAsync`, `WireAsync`, plus browser-session management. Built on `HttpClient` + `System.Text.Json` with no third-party dependencies. ### Get notified Watch the [GitHub repo](https://github.com/Anakin-Inc/anakin-dotnet) for the release, or check [anakin.io/docs/releases](/docs/releases) for the announcement. In the meantime, you can use the REST API directly — see the [API reference](/docs/api-reference) and the [.NET Quick Start](/docs/documentation/getting-started/dotnet). --- # Elixir SDK (/docs/sdks/elixir) Official Elixir Hex package for the Anakin API. Wraps every documented endpoint, runs internal polling, and returns idiomatic `{:ok, result}` / `{:error, exception}` tuples. Built on [Req](https://github.com/wojtekmach/req) and Jason. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Language** | Elixir 1.14+ | | **Hex name** | [`anakin`](https://hex.pm/packages/anakin) on Hex | | **Docs** | [hexdocs.pm/anakin](https://hexdocs.pm/anakin/0.1.0) (auto-generated) | | **Source** | [github.com/Anakin-Inc/anakin-elixir](https://github.com/Anakin-Inc/anakin-elixir) | | **License** | Apache 2.0 | --- ### Install Add to your `mix.exs`: ```elixir def deps do [{:anakin, "~> 0.1"}] end ``` Then: ```bash mix deps.get ``` ### Quickstart ```elixir {:ok, client} = Anakin.Client.new(api_key: "ak-...") # or set ANAKIN_API_KEY env var # Scrape a single URL — returns {:ok, result} or {:error, exception} case Anakin.scrape(client, "https://example.com") do {:ok, doc} -> IO.puts(doc["markdown"]) {:error, e} -> IO.warn(Exception.message(e)) end # Discover URLs on a site {:ok, sitemap} = Anakin.map(client, "https://example.com", limit: 200) IO.inspect(sitemap["links"]) # Crawl pages and get content for each {:ok, crawl} = Anakin.crawl(client, "https://example.com", max_pages: 20) Enum.each(crawl["pages"], fn page -> IO.puts("#{page["url"]}: #{String.length(page["markdown"] || "")}") end) ``` ### What's in v0.1 | Function | Returns | |---|---| | `Anakin.scrape(client, url, opts \\ [])` | `{:ok, map}` — the document | | `Anakin.map(client, url, opts \\ [])` | `{:ok, map}` — discovered links | | `Anakin.crawl(client, url, opts \\ [])` | `{:ok, map}` — crawled pages | | `Anakin.search(client, query, opts \\ [])` | `{:ok, map}` — search results (synchronous API) | | `Anakin.agentic_search(client, prompt, opts \\ [])` | `{:ok, map}` — AI-synthesised answer | | `Anakin.wire(client, action_id, params)` | `{:ok, map}` — Wire action result (run a [Wire](/docs/api-reference/wire) action) | | `Anakin.list_sessions/1`, `create_session/3`, `save_session/3`, `update_session/3`, `delete_session/2` | Browser session CRUD | All response functions return `{:ok, parsed_json_map}` on success or `{:error, %Anakin.Error{...}}` on failure. The body of `{:ok, ...}` is the parsed JSON map — strict struct types are on the v0.2 roadmap. ### Configuration ```elixir {:ok, client} = Anakin.Client.new( api_key: "ak-...", # or ANAKIN_API_KEY env var base_url: "https://api.anakin.io/v1", request_timeout_ms: 60_000, # per-request HTTP timeout max_retries: 4, # retries on 429 / 5xx poll_interval_ms: 1_000, # initial polling delay poll_max_interval_ms: 10_000, # cap on exponential backoff poll_timeout_ms: 300_000, # total wait before %JobTimeout{} req_options: [] # extra options forwarded to Req.request/2 ) ``` You can also use `Anakin.Client.new!/1` to raise on missing API key (useful in app config / supervision trees). ### Errors Pattern-match on the struct module to react to specific failure modes: ```elixir case Anakin.scrape(client, "https://example.com") do {:ok, doc} -> IO.puts(doc["markdown"]) {:error, %Anakin.Error.InsufficientCredits{balance: b, required: r}} -> IO.puts("out of credits: balance=#{b}, needed=#{r}") {:error, %Anakin.Error.Authentication{}} -> IO.puts("invalid API key — get a fresh one at anakin.io/dashboard") {:error, %Anakin.Error.RateLimit{retry_after: ra}} -> IO.puts("rate limited; retry after #{ra}s") {:error, %Anakin.Error.JobFailed{reason: reason}} -> IO.puts("job failed: #{reason}") {:error, e} -> IO.puts("anakin error: #{Exception.message(e)}") end ``` The hierarchy: | Module | When | |---|---| | `Anakin.Error.Authentication` | 401 — invalid or missing API key | | `Anakin.Error.InsufficientCredits` | 402 — out of credits (`:balance`, `:required`) | | `Anakin.Error.InvalidRequest` | 400 — validation failure | | `Anakin.Error.RateLimit` | 429 — after retry budget exhausted (`:retry_after` seconds) | | `Anakin.Error.JobFailed` | Polled job came back with `status="failed"` (`:reason`) | | `Anakin.Error.JobTimeout` | Polling budget exhausted before terminal status | | `Anakin.Error.Server` | 5xx — after retries exhausted | | `Anakin.Error.Network` | DNS / connect / read-timeout | | `Anakin.Error` | Base struct; unmatched failures fall back here | ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ```elixir {:anakin, "0.1.0"} ``` Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-elixir/issues). --- # Go SDK (/docs/sdks/go) Official Go module for the Anakin API. Wraps every documented endpoint with a single blocking call, internal polling, and a typed error hierarchy. Zero runtime dependencies — pure stdlib HTTP. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Language** | Go 1.21+ | | **Module** | [`github.com/Anakin-Inc/anakin-go`](https://pkg.go.dev/github.com/Anakin-Inc/anakin-go) on pkg.go.dev | | **Source** | [github.com/Anakin-Inc/anakin-go](https://github.com/Anakin-Inc/anakin-go) | | **License** | Apache 2.0 | --- ### Install ```bash go get github.com/Anakin-Inc/anakin-go ``` Requires Go 1.21+. ### Quickstart ```go package main import ( "context" "fmt" "github.com/Anakin-Inc/anakin-go" ) func main() { client, err := anakin.New(anakin.WithAPIKey("ak-...")) // or set ANAKIN_API_KEY if err != nil { panic(err) } // Scrape a single URL — returns the final result, no polling required doc, err := client.Scrape(context.Background(), "https://example.com") if err != nil { panic(err) } fmt.Println(doc.Markdown) // Discover URLs on a site sitemap, err := client.Map(context.Background(), "https://example.com") if err != nil { panic(err) } fmt.Println(sitemap.Links) } ``` ### What's in v0.1 | Method | Returns | |---|---| | `client.Scrape(ctx, url, opts...)` | `*Document` | | `client.Map(ctx, url, opts...)` | `*MapResult` | | `client.Crawl(ctx, url, opts...)` | `*CrawlResult` | | `client.Search(ctx, prompt, opts...)` | `*SearchResult` (synchronous API) | | `client.AgenticSearch(ctx, prompt, opts...)` | `*AgenticSearchResult` | | `client.Wire(ctx, actionID, params)` | `*WireResult` (run a [Wire](/docs/api-reference/wire) action) | | `client.Sessions().List / Create / Save / Update / Delete` | Browser session CRUD | | `anakin.SupportedCountries()` | `[]Country` (static, bundled, no network call) | ### Configuration ```go client, err := anakin.New( anakin.WithAPIKey("ak-..."), // or ANAKIN_API_KEY env var anakin.WithBaseURL("https://api.anakin.io/v1"), anakin.WithTimeout(60 * time.Second), // per-request HTTP timeout anakin.WithMaxRetries(4), // retries on 429 / 5xx anakin.WithPollInterval(1 * time.Second), // initial polling delay anakin.WithPollMaxInterval(10 * time.Second), // cap on exponential backoff anakin.WithPollTimeout(5 * time.Minute), // total wait before JobTimeoutError ) ``` ### Errors Every error returned by the SDK is a typed concrete value. Match with `errors.As`: ```go import "errors" doc, err := client.Scrape(ctx, "https://example.com") var creditsErr *anakin.InsufficientCreditsError var authErr *anakin.AuthenticationError var rlErr *anakin.RateLimitError var jobErr *anakin.JobFailedError switch { case errors.As(err, &creditsErr): fmt.Printf("out of credits: balance=%d, needed=%d\n", creditsErr.Balance, creditsErr.Required) case errors.As(err, &authErr): fmt.Println("invalid API key — get a fresh one at anakin.io/dashboard") case errors.As(err, &rlErr): fmt.Printf("rate limited; retry after %s\n", rlErr.RetryAfter) case errors.As(err, &jobErr): fmt.Printf("job failed: %s\n", jobErr.Reason) case err != nil: fmt.Printf("unknown error: %v\n", err) } ``` The error hierarchy: | Type | When | |---|---| | `*AuthenticationError` | 401 — invalid or missing API key | | `*InsufficientCreditsError` | 402 — out of credits (`.Balance`, `.Required`) | | `*InvalidRequestError` | 400 — validation failure | | `*RateLimitError` | 429 — after retry budget exhausted (`.RetryAfter`) | | `*JobFailedError` | Polled job came back with `status="failed"` (`.Reason`) | | `*JobTimeoutError` | Polling budget exhausted before terminal status | | `*ServerError` | 5xx — after retries exhausted | | `*NetworkError` | DNS / connect / read-timeout (`.Cause`) | | `*APIError` | Base type; everything above embeds it | ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ```bash go get github.com/Anakin-Inc/anakin-go@v0.1.0 ``` Full reference docs and examples are on [pkg.go.dev](https://pkg.go.dev/github.com/Anakin-Inc/anakin-go). Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-go/issues). --- # Java SDK (/docs/sdks/java) Official Java SDK for the Anakin API. Wraps every documented endpoint with a single synchronous call, internal polling, and a typed exception hierarchy. Zero HTTP dependencies — uses `java.net.http.HttpClient` from the JDK. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Language** | Java 17+ | | **Artifact** | `io.github.anakin-inc:anakin-sdk:0.1.0` on Maven Central | | **Source** | [github.com/Anakin-Inc/anakin-java](https://github.com/Anakin-Inc/anakin-java) | | **License** | Apache 2.0 | --- ### Install **Gradle:** ```groovy dependencies { implementation 'io.github.anakin-inc:anakin-sdk:0.1.0' } ``` **Maven:** ```xml <dependency> <groupId>io.github.anakin-inc</groupId> <artifactId>anakin-sdk</artifactId> <version>0.1.0</version> </dependency> ``` Requires Java 17+. ### Quickstart ```java import io.anakin.sdk.Anakin; import io.anakin.sdk.types.Document; public class Demo { public static void main(String[] args) { Anakin client = Anakin.builder() .apiKey("ak-...") // or set ANAKIN_API_KEY .build(); // Scrape a single URL — returns the final result, no polling required Document doc = client.scrape("https://example.com"); System.out.println(doc.markdown); // Discover URLs on a site var sitemap = client.map("https://example.com"); System.out.println(sitemap.links); } } ``` ### What's in v0.1 | Method | Returns | |---|---| | `client.scrape(url)` / `client.scrape(url, opts)` | `Document` | | `client.map(url)` / `client.map(url, opts)` | `MapResult` | | `client.crawl(url)` / `client.crawl(url, opts)` | `CrawlResult` | | `client.search(query)` / `client.search(query, opts)` | `SearchResult` (synchronous API) | | `client.agenticSearch(prompt)` / `client.agenticSearch(prompt, opts)` | `AgenticSearchResult` | | `client.wire(actionId, params)` | `WireResult` (run a [Wire](/docs/api-reference/wire) action) | | `client.sessions().list / create / save / update / delete` | Browser session CRUD | | `Countries.supported()` | `Map<String, String>` (bundled, no network call) | ### Configuration ```java Anakin client = Anakin.builder() .apiKey("ak-...") // or ANAKIN_API_KEY env var .baseURL("https://api.anakin.io/v1") .timeout(Duration.ofSeconds(60)) // per-request HTTP timeout .maxRetries(4) // retries on 429 / 5xx .pollInterval(Duration.ofSeconds(1)) // initial polling delay .pollMaxInterval(Duration.ofSeconds(10)) // cap on exponential backoff .pollTimeout(Duration.ofMinutes(5)) // total wait before JobTimeoutException .build(); ``` ### Errors Every error thrown by the SDK is a typed subclass of `AnakinException`: ```java try { Document doc = client.scrape("https://example.com"); } catch (InsufficientCreditsException e) { System.out.printf("out of credits: balance=%d, needed=%d%n", e.getBalance(), e.getRequired()); } catch (AuthenticationException e) { System.out.println("invalid API key — get a fresh one at anakin.io/dashboard"); } catch (RateLimitException e) { System.out.printf("rate limited; retry after %s%n", e.getRetryAfter()); } catch (JobFailedException e) { System.out.printf("job failed: %s%n", e.getReason()); } catch (AnakinException e) { System.out.printf("unknown error: %s%n", e.getMessage()); } ``` The exception hierarchy: | Type | When | |---|---| | `AuthenticationException` | 401 — invalid or missing API key | | `InsufficientCreditsException` | 402 — out of credits (`getBalance()`, `getRequired()`) | | `InvalidRequestException` | 400 — validation failure | | `RateLimitException` | 429 — after retry budget exhausted (`getRetryAfter()`) | | `JobFailedException` | Polled job came back with `status="failed"` (`getReason()`) | | `JobTimeoutException` | Polling budget exhausted before terminal status | | `ServerException` | 5xx — after retries exhausted | | `NetworkException` | DNS / connect / read-timeout | | `AnakinException` | Base type; everything above extends it | ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ```groovy implementation 'io.github.anakin-inc:anakin-sdk:0.1.0' ``` Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-java/issues). --- # Node.js SDK (/docs/sdks/node) Official Node.js / TypeScript SDK for the Anakin API. Wraps every documented endpoint with a single async call, full TypeScript types, and built-in polling. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Runtime** | Node.js 18+ | | **Package** | [`@anakin-io/sdk`](https://www.npmjs.com/package/@anakin-io/sdk) on npm | | **Source** | [github.com/Anakin-Inc/anakin-node](https://github.com/Anakin-Inc/anakin-node) | | **Bundle** | ESM + CJS, ~28 KB | | **License** | Apache 2.0 | --- ### Install ```bash npm install @anakin-io/sdk # or pnpm add @anakin-io/sdk # or yarn add @anakin-io/sdk ``` ### Quickstart ```typescript import { Anakin } from '@anakin-io/sdk' const client = new Anakin({ apiKey: 'ak-...' }) // or set ANAKIN_API_KEY env var // Scrape a single URL — returns the final result, no polling required const doc = await client.scrape('https://example.com', { formats: ['markdown'] }) console.log(doc.markdown) // Discover URLs on a site const sitemap = await client.map('https://example.com', { limit: 200 }) console.log(sitemap.links) // Crawl pages and get content for each const crawl = await client.crawl('https://example.com', { maxPages: 20 }) for (const page of crawl.pages) { console.log(page.url, page.markdown?.length ?? 0) } ``` ### What's in v0.1 | Method | Returns | |---|---| | `client.scrape(url, opts)` | `Promise<Document>` | | `client.map(url, opts)` | `Promise<MapResult>` | | `client.crawl(url, opts)` | `Promise<CrawlResult>` | | `client.search(prompt, opts)` | `Promise<SearchResult>` (synchronous API) | | `client.agenticSearch(prompt, opts)` | `Promise<AgenticSearchResult>` | | `client.wire(actionId, params, opts)` | `Promise<WireResult>` (run a [Wire](/docs/api-reference/wire) action) | | `client.sessions.list / .create / .save / .update / .delete` | Browser session CRUD | | `client.countries()` | `Country[]` (static, bundled with the SDK) | ### Configuration ```typescript const client = new Anakin({ apiKey: 'ak-...', // or ANAKIN_API_KEY env var timeoutMs: 60_000, // per-request HTTP timeout maxRetries: 4, // retries on 429 / 5xx pollIntervalMs: 1_000, // initial polling delay pollMaxIntervalMs: 10_000, // cap on exponential backoff pollTimeoutMs: 300_000, // total wait before JobTimeoutError }) ``` ### Errors ```typescript import { AnakinError, // base for everything below AuthenticationError, // bad/missing API key InsufficientCreditsError, // 402 — exposes .balance, .required InvalidRequestError, // 400 JobFailedError, // job came back with status="failed" JobTimeoutError, // pollTimeoutMs exceeded RateLimitError, // 429 — exposes .retryAfter ServerError, // 5xx after retries NetworkError, // DNS / connection / timeout WireAuthRequiredError, // Wire action needs account connection (.connectUrl) } from '@anakin-io/sdk' ``` ### Examples The [`examples/`](https://github.com/Anakin-Inc/anakin-node/tree/main/examples) folder has copy-paste-able scripts: - `quickstart.mjs` — scrape, map, crawl, search, wire — five recipes - `agentic-extraction.mjs` — multi-stage AI search with a custom JSON schema ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ```json { "dependencies": { "@anakin-io/sdk": "0.1.0" } } ``` Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-node/issues). --- # PHP SDK (/docs/sdks/php) Official PHP SDK for the Anakin API. Wraps every documented endpoint with a single sync call, internal polling, and a typed exception hierarchy. Built on Guzzle 7. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Language** | PHP 8.1+ (with `ext-curl` and `ext-json`) | | **Package** | [`anakin/sdk`](https://packagist.org/packages/anakin/sdk) on Packagist | | **Source** | [github.com/Anakin-Inc/anakin-php](https://github.com/Anakin-Inc/anakin-php) | | **License** | Apache 2.0 | --- ### Install ```bash composer require anakin/sdk ``` ### Quickstart ```php <?php require 'vendor/autoload.php'; use Anakin\Client; $client = new Client(['api_key' => 'ak-...']); // or set ANAKIN_API_KEY env var // Scrape a single URL — returns the final result, no polling required $doc = $client->scrape('https://example.com'); echo $doc['markdown']; // Discover URLs on a site $sitemap = $client->map('https://example.com', ['limit' => 200]); print_r($sitemap['links']); // Crawl pages and get content for each $crawl = $client->crawl('https://example.com', ['max_pages' => 20]); foreach ($crawl['pages'] as $page) { echo $page['url'] . ': ' . strlen($page['markdown'] ?? '') . "\n"; } ``` ### What's in v0.1 | Method | Returns | |---|---| | `$client->scrape($url, $opts)` | `array` — the document | | `$client->map($url, $opts)` | `array` — discovered links | | `$client->crawl($url, $opts)` | `array` — crawled pages | | `$client->search($query, $opts)` | `array` — search results (synchronous API) | | `$client->agenticSearch($prompt, $opts)` | `array` — AI-synthesised answer | | `$client->wire($actionId, $params)` | `array` — Wire action result (run a [Wire](/docs/api-reference/wire) action) | | `$client->sessions()->list / create / save / update / delete` | Browser session CRUD | All response methods return associative arrays (the parsed JSON response). Strong typing via [Psalm](https://psalm.dev) and [PHPStan](https://phpstan.org) generics is on the v0.2 roadmap. ### Configuration ```php $client = new Anakin\Client([ 'api_key' => 'ak-...', // or ANAKIN_API_KEY env var 'base_url' => 'https://api.anakin.io/v1', 'timeout' => 60, // per-request HTTP timeout (seconds) 'max_retries' => 4, // retries on 429 / 5xx 'poll_interval' => 1.0, // initial polling delay 'poll_max_interval' => 10.0, // cap on exponential backoff 'poll_timeout' => 300, // total wait before JobTimeoutException 'http_client' => $myGuzzleClient, // optional — inject your own PSR-18 client ]); ``` ### Errors ```php use Anakin\Exception\{ AnakinException, // base for everything below AuthenticationException, // bad/missing API key InsufficientCreditsException, // 402 — exposes ->balance, ->required InvalidRequestException, // 400 JobFailedException, // job came back with status="failed" JobTimeoutException, // poll_timeout exceeded RateLimitException, // 429 — exposes ->retryAfter ServerException, // 5xx after retries NetworkException, // DNS / connection / timeout }; try { $doc = $client->scrape('https://example.com'); } catch (InsufficientCreditsException $e) { echo "out of credits: balance={$e->balance}, needed={$e->required}"; } catch (RateLimitException $e) { echo "rate limited; retry after {$e->retryAfter}s"; } catch (AnakinException $e) { echo "anakin error: {$e->getMessage()}"; } ``` ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ```json { "require": { "anakin/sdk": "0.1.0" } } ``` Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-php/issues). --- # Python SDK (/docs/sdks/python) Official Python SDK for the Anakin API. Wraps every documented endpoint with a single sync call, typed Pydantic v2 response models, and built-in polling. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Language** | Python 3.10+ | | **Package** | [`anakin-sdk`](https://pypi.org/project/anakin-sdk/) on PyPI | | **Import name** | `anakin` (so you `pip install anakin-sdk` but `from anakin import Anakin`) | | **Source** | [github.com/Anakin-Inc/anakin-py](https://github.com/Anakin-Inc/anakin-py) | | **License** | Apache 2.0 | --- ### Install ```bash pip install anakin-sdk ``` ### Quickstart ```python from anakin import Anakin client = Anakin(api_key="ak-...") # or set ANAKIN_API_KEY env var # Scrape a single URL — returns the final result, no polling required doc = client.scrape("https://example.com", formats=["markdown"]) print(doc.markdown) # Discover URLs on a site sitemap = client.map("https://example.com", limit=200) print(sitemap.links) # Crawl pages and get content for each crawl = client.crawl("https://example.com", max_pages=20) for page in crawl.pages: print(page.url, len(page.markdown or "")) ``` ### What's in v0.1 | Method | Returns | |---|---| | `client.scrape(url, ...)` | `Document` | | `client.map(url, ...)` | `MapResult` | | `client.crawl(url, ...)` | `CrawlResult` | | `client.search(prompt, ...)` | `SearchResult` (synchronous API) | | `client.agentic_search(prompt, ...)` | `AgenticSearchResult` | | `client.wire(action_id, params, ...)` | `WireResult` (run a [Wire](/docs/api-reference/wire) action) | | `client.sessions.list / .create / .save / .update / .delete` | Browser session CRUD | | `client.countries()` | `list[Country]` (static, bundled with the SDK) | ### Configuration ```python client = Anakin( api_key="ak-...", # or ANAKIN_API_KEY env var timeout=60.0, # per-request HTTP timeout (seconds) max_retries=4, # retries on 429 / 5xx poll_interval=1.0, # initial polling delay poll_max_interval=10.0, # cap on exponential backoff poll_timeout=300.0, # total wait before JobTimeoutError ) ``` ### Errors ```python from anakin import ( AnakinError, # base for everything below AuthenticationError, # bad/missing API key InsufficientCreditsError, # 402 — exposes .balance, .required InvalidRequestError, # 400 JobFailedError, # job came back with status="failed" JobTimeoutError, # poll_timeout exceeded RateLimitError, # 429 — exposes .retry_after ServerError, # 5xx after retries NetworkError, # DNS / connection / timeout WireAuthRequiredError, # Wire action needs account connection (.connect_url) ) ``` ### Examples The [`examples/`](https://github.com/Anakin-Inc/anakin-py/tree/main/examples) folder in the repo has copy-paste-able scripts for the common workflows: - `quickstart.py` — scrape, map, crawl, search, wire — five recipes in one file - `agentic_extraction.py` — multi-stage AI search with a custom JSON schema ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ``` anakin==0.1.0 ``` Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-py/issues). --- # Releases (/docs/sdks/releases) Current versions of all published Anakin packages. For full release notes see [/docs/releases](/docs/releases). | Package | Registry | Latest | Install | |---|---|---|---| | `@anakin-io/sdk` | npm | `0.1.0` | `npm install @anakin-io/sdk` | | `@anakin-io/mcp` | npm | `0.1.0` | `npx -y @anakin-io/mcp init --all` | | `anakin-sdk` | PyPI | `0.1.0` | `pip install anakin-sdk` | | `anakin-cli` | PyPI | `0.2.0` | `pip install anakin-cli` | | `anakin-sdk` | crates.io | `0.1.0` | `cargo add anakin-sdk` | | `anakin-sdk` | RubyGems | `0.1.0` | `gem install anakin-sdk` | | `anakin/sdk` | Packagist | `0.1.0` | `composer require anakin/sdk` | | `anakin` | Hex | `0.1.0` | `{:anakin, "~> 0.1"}` | | `github.com/Anakin-Inc/anakin-go` | pkg.go.dev | `v0.1.0` | `go get github.com/Anakin-Inc/anakin-go` | | `io.github.anakin-inc:anakin-sdk` | Maven Central | `0.1.0` | see [Java SDK](/docs/sdks/java) | | `Anakin` | NuGet | coming soon | see [.NET SDK](/docs/sdks/dotnet) | Full changelog with release notes → [Releases](/docs/releases) --- # Ruby SDK (/docs/sdks/ruby) Official Ruby gem for the Anakin API. Wraps every documented endpoint with a single sync call, internal polling, and a typed error hierarchy. Pure stdlib — no runtime dependencies. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Language** | Ruby 2.7+ | | **Gem name** | [`anakin-sdk`](https://rubygems.org/gems/anakin-sdk) on RubyGems | | **Source** | [github.com/Anakin-Inc/anakin-ruby](https://github.com/Anakin-Inc/anakin-ruby) | | **License** | Apache 2.0 | --- ### Install ```bash gem install anakin-sdk ``` Or in your `Gemfile`: ```ruby gem 'anakin-sdk', '~> 0.1' ``` ### Quickstart ```ruby require 'anakin' client = Anakin.new(api_key: 'ak-...') # or set ANAKIN_API_KEY env var # Scrape a single URL — returns the final result, no polling required doc = client.scrape('https://example.com') puts doc['markdown'] # Discover URLs on a site sitemap = client.map('https://example.com', limit: 200) puts sitemap['links'] # Crawl pages and get content for each crawl = client.crawl('https://example.com', max_pages: 20) crawl['pages'].each do |page| puts "#{page['url']}: #{(page['markdown'] || '').length}" end ``` ### What's in v0.1 | Method | Returns | |---|---| | `client.scrape(url, **opts)` | `Hash` — the document | | `client.map(url, **opts)` | `Hash` — discovered links | | `client.crawl(url, **opts)` | `Hash` — crawled pages | | `client.search(query, **opts)` | `Hash` — search results (synchronous API) | | `client.agentic_search(prompt, **opts)` | `Hash` — AI-synthesised answer | | `client.wire(action_id, params)` | `Hash` — Wire action result (run a [Wire](/docs/api-reference/wire) action) | | `client.sessions.list / create / save / update / delete` | Browser session CRUD | All response methods return parsed JSON as Ruby `Hash`/`Array`. Strict struct types via `Anakin::Document`, `Anakin::CrawlResult` etc. are on the v0.2 roadmap. ### Configuration ```ruby client = Anakin.new( api_key: 'ak-...', # or ANAKIN_API_KEY env var base_url: 'https://api.anakin.io/v1', timeout: 60, # per-request HTTP timeout (seconds) max_retries: 4, # retries on 429 / 5xx poll_interval: 1.0, # initial polling delay poll_max_interval: 10.0, # cap on exponential backoff poll_timeout: 300, # total wait before JobTimeoutError ) ``` ### Errors ```ruby require 'anakin' begin doc = client.scrape('https://example.com') rescue Anakin::InsufficientCreditsError => e puts "out of credits: balance=#{e.balance}, needed=#{e.required}" rescue Anakin::AuthenticationError puts "invalid API key — get a fresh one at anakin.io/dashboard" rescue Anakin::RateLimitError => e puts "rate limited; retry after #{e.retry_after}s" rescue Anakin::JobFailedError => e puts "job failed: #{e.reason}" rescue Anakin::Error => e puts "anakin error: #{e.message}" end ``` The hierarchy: | Class | When | |---|---| | `Anakin::AuthenticationError` | 401 — invalid or missing API key | | `Anakin::InsufficientCreditsError` | 402 — out of credits (`#balance`, `#required`) | | `Anakin::InvalidRequestError` | 400 — validation failure | | `Anakin::RateLimitError` | 429 — after retry budget exhausted (`#retry_after`) | | `Anakin::JobFailedError` | Polled job came back with `status="failed"` (`#reason`) | | `Anakin::JobTimeoutError` | Polling budget exhausted before terminal status | | `Anakin::ServerError` | 5xx — after retries exhausted | | `Anakin::NetworkError` | DNS / connect / read-timeout | | `Anakin::Error` | Base class; everything above inherits from it | ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ```ruby gem 'anakin-sdk', '0.1.0' ``` Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-ruby/issues). --- # Rust SDK (/docs/sdks/rust) Official Rust crate for the Anakin API. Wraps every documented endpoint with a single async call, internal polling, and a strongly typed `Error` enum. Built on `reqwest` (rustls-tls) and `tokio`. | | | |---|---| | **Status** | Alpha (v0.1.x) | | **Language** | Rust 1.70+ | | **Crate** | [`anakin-sdk`](https://crates.io/crates/anakin-sdk) on crates.io | | **Docs** | [docs.rs/anakin-sdk](https://docs.rs/anakin-sdk) (auto-generated) | | **Source** | [github.com/Anakin-Inc/anakin-rust](https://github.com/Anakin-Inc/anakin-rust) | | **License** | Apache 2.0 | --- ### Install Add to your `Cargo.toml`: ```toml [dependencies] anakin-sdk = "0.1" tokio = { version = "1", features = ["full"] } ``` The import name is `anakin` (crate name on crates.io is `anakin-sdk`). ### Quickstart ```rust use anakin::Client; #[tokio::main] async fn main() -> anakin::Result<()> { let client = Client::builder() .api_key("ak-...") // or set ANAKIN_API_KEY .build()?; // Scrape a single URL — returns the final result, no polling required let doc = client.scrape("https://example.com").await?; println!("{}", doc.markdown.unwrap_or_default()); // Discover URLs on a site let sitemap = client.map("https://example.com").await?; println!("{:?}", sitemap.links); // Crawl pages and get content for each let crawl = client.crawl("https://example.com").await?; for page in &crawl.pages { println!("{}: {} chars", page.url.as_deref().unwrap_or(""), page.markdown.as_deref().unwrap_or("").len()); } Ok(()) } ``` ### What's in v0.1 | Method | Returns | |---|---| | `client.scrape(url)` / `scrape_with(url, opts)` | `Result<Document>` | | `client.map(url)` / `map_with(url, opts)` | `Result<MapResult>` | | `client.crawl(url)` / `crawl_with(url, opts)` | `Result<CrawlResult>` | | `client.search(query)` / `search_with(query, opts)` | `Result<SearchResult>` (synchronous API) | | `client.agentic_search(prompt)` / `agentic_search_with(...)` | `Result<AgenticSearchResult>` | | `client.wire(action_id, params)` | `Result<WireResult>` (run a [Wire](/docs/api-reference/wire) action) | | `client.sessions().list / create / save / update / delete` | Browser session CRUD | | `anakin::supported_countries()` | `&'static [(&'static str, &'static str)]` (bundled, no network call) | Extra opts can be passed as `Some(serde_json::json!({ "key": "value" }))`. ### Configuration ```rust use std::time::Duration; let client = anakin::Client::builder() .api_key("ak-...") // or ANAKIN_API_KEY env var .base_url("https://api.anakin.io/v1") .timeout(Duration::from_secs(60)) // per-request HTTP timeout .max_retries(4) // retries on 429 / 5xx .poll_interval(Duration::from_secs(1)) // initial polling delay .poll_max_interval(Duration::from_secs(10)) // cap on exponential backoff .poll_timeout(Duration::from_secs(300)) // total wait before JobTimeout .build()?; ``` You can also inject a pre-built `reqwest::Client` via `.http_client(c)` for custom proxy or TLS configuration. ### Errors All errors come back as variants of `anakin::Error`: ```rust use anakin::Error; match client.scrape("https://example.com").await { Ok(doc) => println!("{}", doc.markdown.unwrap_or_default()), Err(Error::InsufficientCredits { balance, required, .. }) => { eprintln!("out of credits: balance={balance}, needed={required}"); } Err(Error::Authentication { .. }) => { eprintln!("invalid API key — get a fresh one at anakin.io/dashboard"); } Err(Error::RateLimit { retry_after, .. }) => { eprintln!("rate limited; retry after {retry_after:?}"); } Err(Error::JobFailed { reason, .. }) => { eprintln!("job failed: {reason}"); } Err(e) => eprintln!("unknown error: {e}"), } ``` The error enum: | Variant | When | |---|---| | `Error::Authentication` | 401 — invalid or missing API key | | `Error::InsufficientCredits` | 402 — out of credits (`balance`, `required`) | | `Error::InvalidRequest` | 400 — validation failure | | `Error::RateLimit` | 429 — after retry budget exhausted (`retry_after`) | | `Error::JobFailed` | Polled job came back with `status="failed"` (`reason`) | | `Error::JobTimeout` | Polling budget exhausted before terminal status | | `Error::Server` | 5xx — after retries exhausted | | `Error::Network` | DNS / connect / read-timeout | | `Error::Other` | Decoding failures, missing fields | ### Stability `v0.1.x` is alpha. The public API may change between minor versions until v1.0. Pin a specific version in production: ```toml anakin-sdk = "=0.1.0" ``` Full reference docs and examples are on [docs.rs](https://docs.rs/anakin-sdk). Raise issues on [GitHub](https://github.com/Anakin-Inc/anakin-rust/issues). --- # CLI Commands (/docs/sdks/cli/commands) All available commands for the Anakin CLI. --- ### login Save your API key for future sessions. ```bash anakin login --api-key "ak-your-key-here" ``` | Flag | Description | |------|-------------| | `--api-key` | Your AnakinScraper API key | --- ### status Check the CLI version and whether you're authenticated. ```bash anakin status ``` --- ### scrape Scrape a single URL. Returns clean markdown by default. ```bash anakin scrape "https://example.com" ``` | Flag | Type | Description | Default | |------|------|-------------|---------| | `--format` | string | Output format: `markdown`, `json`, or `raw` | `markdown` | | `--browser` | flag | Use headless browser for JS-heavy sites | off | | `--country` | string | Two-letter country code for geo-located scraping | `us` | | `--session-id` | string | Browser session ID for authenticated scraping | — | | `--timeout` | number | Polling timeout in seconds | `120` | | `-o, --output` | string | Save output to a file instead of stdout | stdout | #### Output formats | Format | What you get | Best for | |--------|-------------|----------| | `markdown` | Clean readable page text | Reading, LLM context | | `json` | AI-extracted structured data | Data pipelines | | `raw` | Full API response (HTML, metadata, everything) | Debugging | #### Examples ```bash # Default markdown output anakin scrape "https://example.com" # Save to file anakin scrape "https://example.com" -o page.md # Extract structured JSON anakin scrape "https://example.com/product" --format json -o product.json # Full raw API response anakin scrape "https://example.com" --format raw -o debug.json # JavaScript-heavy site anakin scrape "https://example.com/spa" --browser # Scrape from the UK anakin scrape "https://example.com" --country gb # Longer timeout for slow sites anakin scrape "https://example.com" --timeout 300 # Authenticated scraping with a saved browser session anakin scrape "https://example.com/dashboard" --session-id "session_abc123" ``` --- ### scrape-batch Scrape up to 10 URLs simultaneously. All URLs are processed in parallel. ```bash anakin scrape-batch "https://a.com" "https://b.com" "https://c.com" ``` | Flag | Type | Description | Default | |------|------|-------------|---------| | `-o, --output` | string | Save output to a file | stdout | #### Examples ```bash # Scrape 3 URLs anakin scrape-batch "https://a.com" "https://b.com" "https://c.com" # Save batch results to file anakin scrape-batch "https://a.com" "https://b.com" -o results.json ``` --- ### search AI-powered web search. Returns results instantly (synchronous). ```bash anakin search "your search query" ``` | Flag | Type | Description | Default | |------|------|-------------|---------| | `-o, --output` | string | Save output to a file | stdout | #### Examples ```bash # Search the web anakin search "python async best practices" # Save search results anakin search "best web scraping tools 2025" -o results.json # Pipe to jq anakin search "latest AI news" | jq '.results[0]' ``` --- ### research Deep agentic research. Runs a multi-stage pipeline: query refinement, web search, citation scraping, and AI synthesis. Takes 1–5 minutes. ```bash anakin research "your research topic" ``` | Flag | Type | Description | Default | |------|------|-------------|---------| | `-o, --output` | string | Save output to a file | stdout | #### Examples ```bash # Run deep research anakin research "comparison of web frameworks 2025" # Save research report anakin research "quantum computing industry trends" -o report.json ``` --- ## Error handling The CLI provides clear error messages: | Error | Code | Fix | |-------|------|-----| | Authentication failed | 401 | Run `anakin login --api-key "ak-xxx"` | | Plan upgrade required | 402 | Visit [Pricing](/docs/documentation/pricing) | | Rate limit exceeded | 429 | Wait a few seconds and retry | | Job timed out | — | Increase with `--timeout 300` | | Job failed | — | Check if the URL is accessible | **Exit codes:** `0` for success, `1` for any error. --- # CLI Examples (/docs/sdks/cli/examples) Real-world usage patterns for the Anakin CLI. --- ### Scrape a page to markdown The most common use case — get clean, readable content from any URL: ```bash anakin scrape "https://docs.python.org/3/tutorial/index.html" -o tutorial.md ``` --- ### Extract structured data with AI Use `--format json` to get AI-extracted structured data instead of raw text: ```bash anakin scrape "https://amazon.com/dp/B0EXAMPLE" --format json -o product.json ``` The AI analyzes the page and returns structured fields like title, price, description, etc. --- ### Scrape a JavaScript-heavy site For SPAs, React/Next.js sites, or pages with dynamically loaded content: ```bash anakin scrape "https://app.example.com/dashboard" --browser ``` The `--browser` flag launches a headless browser to render JavaScript before extracting content. --- ### Batch scrape multiple pages Scrape up to 10 URLs in a single command. All are processed in parallel: ```bash anakin scrape-batch \ "https://example.com/page-1" \ "https://example.com/page-2" \ "https://example.com/page-3" \ "https://example.com/page-4" \ "https://example.com/page-5" \ -o pages.json ``` --- ### Scrape from a specific country Route your request through a proxy in a specific country. Useful for geo-restricted content: ```bash # Scrape from the UK anakin scrape "https://example.co.uk/deals" --country gb # Scrape from Japan anakin scrape "https://example.jp/products" --country jp ``` See the full list of [207 supported countries](/docs/api-reference/supported-countries). --- ### Scrape authenticated pages Use a saved browser session to scrape pages that require login: ```bash # First, create a session from the dashboard at anakin.io/dashboard # Then use the session ID: anakin scrape "https://example.com/my-account" --session-id "session_abc123" ``` Learn more about [Browser Sessions](/docs/api-reference/browser-sessions). --- ### Pipe output to other tools Progress messages go to stderr, so piping works cleanly: ```bash # Extract a specific field with jq anakin scrape "https://example.com" --format json | jq '.title' # Count words in scraped markdown anakin scrape "https://example.com" | wc -w # Feed into another script anakin search "latest AI papers" | python process_results.py ``` --- ### Research a topic Run deep multi-stage research that scrapes and synthesizes multiple sources: ```bash anakin research "best practices for web scraping in 2025" -o research.json ``` This takes 1–5 minutes and produces a comprehensive report with citations. --- ### Use in shell scripts ```bash #!/bin/bash # scrape-urls.sh — Scrape a list of URLs from a file while IFS= read -r url; do filename=$(echo "$url" | sed 's|https\?://||;s|/|_|g').md echo "Scraping: $url -> $filename" >&2 anakin scrape "$url" -o "$filename" done < urls.txt ``` --- ### Debug a failing scrape Use `--format raw` to see the full API response including headers, status codes, and error details: ```bash anakin scrape "https://example.com" --format raw -o debug.json ``` If the default HTTP handler fails, try with `--browser` to use the headless browser: ```bash anakin scrape "https://example.com" --browser --format raw -o debug.json ``` --- # Integrations (/docs/integrations) Use AnakinScraper inside your IDE, AI frameworks, and workflow automation platforms. --- ### Plugins & Skills Use AnakinScraper inside your AI agent or code editor. Scrape, search, and research without leaving your environment. | Integration | Status | Description | |-------------|--------|-------------| | [Claude Code](/docs/integrations/ide-plugins/claude-code) | Available | Plugin for Claude Code with skills, agents, and hooks | | [Cursor](/docs/integrations/ide-plugins/cursor) | Available | Plugin for Cursor with rules, skills, and agents | | [OpenClaw](/docs/integrations/ide-plugins/openclaw) | Available | Skill for OpenClaw AI agents on ClawHub | --- ### AI Frameworks Integrate AnakinScraper into AI application frameworks. | Integration | Status | Description | |-------------|--------|-------------| | [Google ADK](/docs/integrations/ai-frameworks/google-adk) | Available | ADK tools for scraping, search, and research in Gemini agents | | [LangChain](/docs/integrations/ai-frameworks/langchain) | Coming Soon | Document Loader, Tools, and Retriever for scraping and search | | [LlamaIndex](/docs/integrations/ai-frameworks/llamaindex) | Coming Soon | Reader and Tools for RAG pipelines and agents | | [CrewAI](/docs/integrations/ai-frameworks/crewai) | Coming Soon | Tools for multi-agent scraping, search, and research | | [Langflow](/docs/integrations/ai-frameworks/langflow) | Coming Soon | Visual drag-and-drop components for scraping and search | | [Flowise](/docs/integrations/ai-frameworks/flowise) | Coming Soon | Visual chatflow nodes for scraping and search | --- ### Workflow Automation Trigger scrapes, searches, and research from your automation workflows. | Integration | Status | Description | |-------------|--------|-------------| | [Dify](/docs/integrations/workflow/dify) | Available | Plugin with 5 tools for Dify AI workflows | | [n8n](/docs/integrations/workflow/n8n) | Coming Soon | Community node with scrape, search, and agentic search | | [Zapier](/docs/integrations/workflow/zapier) | Available | 4 actions: scrape, AI search, agentic search, get results | | [Make](/docs/integrations/workflow/make) | Available | 4 native modules: scrape, poll, agentic search, AI search | --- # Claude Connector (/docs/integrations/ai-agents/claude-connector) The Anakin connector gives Claude live web-data tools over remote [MCP](https://modelcontextprotocol.io). Connect by URL and authorize with OAuth 2.1 — you never paste an API key. Anakin mints a dedicated, revocable key scoped to your account. > **Server URL:** `https://mcp.anakin.io/mcp` · Transport: Streamable HTTP · Auth: OAuth 2.1 (Dynamic Client Registration). --- ## Requirements - An Anakin account with available credits — get one free at the [Dashboard](/dashboard) (300 credits, no card required). Tool calls consume credits. - Claude.ai (Connectors), Claude Desktop (custom connector), or the Anthropic API MCP connector. --- ## Connect 1. In Claude, add a custom connector with the URL **`https://mcp.anakin.io/mcp`**. 2. Claude opens an Anakin sign-in page. Log in (Google or email/password). 3. Approve the consent screen. Anakin grants a dedicated **Claude (MCP)** key scoped to your account. 4. The tools below become available. From Claude Code: ```bash claude mcp add --transport http anakin https://mcp.anakin.io/mcp ``` --- ## Tools The connector exposes 21 tools across the Anakin product surface. **Read** tools are auto-permitted; **write** tools (marked below) are annotated as state-changing, and your MCP client asks for your confirmation before each call. The connector never executes payments or transfers funds — such actions are refused. | Tool | Purpose | |---|---| | `scrape` | Fetch one URL as clean markdown, optionally AI-structured JSON; optional headless browser for SPAs. | | `search` | AI web search returning URLs, titles, and snippets. | | `map` | Discover reachable URLs under a site. | | `crawl` | Bulk-fetch markdown across many pages (scope with include/exclude patterns). | | `agentic_search` | Multi-source deep research returning a summary plus structured data. | | `wire_discover` | Find catalog actions for a natural-language intent. | | `wire_catalog` | Browse supported sites and their actions/parameters. | | `wire_read_action` | Run a data-extraction Wire action against a specific site. | | `wire_write_action` | **Write.** Run a state-changing Wire action on your own accounts (submit a form, update a listing). | | `wire_identities` | List your own saved site sign-ins (created from your Anakin dashboard). | | `wire_login` | **Write.** Sign in to a supported site — the password is never stored, only an encrypted session. | | `wire_build` | **Write.** Request a brand-new action for a site not yet in the catalog. | | `monitor_create` | **Write.** Watch a page, site, or Wire action on a schedule and alert on change (webhook/email). | | `monitor_list` | List your website monitors and their status. | | `monitor_changes` | Read the changes a monitor has detected. | | `monitor_control` | **Write.** Pause, resume, run now, or delete a monitor. | | `ai_visibility_search` | Ask ChatGPT, Gemini, and Google AI Overview the same question and compare their answers. | | `ai_visibility_sources` | List the available AI answer engines. | | `session_list` | List your saved browser login sessions (create them in the [Dashboard](/dashboard)). | | `session_delete` | **Write.** Permanently delete a saved browser session. | | `browser_task` | **Write.** Run a natural-language task in an AI-driven cloud browser (multi-step navigation and extraction). Never accepts passwords — use a saved session for authenticated tasks. | --- ## Example prompts ``` Scrape https://example.com/pricing and give me the plans as a table. ``` ``` Search for the latest EU AI Act compliance deadlines and summarize the key dates. ``` ``` Use a Wire action to get the top products in Walmart's electronics category and list their prices. ``` ``` Create a monitor on https://example.com/pricing that checks every hour and emails me when the plans change. ``` ``` Ask ChatGPT, Gemini, and Google AI Overview what they say about Anakin and compare their answers. ``` --- ## Limits and costs - Tool results are capped at roughly 100,000 characters to fit client limits; narrow large requests (fewer pages, include/exclude patterns). - Calls consume Anakin credits — see [Pricing](/pricing). - The connector does not execute payments or transfer funds; payment-style Wire actions are refused. --- ## Privacy and security OAuth 2.1 with a per-user encrypted key, no access to your Claude memory/chat/files, and no model training on your data. See the [Connectors and Plugins Privacy Policy](/connector-privacy). --- ## Support Email [support@anakin.io](mailto:support@anakin.io) · status at [/status](/status). --- # MCP Server (/docs/integrations/ai-agents/mcp-server) The [`@anakin-io/mcp`](https://www.npmjs.com/package/@anakin-io/mcp) package is an [MCP](https://modelcontextprotocol.io) server that exposes six Anakin tools — `scrape`, `search`, `map`, `crawl`, `agentic_search`, `wire_action` — to any MCP-compatible agent client. With it installed, Claude Desktop, Cursor, etc. can fetch web pages, run AI search, crawl sites, and execute Wire actions directly from a chat or coding session. > **Status: alpha (v0.1.x).** Tool surface and arguments may change between minor versions until v1.0. > Package: [`@anakin-io/mcp` on npm](https://www.npmjs.com/package/@anakin-io/mcp) · Source: [github.com/Anakin-Inc/anakin-mcp](https://github.com/Anakin-Inc/anakin-mcp). --- ## Quick install (recommended) One command auto-configures every detected agent client on your machine: ```bash npx -y @anakin-io/mcp init --all ``` You'll be prompted for your API key (or set `ANAKIN_API_KEY` first to skip the prompt). Get one free at the [Dashboard](/dashboard) — 300 credits, no card required. After it finishes, **restart your agent client(s)**. The `anakin` MCP server appears in the tool list, exposing six tools. If you'd rather configure one specific client: ```bash npx -y @anakin-io/mcp init --client=cursor ``` Available client names: `claude-desktop`, `claude-code`, `cursor`, `cline`, `continue`, `zed`, `windsurf`, `vscode`. --- ## What gets exposed Each tool is a thin wrapper around the matching Anakin REST endpoint. Agents see typed input schemas (so they know what arguments each tool takes) and get back a string payload they can quote in their replies. | Tool | Purpose | Async? | |---|---|---| | `scrape` | Fetch one URL → markdown by default. Set `generateJson: true` for AI-extracted structured data. Set `useBrowser: true` for SPAs. | async (3–15s typical) | | `search` | AI web search with citations. Returns URL + title + snippet for each hit. | sync | | `map` | Discover all reachable URLs on a domain. | async | | `crawl` | Bulk-fetch markdown across a site (with include / exclude patterns). | async | | `agentic_search` | Multi-source deep research. Searches the web, scrapes citations, structures the combined data with an LLM. | async (1–5 min) | | `wire_action` | Execute a pre-built website action via [Wire](/docs/api-reference/wire) (login flows, form fills, etc.). | async | --- ## Manual setup per client If you'd rather edit config files yourself, copy the right snippet for your client. **Set your real API key** in `env.ANAKIN_API_KEY` — the agent never sees it; it lives only in this local config file. ### Claude Desktop | Platform | Path | |---|---| | macOS | `~/Library/Application Support/Claude/claude_desktop_config.json` | | Windows | `%APPDATA%/Claude/claude_desktop_config.json` | ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Restart Claude Desktop. ### Claude Code (Anthropic CLI) Edit `~/.claude/settings.json`: ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Or use Claude Code's built-in command: ```bash claude mcp add anakin npx -y @anakin-io/mcp -e ANAKIN_API_KEY=ak-... ``` ### Cursor Edit `~/.cursor/mcp.json` (user-scoped) or `./.cursor/mcp.json` (project-scoped): ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Restart Cursor. ### Cline (VS Code extension) | Platform | Path | |---|---| | macOS | `~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json` | | Linux | `~/.config/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json` | | Windows | `%APPDATA%/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json` | ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Reload the Cline extension (or restart VS Code). ### Continue (IDE extension) Edit `~/.continue/config.json`: ```json { "experimental": { "modelContextProtocolServers": [ { "name": "anakin", "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } ] } } ``` If you have other entries under `experimental.modelContextProtocolServers`, append the `anakin` object — don't overwrite. Reload the Continue extension. ### Zed | Platform | Path | |---|---| | macOS | `~/.config/zed/settings.json` | | Linux | `~/.config/zed/settings.json` | | Windows | `%APPDATA%/Zed/settings.json` | ```json { "context_servers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Reload Zed. ### Windsurf Edit `~/.codeium/windsurf/mcp_config.json`: ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Restart Windsurf. ### VS Code (with the MCP extension) Edit `.vscode/mcp.json` in your workspace: ```json { "servers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Reload VS Code. --- ## Verify it works In Claude Desktop / Cursor / etc., start a new chat and try: > Use anakin to scrape https://example.com and return the markdown. The agent should: 1. Find the `scrape` tool in `anakin`'s tool list 2. Call `scrape({ url: "https://example.com" })` 3. Quote back the rendered markdown You can also try other tools: > Use anakin's search to find recent articles about MCP servers. > Use anakin to map the URLs on https://docs.anakin.io and pick three to scrape. > Use anakin's agentic_search for a comparative analysis of TypeScript HTTP libraries. --- ## How the API key flows Your key never enters the conversation context. It lives **only** in the agent client's local config file (e.g., `claude_desktop_config.json`), gets passed to the MCP server subprocess as an environment variable when the client starts, and from there is sent in the `X-API-Key` header on every Anakin REST request. If `ANAKIN_API_KEY` is unset or empty when the server starts, it exits with a message pointing at the [Dashboard](/dashboard). The agent client typically surfaces that error in its MCP servers panel; the agent itself just sees `anakin` missing from its tool list. If the key is invalid or you're out of credits, the first tool call returns an error (`Invalid API key` or `Insufficient credits`) which the agent reports back to you. Update the config and restart the client. --- ## Troubleshooting **The MCP servers panel shows `anakin` as "Failed"** → Click the entry to see the stderr output. Most common: `ANAKIN_API_KEY is not set` (fix: fill in `env.ANAKIN_API_KEY` and restart) or a JSON syntax error in your config (fix: validate with `cat <config> | jq .`). **`anakin` doesn't appear in the tool list at all** → The client hasn't reloaded the config. Quit fully (⌘Q on macOS) and reopen. **Tool call returns "Invalid API key"** → Key is wrong or revoked. Get a fresh one from the [Dashboard](/dashboard). **Tool call returns "Insufficient credits"** → Top up at the [Dashboard](/dashboard). Failed jobs aren't charged, so credits only deduct on successful operations. **`init --all` says "No supported MCP clients detected"** → The init command checks for the existence of each client's config directory. If a client is installed but you've never opened it (or you installed via an unusual method that doesn't create the standard dir), pass the client name explicitly: ```bash npx -y @anakin-io/mcp init --client=cursor ``` That always writes the config, creating any missing parent directories. --- ## Two paths agents can use Anakin This MCP server is one of two complementary integration paths: 1. **MCP** (this page) — for clients that support it. Tools are typed and called natively. Best when available. 2. **[SKILL.md](https://anakin.io/agent-onboarding/SKILL.md)** — for any agent that can fetch a URL. The markdown describes the API end-to-end so an agent can use it via plain HTTP calls. Works in environments without subprocess support. Most users on Claude Desktop / Cursor / Windsurf / VS Code will want path 1. Custom agents and CI agents can use path 2. --- ## Related - [`anakin-mcp` on GitHub](https://github.com/Anakin-Inc/anakin-mcp) — source, releases, issue tracker - [`@anakin-io/sdk`](https://github.com/Anakin-Inc/anakin-node) — Node.js / TypeScript SDK - [`anakin`](https://github.com/Anakin-Inc/anakin-py) — Python SDK - [`anakin-cli`](https://github.com/Anakin-Inc/anakin-cli) — Python CLI for terminal-driven use - [SKILL.md](https://anakin.io/agent-onboarding/SKILL.md) — agent-onboarding doc for non-MCP agents - [API reference](/docs/api-reference) — endpoint-by-endpoint REST docs --- # Agent Onboarding (SKILL.md) (/docs/integrations/ai-agents/skill-md) Anakin publishes a `SKILL.md` file at a permanent public URL. Any AI agent that supports tool discovery via a URL can load this file to understand Anakin's full capability surface — no manual configuration required. **Direct URL:** ``` https://anakin.io/agent-onboarding/SKILL.md ``` --- ### What it contains The file is a structured Markdown document covering: - What Anakin does (web scraping, crawling, AI search, Wire actions, browser sessions) - All available API endpoints with parameters and response shapes - Authentication instructions (`ANAKIN_API_KEY` environment variable or `X-API-Key` header) - Usage examples in plain language the agent can follow - Links to the full API reference and SDK docs ### Who it's for `SKILL.md` follows the emerging convention (popularised by Firecrawl, Exa, and others) of publishing a single machine-readable onboarding document that AI agents can fetch to self-configure. It works in any context where an agent can retrieve a URL: - **MCP clients** — reference it as a resource in your MCP config - **Custom agents** — fetch it at startup and inject it as system context - **AI coding assistants** — paste the URL into the system prompt or initial message - **LLM pipelines** — use it as a tool-definition source for function-calling setups ### Using it in a prompt ``` Before using any Anakin tools, read the onboarding document at: https://anakin.io/agent-onboarding/SKILL.md ``` The agent will fetch the file, understand what Anakin can do, and start using the correct endpoints and parameters without further instruction. ### Format The file is served as `Content-Type: text/markdown` with no authentication. It is always current — updated in sync with the API. --- # Zero Touch (/docs/integrations/ai-agents/zero-touch) **Zero Touch** is Anakin's no-API-key, no-signup free tier. An AI agent or developer goes from nothing to scraped data — or a live read-only site action — in a **single HTTP call with no auth header**, then upgrades in place by adding a key or bearer. It inverts the usual order: prove value first, sign up second. > **No auth header required.** The endpoints below accept requests with no `X-API-Key` and no `Authorization`. Metering is by a per-IP free allowance. --- ## When to use it - **Autonomous agents.** An agent can't complete a signup form or an OAuth consent screen unattended — but it can make an HTTP call or open an MCP connection. Zero Touch is the surface it can use on its own. - **First contact.** Try the API in one call before deciding to sign up — no account, no key, no SDK. - **Read-only work.** Scraping and read-only site actions are open. Writes, crawl, agentic search, and account-connected actions require a free account. --- ## Scrape — no key Send a URL, get clean Markdown back. No auth header. <EndpointCard method="POST" path="/v1/url-scraper/scrape" description="Scrape a single URL and get the result inline — no key, no polling" href="/docs/api-reference/url-scraper/scrape" /> <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -s https://api.anakin.io/v1/url-scraper/scrape \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` </Tab> <Tab value="Python"> ```python import requests r = requests.post( "https://api.anakin.io/v1/url-scraper/scrape", json={"url": "https://example.com"}, ) data = r.json() print(data["markdown"]) print(r.headers["X-Trial-Credits-Remaining"]) ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch("https://api.anakin.io/v1/url-scraper/scrape", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ url: "https://example.com" }), }); const data = await res.json(); console.log(data.markdown); console.log(res.headers.get("X-Trial-Credits-Remaining")); ``` </Tab> </Tabs> The response carries the scraped `markdown` plus a `trial` object describing your remaining free allowance: ```json { "markdown": "# Example Domain\n...", "trial": { "remaining_credits": 47, "signup_url": "https://anakin.io/signup" } } ``` The same information is mirrored in response headers so you don't have to parse the body: | Header | Meaning | |---|---| | `X-Trial-Credits-Remaining` | Free per-IP credits left. | | `X-Anakin-Signup` | Signup URL for higher limits. | --- ## Wire — read-only site actions, no account Wire turns supported sites into managed API actions. The **discovery** and **read** surface is open with no key: discover an action by intent, then run it. This is the same Wire documented in the [Wire API reference](/docs/api-reference/wire) — Zero Touch is its **synchronous, no-key** run mode (`POST /v1/wire-run`, result inline). The keyed, asynchronous mode (`POST /v1/wire/task` + poll) handles write actions and account-connected runs. <ZeroTouchWireEndpoints /> ```bash # 1. Find an action for your intent curl -s "https://api.anakin.io/v1/wire/resolve?q=search%20products%20on%20a%20marketplace" # 2. Run it — no auth header, no account connected curl -s https://api.anakin.io/v1/wire-run \ -H "Content-Type: application/json" \ -d '{"action_id": "act_search_products", "params": {"query": "wireless earbuds"}}' ``` > Zero Touch runs **read-only** Wire actions only, synchronously. State-changing (write) actions, account-connected runs, and durable async jobs use the keyed [`POST /v1/wire/task`](/docs/api-reference/wire/execute-task) flow with a signed-in Anakin account and a key. --- ## MCP — no OAuth Point any [MCP](https://modelcontextprotocol.io) client at Anakin with **no bearer token** to get the Zero Touch tool subset. There's no OAuth flow, so an agent can connect unattended. ```json { "mcpServers": { "anakin": { "type": "http", "url": "https://mcp.anakin.io/mcp" } } } ``` With no bearer, the connector exposes: | Tool | Purpose | |---|---| | `scrape` | Fetch one URL as clean Markdown. | | `wire_discover` | Find catalog actions for a natural-language intent. | | `wire_catalog` | Browse supported sites and their actions/parameters. | | `wire_read_action` | Run a read-only Wire action against a site. | Add an `Authorization: Bearer <key>` header to upgrade the same connection to the **full toolset** — writes, crawl, agentic search, monitors, and account-connected actions. See the [Claude Connector](/docs/integrations/ai-agents/claude-connector) for the authenticated tool surface. --- ## Limits and the 402 nudge Zero Touch is metered by a **per-IP free allowance**. When it's exhausted, the API returns a graceful `402` with a signup URL instead of a hard failure: ```json HTTP/1.1 402 Payment Required X-Anakin-Signup: https://anakin.io/signup X-Trial-Credits-Remaining: 0 { "message": "Free allowance used — sign up for 300 credits and your own key.", "trial": { "remaining_credits": 0, "signup_url": "https://anakin.io/signup" } } ``` Sign up free at [anakin.io/signup](https://anakin.io/signup) for **300 credits**, higher limits, writes, and account-connected actions. --- ## Upgrade in place Upgrading never means rewriting your integration. Add a single header to the **same request** — same endpoint, same body, same response shape: <Tabs items={["cURL", "Python", "MCP"]}> <Tab value="cURL"> ```bash curl -s https://api.anakin.io/v1/url-scraper/scrape \ -H "X-API-Key: ak_live_..." \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` </Tab> <Tab value="Python"> ```python r = requests.post( "https://api.anakin.io/v1/url-scraper/scrape", headers={"X-API-Key": "ak_live_..."}, json={"url": "https://example.com"}, ) ``` </Tab> <Tab value="MCP"> ```json { "mcpServers": { "anakin": { "type": "http", "url": "https://mcp.anakin.io/mcp", "headers": { "Authorization": "Bearer ak_live_..." } } } } ``` </Tab> </Tabs> Get your key from the [Dashboard](/dashboard) after signing up. --- ## Next steps <CardGrid> <ProductCard title="URL Scraper" description="The full scrape endpoint — parameters, browser mode, and AI JSON extraction" href="/docs/api-reference/url-scraper" /> <ProductCard title="Wire API" description="Discover and run managed actions across supported sites" href="/docs/api-reference/wire" /> <ProductCard title="MCP Server" description="Run Anakin as an MCP server for Claude, Cursor, and more" href="/docs/integrations/ai-agents/mcp-server" /> <ProductCard title="Pricing & Credits" description="Free tier, credit costs, and usage-based pricing above it" href="/docs/documentation/pricing" /> </CardGrid> --- # CrewAI (/docs/integrations/ai-frameworks/crewai) <ComingSoonBanner description="This is currently under active development. Stay tuned for updates." /> | | | |---|---| | **Framework** | [CrewAI](https://www.crewai.com) | | **Type** | Tool | --- ### What to expect The CrewAI integration will provide tools that agents can use autonomously: - **Scrape Tool** — Agents scrape any URL and get back clean markdown or structured JSON - **Search Tool** — Agents perform AI-powered web searches with instant results - **Research Tool** — Agents run deep multi-stage research on any topic --- ### In the meantime You can integrate AnakinScraper into your CrewAI agents today using the [REST API](/docs/api-reference) with CrewAI's custom tool support, or use the [Anakin CLI](/docs/sdks/cli) as a subprocess tool. --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # Flowise (/docs/integrations/ai-frameworks/flowise) <ComingSoonBanner description="This is currently under active development. Stay tuned for updates." /> | | | |---|---| | **Framework** | [Flowise](https://flowiseai.com) | | **Type** | Component | --- ### What to expect The Flowise integration will provide visual nodes for: - **Scrape Node** — Scrape any URL and pass clean data to downstream nodes - **Search Node** — AI-powered web search as a chatflow node - **Research Node** — Deep agentic research as a chatflow node --- ### In the meantime You can use AnakinScraper in Flowise today via the **Custom Tool** or **HTTP Request** node with the [REST API](/docs/api-reference). --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # Google ADK (/docs/integrations/ai-frameworks/google-adk) Google ADK tools for web scraping, search, and research — powered by Anakin. Build AI agents with Gemini that can extract data from any website, perform intelligent web searches, and conduct deep autonomous research. | | | |---|---| | **PyPI** | [pypi.org/project/anakin-adk](https://pypi.org/project/anakin-adk/) | | **Source** | [GitHub](https://github.com/Anakin-Inc/anakin-adk) | | **Type** | Tool | | **Version** | 0.1.2 | | **Tools** | 4 | | **License** | MIT | | **Requires** | Python >=3.10 | --- ### How it works You register Anakin tools with your Google ADK agent. When a user asks something that requires web data, Gemini automatically selects the right tool, fills in the parameters, and returns the results — no manual configuration needed. ``` User → "What's on this page?" → Gemini agent → scrape_website → Anakin API → results → Gemini → response ``` The tools expose their parameter schemas to Gemini via ADK's tool protocol, so the model knows when to use the browser, which country to route through, and whether to extract structured JSON — all based on the conversation context and your agent's instructions. --- ### Key features - **Anti-detection** — Proxy routing across 207 countries prevents blocking - **Intelligent Caching** — Up to 30x faster on repeated requests - **AI Extraction** — Convert any webpage into structured JSON - **Browser Automation** — Full headless Chrome support for SPAs and JS-heavy sites - **Batch Processing** — Scrape up to 10 URLs in a single request - **Deep Research** — Autonomous multi-stage research combining search, scraping, and AI synthesis --- ### Setup #### 1. Get your API key 1. Sign up at [anakin.io/signup](/signup) 2. Go to your [Dashboard](/dashboard) 3. Copy your API key (starts with `ask_`) #### 2. Install the package ```bash pip install anakin-adk ``` You also need the Anakin CLI installed and authenticated: ```bash pip install anakin-cli anakin login --api-key "ask_your-key-here" ``` --- ### Tools Each tool is exposed to Gemini with a full parameter schema. The model decides which parameters to use based on the user's request and your agent instructions. You can guide tool behavior by including hints in your agent's `instruction` field (e.g., "always use the browser for JavaScript-heavy sites" or "route through UK proxies"). #### 1. scrape_website Scrape a single URL and return clean markdown or structured JSON. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `url` | string | Yes | — | Target URL to scrape (HTTP/HTTPS) | | `country` | string | No | `us` | Proxy location from [207 countries](/docs/api-reference/supported-countries) | | `use_browser` | boolean | No | `false` | Enable headless Chrome for JavaScript-heavy sites | | `generate_json` | boolean | No | `false` | Use AI to extract structured data | | `session_id` | string | No | — | Browser session ID for [authenticated pages](/docs/api-reference/browser-sessions) | **Response includes:** Raw HTML, cleaned HTML, markdown conversion, structured JSON (if `generate_json` enabled), cache status, timing metrics. --- #### 2. batch_scrape Scrape up to 10 URLs at once and return combined results. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `urls` | string | Yes | — | Comma-separated list of URLs (1–10) | | `country` | string | No | `us` | Proxy location from [207 countries](/docs/api-reference/supported-countries) | | `use_browser` | boolean | No | `false` | Enable headless Chrome for JavaScript-heavy sites | | `generate_json` | boolean | No | `false` | Use AI to extract structured data from each page | **Response includes:** Per-URL results with HTML, markdown, and optional structured JSON. --- #### 3. search_web AI-powered web search returning results with citations. Results are returned immediately without polling. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `prompt` | string | Yes | — | Search query or question | | `limit` | number | No | `5` | Maximum results to return | **Response includes:** Array of results with URLs, titles, snippets, publication dates, last updated timestamps. --- #### 4. deep_research Autonomous multi-stage research pipeline combining search, scraping, and AI synthesis. Takes 1–5 minutes. | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `prompt` | string | Yes | Research question or topic | **Response includes:** Comprehensive AI-generated report, structured findings, citations with source URLs, scraped source data, processing metrics. --- ### Processing times | Tool | Type | Typical Duration | |------|------|------------------| | scrape_website | Async | 3–15 seconds | | batch_scrape | Async | 5–30 seconds | | search_web | **Sync** | Immediate | | deep_research | Async | 1–5 minutes | --- ### Usage #### Full toolkit Pass all 4 tools to your agent at once: ```python from anakin_adk import AnakinToolkit from google.adk.agents import Agent agent = Agent( model="gemini-2.5-pro", name="web_researcher", instruction="Help users extract data from the web", tools=AnakinToolkit().get_tools(), ) ``` Run with the ADK dev UI: ```bash adk web ``` #### Individual tools Use specific tools instead of the full toolkit: ```python from anakin_adk import ScrapeWebsiteTool, SearchWebTool from google.adk.agents import Agent agent = Agent( model="gemini-2.5-pro", name="search_and_scrape", instruction="Search the web and scrape relevant pages", tools=[SearchWebTool(), ScrapeWebsiteTool()], ) ``` #### Product research agent An agent that compares products by scraping multiple pages: ```python from anakin_adk import AnakinToolkit from google.adk.agents import Agent agent = Agent( model="gemini-2.5-pro", name="product_researcher", instruction="""You are a product research assistant. When asked to compare products: 1. Use search_web to find relevant product pages 2. Use batch_scrape with generate_json=true to extract structured data 3. Summarize findings in a comparison table""", tools=AnakinToolkit().get_tools(), ) ``` #### Deep research agent An agent for comprehensive research reports: ```python from anakin_adk import DeepResearchTool from google.adk.agents import Agent agent = Agent( model="gemini-2.5-pro", name="deep_researcher", instruction="""You are a research analyst. Use deep_research for broad questions that need multiple sources. Use search_web + scrape_website for targeted fact-checking.""", tools=AnakinToolkit().get_tools(), ) ``` #### Geo-targeted scraping agent An agent that routes through specific country proxies: ```python from anakin_adk import AnakinToolkit from google.adk.agents import Agent agent = Agent( model="gemini-2.5-pro", name="geo_scraper", instruction="""You scrape websites for users. Always ask which country to route through. Use use_browser=true for JavaScript-heavy sites. Use generate_json=true when the user wants structured data.""", tools=AnakinToolkit().get_tools(), ) ``` #### More examples The [examples directory](https://github.com/Anakin-Inc/anakin-adk/tree/main/examples) includes: - **`basic_scraping.py`** — Simple scrape agent - **`research_agent.py`** — Deep research agent - **`search_and_scrape.py`** — Multi-step: search then scrape --- ### Troubleshooting | Code | Meaning | Action | |------|---------|--------| | 400 | Invalid parameters | Check your agent's instructions — Gemini may be passing unexpected values | | 401 | Invalid API key | Run `anakin login` to re-authenticate | | 402 | Plan upgrade required | Upgrade at [Pricing](/docs/documentation/pricing) | | 404 | Job not found | Job may have expired | | 429 | Rate limit exceeded | Reduce request frequency or upgrade your plan | | 5xx | Server error | Retry with backoff | **Common issues:** | Issue | Fix | |-------|-----| | Agent never uses tools | Check that `tools=` is set correctly and the instruction mentions web tasks | | Empty scrape results | Add `use_browser=true` hint to your agent instruction for JS-heavy sites | | Wrong country data | Add a country hint to your instruction (e.g., "always route through `gb`") | | CLI not authenticated | Run `anakin status` to check, then `anakin login` if needed | --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # LangChain (/docs/integrations/ai-frameworks/langchain) <ComingSoonBanner description="This is currently under active development. Stay tuned for updates." /> --- ### What to expect The LangChain integration will provide: - **Document Loader** — Load web pages as LangChain documents using AnakinScraper's scraping engine - **Search Tool** — AI-powered web search as a LangChain tool for agents - **Research Tool** — Deep agentic research as a LangChain tool --- ### In the meantime You can integrate AnakinScraper into your LangChain applications today using the [REST API](/docs/api-reference) with LangChain's custom tool support, or use the [Anakin CLI](/docs/sdks/cli) as a subprocess tool. --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # Langflow (/docs/integrations/ai-frameworks/langflow) <ComingSoonBanner description="This is currently under active development. Stay tuned for updates." /> | | | |---|---| | **Framework** | [Langflow](https://www.langflow.org) | | **Type** | Component | --- ### What to expect The Langflow integration will provide visual drag-and-drop components for: - **Scrape Component** — Scrape any URL and feed clean data into your flow - **Search Component** — AI-powered web search as a flow node - **Research Component** — Deep agentic research as a flow node --- ### In the meantime You can use AnakinScraper in Langflow today via the **HTTP Request** component with the [REST API](/docs/api-reference). --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # LlamaIndex (/docs/integrations/ai-frameworks/llamaindex) <ComingSoonBanner description="This is currently under active development. Stay tuned for updates." /> | | | |---|---| | **Framework** | [LlamaIndex](https://www.llamaindex.ai) | | **Type** | Reader | --- ### What to expect The LlamaIndex integration will provide: - **AnakinReader** — Load web pages as LlamaIndex `Document` objects for RAG pipelines, indexing, and querying - **Search Tool** — AI-powered web search as a LlamaIndex tool for agents - **Research Tool** — Deep agentic research as a LlamaIndex tool --- ### In the meantime You can integrate AnakinScraper into your LlamaIndex applications today using the [REST API](/docs/api-reference) with LlamaIndex's custom reader support, or use the [Anakin CLI](/docs/sdks/cli) as a subprocess tool. --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # Claude Code (/docs/integrations/ide-plugins/claude-code) Scrape websites, search the web, and run deep research directly inside Claude Code. | | | |---|---| | **Source** | [GitHub](https://github.com/Anakin-Inc/anakin-claude-plugin) | | **License** | MIT | | **Requires** | [anakin-cli](/docs/sdks/cli) (Python 3.10+) | --- ### Prerequisites - **Claude Code** installed and working - **Python 3.10+** — required for [anakin-cli](/docs/sdks/cli) (installed automatically by the plugin) - **API key** — get one from the [Dashboard](/dashboard) (the plugin will prompt you if needed) --- ### Setup Clone the plugin from GitHub and point Claude Code to it: ```bash git clone https://github.com/Anakin-Inc/anakin-claude-plugin.git claude --plugin-dir ./anakin-claude-plugin ``` The plugin handles the rest automatically — its `check-auth` hook verifies that `anakin-cli` is installed and authenticated before running commands, and the `/anakin:setup` skill can install the CLI and configure your API key if needed. --- ### Skills The plugin adds these skills to Claude Code: | Skill | Command | Description | |-------|---------|-------------| | Scrape Website | `/anakin:scrape-website [url]` | Scrape a single URL to markdown, JSON, or raw | | Scrape Batch | `/anakin:scrape-batch [url1] [url2]` | Scrape up to 10 URLs at once | | Search Web | `/anakin:search-web [query]` | AI-powered web search with instant results | | Deep Research | `/anakin:deep-research [topic]` | Deep agentic multi-step research (1–5 min) | | Setup | `/anakin:setup` | Install CLI, configure API key, set up output directory | | CLI Knowledge | *(auto)* | Background knowledge: escalation workflow, CLI rules, output organization | --- ### Agents | Agent | Description | |-------|-------------| | `data-extraction-architect` | Plans which anakin-cli commands to use for complex extraction tasks | --- ### Hooks | Hook | Event | Description | |------|-------|-------------| | `check-auth` | `PreToolUse` (Bash) | Verifies anakin-cli is installed and authenticated before running commands | --- ### Usage Once the plugin is active, Claude will use Anakin automatically for scraping and search tasks. You can also invoke skills directly: ``` /anakin:search-web latest React documentation /anakin:scrape-website https://example.com /anakin:deep-research pros and cons of microservices vs monolith /anakin:scrape-batch https://a.com https://b.com https://c.com ``` All output is saved to the `.anakin/` directory to keep your context window clean: ``` .anakin/ ├── search-react-docs.json ├── example.com.md ├── batch-results.json └── research-microservices.json ``` --- ### Configuration | Variable | Description | |----------|-------------| | `ANAKIN_API_KEY` | API key (env var, takes precedence over config file) | | `~/.anakin/config.json` | Stored API key (set via `anakin login`) | --- # Cursor (/docs/integrations/ide-plugins/cursor) Scrape websites, search the web, and run deep research directly inside Cursor. | | | |---|---| | **Source** | [GitHub](https://github.com/Anakin-Inc/anakin-cursor-plugin) | | **License** | MIT | | **Requires** | [anakin-cli](/docs/sdks/cli) (Python 3.10+) | --- ### Prerequisites - **Cursor** installed and working - **Python 3.10+** — required for [anakin-cli](/docs/sdks/cli) (installed automatically by the plugin) - **API key** — get one from the [Dashboard](/dashboard) (the plugin will prompt you if needed) --- ### Setup Clone the plugin from GitHub and add it to Cursor: ```bash git clone https://github.com/Anakin-Inc/anakin-cursor-plugin.git /add-plugin anakin ``` The plugin handles the rest automatically — its `anakin-setup` rule ensures `anakin-cli` is installed and authenticated before running commands. --- ### Skills | Skill | Description | |-------|-------------| | `scrape-website` | Scrape a single URL to markdown, JSON, or raw using `anakin scrape` | | `scrape-batch` | Scrape up to 10 URLs at once using `anakin scrape-batch` | | `search-web` | AI-powered web search using `anakin search` | | `deep-research` | Deep agentic research across multiple sources using `anakin research` | --- ### Rules | Rule | Description | |------|-------------| | `anakin-setup` | Ensures anakin-cli is installed and authenticated before running commands | | `anakin-cli-usage` | URL quoting, output handling, format defaults, and error recovery | --- ### Agents | Agent | Description | |-------|-------------| | `data-extraction-architect` | Plans which anakin-cli commands to use for complex extraction tasks | --- ### Usage Once the plugin is installed, Cursor's AI agent will automatically use Anakin for web scraping and search tasks. You can also reference the skills directly in your prompts. --- ### Configuration | Variable | Description | |----------|-------------| | `ANAKIN_API_KEY` | API key (env var, takes precedence over config file) | | `~/.anakin/config.json` | Stored API key (set via `anakin login`) | --- # MCP Server (/docs/integrations/ide-plugins/mcp-server) The [`@anakin-io/mcp`](https://www.npmjs.com/package/@anakin-io/mcp) package is an [MCP](https://modelcontextprotocol.io) server that exposes six Anakin tools — `scrape`, `search`, `map`, `crawl`, `agentic_search`, `wire_action` — to any MCP-compatible agent client. With it installed, Claude Desktop, Cursor, etc. can fetch web pages, run AI search, crawl sites, and execute Wire actions directly from a chat or coding session. > **Status: alpha (v0.1.x).** Tool surface and arguments may change between minor versions until v1.0. > Package: [`@anakin-io/mcp` on npm](https://www.npmjs.com/package/@anakin-io/mcp) · Source: [github.com/Anakin-Inc/anakin-mcp](https://github.com/Anakin-Inc/anakin-mcp). --- ## Quick install (recommended) One command auto-configures every detected agent client on your machine: ```bash npx -y @anakin-io/mcp init --all ``` You'll be prompted for your API key (or set `ANAKIN_API_KEY` first to skip the prompt). Get one free at the [Dashboard](/dashboard) — 300 credits, no card required. After it finishes, **restart your agent client(s)**. The `anakin` MCP server appears in the tool list, exposing six tools. If you'd rather configure one specific client: ```bash npx -y @anakin-io/mcp init --client=cursor ``` Available client names: `claude-desktop`, `claude-code`, `cursor`, `cline`, `continue`, `zed`, `windsurf`, `vscode`. --- ## What gets exposed Each tool is a thin wrapper around the matching Anakin REST endpoint. Agents see typed input schemas (so they know what arguments each tool takes) and get back a string payload they can quote in their replies. | Tool | Purpose | Async? | |---|---|---| | `scrape` | Fetch one URL → markdown by default. Set `generateJson: true` for AI-extracted structured data. Set `useBrowser: true` for SPAs. | async (3–15s typical) | | `search` | AI web search with citations. Returns URL + title + snippet for each hit. | sync | | `map` | Discover all reachable URLs on a domain. | async | | `crawl` | Bulk-fetch markdown across a site (with include / exclude patterns). | async | | `agentic_search` | Multi-source deep research. Searches the web, scrapes citations, structures the combined data with an LLM. | async (1–5 min) | | `wire_action` | Execute a pre-built website action via [Wire](/docs/api-reference/wire) (login flows, form fills, etc.). | async | --- ## Manual setup per client If you'd rather edit config files yourself, copy the right snippet for your client. **Set your real API key** in `env.ANAKIN_API_KEY` — the agent never sees it; it lives only in this local config file. ### Claude Desktop | Platform | Path | |---|---| | macOS | `~/Library/Application Support/Claude/claude_desktop_config.json` | | Windows | `%APPDATA%/Claude/claude_desktop_config.json` | ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Restart Claude Desktop. ### Claude Code (Anthropic CLI) Edit `~/.claude/settings.json`: ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Or use Claude Code's built-in command: ```bash claude mcp add anakin npx -y @anakin-io/mcp -e ANAKIN_API_KEY=ak-... ``` ### Cursor Edit `~/.cursor/mcp.json` (user-scoped) or `./.cursor/mcp.json` (project-scoped): ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Restart Cursor. ### Cline (VS Code extension) | Platform | Path | |---|---| | macOS | `~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json` | | Linux | `~/.config/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json` | | Windows | `%APPDATA%/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json` | ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Reload the Cline extension (or restart VS Code). ### Continue (IDE extension) Edit `~/.continue/config.json`: ```json { "experimental": { "modelContextProtocolServers": [ { "name": "anakin", "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } ] } } ``` If you have other entries under `experimental.modelContextProtocolServers`, append the `anakin` object — don't overwrite. Reload the Continue extension. ### Zed | Platform | Path | |---|---| | macOS | `~/.config/zed/settings.json` | | Linux | `~/.config/zed/settings.json` | | Windows | `%APPDATA%/Zed/settings.json` | ```json { "context_servers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Reload Zed. ### Windsurf Edit `~/.codeium/windsurf/mcp_config.json`: ```json { "mcpServers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Restart Windsurf. ### VS Code (with the MCP extension) Edit `.vscode/mcp.json` in your workspace: ```json { "servers": { "anakin": { "command": "npx", "args": ["-y", "@anakin-io/mcp"], "env": { "ANAKIN_API_KEY": "ak-..." } } } } ``` Reload VS Code. --- ## Verify it works In Claude Desktop / Cursor / etc., start a new chat and try: > Use anakin to scrape https://example.com and return the markdown. The agent should: 1. Find the `scrape` tool in `anakin`'s tool list 2. Call `scrape({ url: "https://example.com" })` 3. Quote back the rendered markdown You can also try other tools: > Use anakin's search to find recent articles about MCP servers. > Use anakin to map the URLs on https://docs.anakin.io and pick three to scrape. > Use anakin's agentic_search for a comparative analysis of TypeScript HTTP libraries. --- ## How the API key flows Your key never enters the conversation context. It lives **only** in the agent client's local config file (e.g., `claude_desktop_config.json`), gets passed to the MCP server subprocess as an environment variable when the client starts, and from there is sent in the `X-API-Key` header on every Anakin REST request. If `ANAKIN_API_KEY` is unset or empty when the server starts, it exits with a message pointing at the [Dashboard](/dashboard). The agent client typically surfaces that error in its MCP servers panel; the agent itself just sees `anakin` missing from its tool list. If the key is invalid or you're out of credits, the first tool call returns an error (`Invalid API key` or `Insufficient credits`) which the agent reports back to you. Update the config and restart the client. --- ## Troubleshooting **The MCP servers panel shows `anakin` as "Failed"** → Click the entry to see the stderr output. Most common: `ANAKIN_API_KEY is not set` (fix: fill in `env.ANAKIN_API_KEY` and restart) or a JSON syntax error in your config (fix: validate with `cat <config> | jq .`). **`anakin` doesn't appear in the tool list at all** → The client hasn't reloaded the config. Quit fully (⌘Q on macOS) and reopen. **Tool call returns "Invalid API key"** → Key is wrong or revoked. Get a fresh one from the [Dashboard](/dashboard). **Tool call returns "Insufficient credits"** → Top up at the [Dashboard](/dashboard). Failed jobs aren't charged, so credits only deduct on successful operations. **`init --all` says "No supported MCP clients detected"** → The init command checks for the existence of each client's config directory. If a client is installed but you've never opened it (or you installed via an unusual method that doesn't create the standard dir), pass the client name explicitly: ```bash npx -y @anakin-io/mcp init --client=cursor ``` That always writes the config, creating any missing parent directories. --- ## Two paths agents can use Anakin This MCP server is one of two complementary integration paths: 1. **MCP** (this page) — for clients that support it. Tools are typed and called natively. Best when available. 2. **[SKILL.md](https://anakin.io/agent-onboarding/SKILL.md)** — for any agent that can fetch a URL. The markdown describes the API end-to-end so an agent can use it via plain HTTP calls. Works in environments without subprocess support. Most users on Claude Desktop / Cursor / Windsurf / VS Code will want path 1. Custom agents and CI agents can use path 2. --- ## Related - [`anakin-mcp` on GitHub](https://github.com/Anakin-Inc/anakin-mcp) — source, releases, issue tracker - [`@anakin-io/sdk`](https://github.com/Anakin-Inc/anakin-node) — Node.js / TypeScript SDK - [`anakin`](https://github.com/Anakin-Inc/anakin-py) — Python SDK - [`anakin-cli`](https://github.com/Anakin-Inc/anakin-cli) — Python CLI for terminal-driven use - [SKILL.md](https://anakin.io/agent-onboarding/SKILL.md) — agent-onboarding doc for non-MCP agents - [API reference](/docs/api-reference) — endpoint-by-endpoint REST docs --- # OpenClaw (/docs/integrations/ide-plugins/openclaw) Skill for [OpenClaw](https://openclaw.ai) that gives your AI agent web scraping, batch scraping, AI search, and autonomous research capabilities. Available on [ClawHub](https://clawhub.ai/Viraal-Bambori/anakin). | | | |---|---| | **Marketplace** | [ClawHub](https://clawhub.ai/Viraal-Bambori/anakin) | | **Platform** | [OpenClaw](https://openclaw.ai) | | **Version** | 1.0.0 | | **Security** | VirusTotal Benign, OpenClaw Benign (high confidence) | | **Requirements** | `anakin` binary, `ANAKIN_API_KEY` env var | --- ### Setup #### 1. Install the skill ```bash clawhub install anakin ``` This downloads the skill and its dependencies (including `anakin-cli`) into your `./skills` directory automatically. Or browse and download directly from [ClawHub](https://clawhub.ai/Viraal-Bambori/anakin). #### 2. Configure your API key 1. Sign up at [anakin.io/signup](/signup) and get your API key from the [Dashboard](/dashboard) 2. Set the `ANAKIN_API_KEY` environment variable in your OpenClaw config #### 3. Restart OpenClaw Start a new OpenClaw session so it picks up the installed skill. --- ### What the agent gets Once installed, your OpenClaw agent can autonomously: - **Scrape URLs** — Extract content from any web page as clean markdown, structured JSON, or raw HTML - **Batch scrape** — Scrape up to 10 URLs in parallel with a single call - **AI search** — Run intelligent web searches with citations and relevance scoring - **Deep research** — Autonomous multi-source research that synthesizes comprehensive reports (1–5 minutes) The agent decides which capability to use based on the user's request. The skill's `SKILL.md` includes a decision guide so the agent picks the right tool automatically. --- ### How it works The skill is a `SKILL.md` file that instructs the OpenClaw agent how to use the `anakin` CLI. When a user asks the agent to scrape a website, search the web, or research a topic, the agent: 1. Creates a `.anakin/` output folder in the working directory 2. Runs the appropriate `anakin` CLI command 3. Reads the output file and summarizes the results All output is saved to files (never floods the agent's context), and the agent handles errors like rate limits and timeouts automatically. --- ### Security The skill has been scanned and verified: | Scanner | Status | Detail | |---------|--------|--------| | VirusTotal | Benign | [View report](https://www.virustotal.com/gui/file/d23ce9f141cfc820b1031d3e7664da171b75709bb58517ab6a122562a4cc5b39) | | OpenClaw | Benign | High confidence — only requires `anakin` binary and `ANAKIN_API_KEY` | The skill does not request unrelated credentials, system files, or hidden endpoints. It only uses the `anakin` CLI and a single API key. --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # Website Monitor Webhooks (/docs/integrations/website-monitoring/webhooks) > **Monitor alerts are now part of Anakin's platform-wide webhook system** — the same signing > convention (`X-Anakin-Signature`, HMAC-SHA256 over the raw body) used by every product, and the > `monitor.change` payload documented on this page is unchanged. Monitor deliveries also appear in > your account-wide [delivery log](/docs/api-reference/webhooks/managing-endpoints#delivery-log), > and registered endpoints can subscribe to `monitor.change` alongside every other event. See > [Webhooks](/docs/api-reference/webhooks). A **Website Monitor** watches a URL and, when the page changes, sends an HTTP `POST` to a webhook you configure. This page covers how to receive those calls, verify they're genuine, and fetch the full change details. > Webhooks need no email provider and work out of the box. Email is the other live channel; > Slack, Discord, and Telegram are planned. --- ## Set up 1. Open your monitor and go to **Notifications → Webhook URL**. 2. Enter the URL we should `POST` to and save. 3. We generate a **signing secret** (`whsec_…`) and show it in the Notifications card. Copy it into your receiving app — you'll use it to verify each delivery. --- ## Why deliveries are signed Your webhook URL is public: anyone who learns it could `POST` forged "the page changed" payloads. So every delivery is **HMAC-SHA256 signed** with your secret, and the signature is sent in the `X-Anakin-Signature` header. - The secret is **shared** — we hold it, you hold a copy. It never travels in the request; only the signature (a fingerprint derived from the body + secret) does. - **We always send.** We don't check anything about your secret before sending — verification happens **on your side**. Your app recomputes the signature from the received body and its copy of the secret and compares: - **Match** → genuinely from us, untampered → process it. - **No match** → forged or wrong secret → reject it (return `401`, ignore the body). Verifying is optional but strongly recommended for any endpoint that acts on the data. --- ## Request format `POST` to your webhook URL with these headers: | Header | Example | Notes | |---|---|---| | `Content-Type` | `application/json` | Body is JSON. | | `User-Agent` | `AnakinScraper-Monitor/1` | Identifies our sender. | | `X-Anakin-Timestamp` | `1783009089` | Unix seconds when the `POST` was made. | | `X-Anakin-Delivery-Id` | `c1e8fbac-…` | The change id. **Stable across retries** — use it to de-duplicate. | | `X-Anakin-Signature` | `sha256=7987…` | `sha256=` + hex `HMAC_SHA256(secret, rawBody)`. | Per-request timeout is **10 seconds** — acknowledge quickly (see [Best practices](#best-practices)). The signature is computed over the **exact raw bytes** of the body, so verify against the raw body, never a re-serialized object (JSON key order or whitespace would differ and the check would fail). ### Verify the signature **Node (Express) — capture the raw body:** ```js import express from "express" import crypto from "crypto" const app = express() app.use("/anakin-webhook", express.raw({ type: "application/json" })) // raw bytes, not parsed app.post("/anakin-webhook", (req, res) => { const secret = process.env.ANAKIN_WEBHOOK_SECRET // the whsec_… you copied const got = req.header("X-Anakin-Signature") || "" const expected = "sha256=" + crypto.createHmac("sha256", secret).update(req.body).digest("hex") const ok = got.length === expected.length && crypto.timingSafeEqual(Buffer.from(got), Buffer.from(expected)) // constant-time compare if (!ok) return res.status(401).send("bad signature") const event = JSON.parse(req.body.toString()) // de-dupe on req.header("X-Anakin-Delivery-Id"), then process asynchronously res.status(200).send("ok") }) ``` **Python (Flask):** ```python import hmac, hashlib, os from flask import Flask, request, abort app = Flask(__name__) @app.post("/anakin-webhook") def webhook(): secret = os.environ["ANAKIN_WEBHOOK_SECRET"].encode() raw = request.get_data() # raw bytes — do NOT use request.json here expected = "sha256=" + hmac.new(secret, raw, hashlib.sha256).hexdigest() got = request.headers.get("X-Anakin-Signature", "") if not hmac.compare_digest(expected, got): # constant-time compare abort(401) event = request.get_json() # de-dupe on request.headers.get("X-Anakin-Delivery-Id"), then process return "", 200 ``` --- ## Payload ```json { "type": "monitor.change", "monitorId": "62e555fb-…", "url": "https://flipkart.com", "watchMode": "full_page", "changeId": "c1e8fbac-…", "changedAt": "2026-07-02T16:16:52.289Z", "changedFields": ["content"], "summary": "…", "diff": { } } ``` - `type` — always `monitor.change`. - `watchMode` — `full_page` or `specific_data` (shapes `diff`, below). - `changeId` — equals `X-Anakin-Delivery-Id`. - `changedFields` — present when non-empty. - `summary` — present only when AI meaningful-change wrote one. **`diff` for `full_page`** (whole-page content change) — metadata plus snapshot references: ```json { "type": "content", "before": { "contentHash": "8366…" }, "after": { "contentHash": "0c24…" }, "stats": { "adds": 91, "dels": 91 }, "changedFields": ["content"], "beforeSnapshotId": "63ca1f01-…", "afterSnapshotId": "45874b38-…" } ``` **`diff` for `specific_data`** (tracked fields changed) — the values are inline; `before` / `after` hold the fields that changed, e.g. `{ "price": 999 }` → `{ "price": 899 }`. **`diff.aiMeaningful`** — present only for **AI meaningful-change** monitors: the AI's curated list of the substantive changes with trivial noise removed. Each entry is `{ "description", "before", "after" }`. Use it for a clean, human-readable "what actually changed" (the `stats`/snapshot bodies still carry the full raw diff). ```json "aiMeaningful": [ { "description": "Price dropped", "before": "$56,586", "after": "$16,988" }, { "description": "Availability changed", "before": "In Stock", "after": "Confirm Availability" } ] ``` ### Design note — a predictable contract For `full_page`, the payload carries the change *metadata* — `stats` (how much changed) plus `beforeSnapshotId` / `afterSnapshotId` — and you retrieve the full before/after text by snapshot id (next section). The contract is the same on every delivery: the webhook reports **that** the page changed and by how much, and the snapshot ids are always how you fetch the content — so there's nothing per-delivery to interpret. `specific_data` values are small, so they're delivered inline. --- ## Fetch the full before / after content For a `full_page` change, the payload gives you `monitorId` and, inside `diff`, the `beforeSnapshotId` and `afterSnapshotId`. Each snapshot's stored page body is fetched from: ``` GET https://api.anakin.io/v1/monitors/{monitorId}/snapshots/{snapshotId}/content ``` Authenticate with your Anakin API key in the `X-API-Key` header (see the [API reference](/docs/api-reference) for keys and base URL). ```bash curl -s -H "X-API-Key: $ANAKIN_API_KEY" \ "https://api.anakin.io/v1/monitors/$MONITOR_ID/snapshots/$BEFORE_SNAPSHOT_ID/content" ``` **Response** ```json { "available": true, "content": "…the stored page body…", "capturedAt": "2026-07-02T16:16:52.289Z", "contentHash": "8366…" } ``` | Field | Type | Notes | |---|---|---| | `available` | boolean | `false` if no body was stored for this snapshot (e.g. a `specific_data` monitor, or storage not configured) or it couldn't be loaded. When `false`, `content` is `""`. | | `content` | string | The full stored page body — the **same representation the monitor compares** (markdown, cleaned HTML, or raw HTML, per the monitor's compare format). | | `capturedAt` | string | When the snapshot was taken. | | `contentHash` | string | SHA-256 of the body; equals the `before`/`after` `contentHash` in the `diff`. | A missing or non-owned monitor/snapshot returns `404`. ### End-to-end: from webhook to full diff Fetch both sides and compare them however you like. Continuing the verified handler from above: ```js const event = JSON.parse(req.body.toString()) if (event.watchMode === "full_page" && event.diff?.beforeSnapshotId) { const { monitorId } = event const { beforeSnapshotId, afterSnapshotId } = event.diff const getBody = (snapshotId) => fetch(`https://api.anakin.io/v1/monitors/${monitorId}/snapshots/${snapshotId}/content`, { headers: { "X-API-Key": process.env.ANAKIN_API_KEY }, }).then((r) => r.json()) const [before, after] = await Promise.all([getBody(beforeSnapshotId), getBody(afterSnapshotId)]) if (before.available && after.available) { // before.content / after.content are the full page bodies — diff them with any library // (e.g. `diff`, `jsdiff`), or store them, or feed them to an LLM. } } ``` Or entirely in the shell: ```bash url="https://api.anakin.io/v1/monitors/$MONITOR_ID/snapshots" before=$(curl -s -H "X-API-Key: $ANAKIN_API_KEY" "$url/$BEFORE_SNAPSHOT_ID/content" | jq -r .content) after=$(curl -s -H "X-API-Key: $ANAKIN_API_KEY" "$url/$AFTER_SNAPSHOT_ID/content" | jq -r .content) diff <(printf '%s' "$before") <(printf '%s' "$after") ``` ### Notes - **`specific_data` monitors** already include the changed values inline in `diff.before` / `diff.after`, so you usually don't fetch anything. Their snapshots store field values, not a page body, so the content endpoint returns `available: false` — read `extractedData` from the snapshots endpoint instead if you want the full field set. - **Handle `available: false`** — if a body isn't retrievable, fall back to the `stats` / `summary` in the payload (and `contentHash` to detect duplicates). ### Related endpoints Same `X-API-Key` auth: - `GET /v1/monitors/{id}/changes` — recorded changes, each with the same `diff` object (including the snapshot ids). - `GET /v1/monitors/{id}/snapshots` — recent snapshots (`id`, `capturedAt`, `contentHash`, `extractedData`, `s3Path`). --- ## Delivery, retries & idempotency Each change enqueues one delivery per configured channel; a background worker drains the queue. - **Success:** your endpoint returns **2xx** → recorded as `success`. - **Permanent failure (no retry):** a **4xx other than 429** (e.g. 400, 401, 404) → we assume the endpoint is misconfigured and stop → marked `exhausted`. - **Retried with backoff:** a connection error / timeout, any **5xx**, or **429** → retried up to **6 attempts** (~8.5h total), then `exhausted`: | After attempt | Wait before next | |---|---| | 1 | 1 min | | 2 | 5 min | | 3 | 30 min | | 4 | 2 h | | 5 | 6 h | - **At-least-once — de-duplicate.** A delivery can arrive more than once (e.g. you processed it but replied slowly, so we retried). De-dupe on `X-Anakin-Delivery-Id` (= `changeId`), which is stable across retries. - **Manual re-drive.** A `failed` / `exhausted` delivery can be re-queued from the monitor's Notifications panel. --- ## Best practices - **Verify the signature** with a constant-time compare, over the **raw** body. - **Respond 2xx fast** (within 10s). Acknowledge first, do heavy work asynchronously — slow endpoints get retried and look like failures. - **Be idempotent** — key processing on `X-Anakin-Delivery-Id`. - **Return the right status** — 2xx = accepted; 5xx / 429 to ask for a retry; a 4xx (non-429) tells us to stop permanently, so use it only for genuinely bad requests. - **Keep the secret secret** — treat it like a password; re-save the monitor to rotate it. - **Don't assume ordering** — deliveries can overlap under retries; use `changedAt` / `X-Anakin-Timestamp` if you need sequence. --- ## Test it locally 1. Run a small receiver that logs the request and verifies the signature (read the raw body, recompute `sha256=HMAC_SHA256(secret, body)`, compare to `X-Anakin-Signature`). 2. Create a monitor with the webhook URL pointing at it (e.g. `http://localhost:4000/hook`) and copy the signing secret into the receiver. 3. Click **Test** in the Notifications card for an immediate sample `POST`, or let a real check detect a change (which also exercises the retry queue). 4. Confirm the receiver logs a signature match and the Notifications card shows a `success` delivery. > The **Test** button delivers synchronously and skips the retry queue; a **real detected change** > goes through the full queue + retry path. Try both before relying on webhooks. --- # Dify (/docs/integrations/workflow/dify) Web scraping and AI-powered search plugin for Dify. Extract data from any website, perform intelligent web searches, and conduct deep research — all inside your Dify workflows and agents. | | | |---|---| | **Marketplace** | [Dify Plugin Store](https://marketplace.dify.ai/plugin/anakin/anakin) | | **Source** | [GitHub](https://github.com/Anakin-Inc/anakin-dify-plugins/tree/main/anakin) | | **Type** | Tool Plugin | | **Version** | 0.0.1 | | **Tools** | 5 | --- ### Key features - **Anti-detection** — Proxy routing across 207 countries prevents blocking - **Intelligent Caching** — Up to 30x faster on repeated requests - **AI Extraction** — Convert any webpage into structured JSON - **Browser Automation** — Full headless Chrome support for SPAs and JS-heavy sites - **Session Management** — Authenticated scraping with encrypted session storage (AES-256-GCM) - **Batch Processing** — Submit multiple URLs in a single request --- ### Setup #### 1. Get your API key 1. Sign up at [anakin.io/signup](/signup) 2. Go to your [Dashboard](/dashboard) 3. Copy your API key (starts with `ask_`) #### 2. Install in Dify 1. Install the Anakin plugin in your Dify workspace from the [Plugin Store](https://marketplace.dify.ai/plugin/anakin/anakin) 2. Go to **Plugins** > **Anakin** > **Configure** 3. Enter a name for the authorization (e.g., "Production") 4. Paste your API key 5. Click **Save** --- ### Tools #### 1. URL Scraper Scrapes a single URL, returning HTML, markdown, and optionally structured JSON. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `url` | string | Yes | — | Target URL to scrape (HTTP/HTTPS) | | `country` | string | No | `us` | Proxy location from 207 countries | | `use_browser` | boolean | No | `false` | Enable headless Chrome for JavaScript-heavy sites | | `generate_json` | boolean | No | `false` | Use AI to extract structured data | | `session_id` | string | No | — | Browser session ID for authenticated pages | **Response includes:** Raw HTML, cleaned HTML, markdown conversion, structured JSON (if `generate_json` enabled), cache status, timing metrics. --- #### 2. Batch URL Scraper Scrape up to 10 URLs simultaneously in parallel. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `urls` | string | Yes | — | Comma-separated list of URLs (1–10) | | `country` | string | No | `us` | Proxy location from 207 countries | | `use_browser` | boolean | No | `false` | Enable headless Chrome for JavaScript-heavy sites | | `generate_json` | boolean | No | `false` | Use AI to extract structured data from each page | --- #### 3. AI Search Synchronous AI-powered web search returning results with citations and relevance scoring. Results are returned immediately without polling. | Parameter | Type | Required | Default | Description | |-----------|------|----------|---------|-------------| | `prompt` | string | Yes | — | Search query or question | | `limit` | number | No | `5` | Maximum results to return | **Response includes:** Array of results with URLs, titles, snippets, publication dates, last updated timestamps. --- #### 4. Deep Research (Agentic Search) Multi-stage automated research pipeline combining search, scraping, and AI synthesis. Takes 1–5 minutes. | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `prompt` | string | Yes | Research question or topic | **Response includes:** AI-generated comprehensive answers, summaries, structured findings, citations with source URLs, scraped source data, processing metrics. --- ### Examples #### In a Workflow 1. Add a **Tool** node to your workflow 2. Select **Anakin** and choose your tool 3. Configure parameters (e.g., enter URL, enable `generate_json`) 4. Connect to the next node for processing #### In an Agent 1. Create an Agent app 2. Add Anakin tools to the agent's toolset 3. The agent will automatically use scraping/search based on user queries #### Scraping with AI extraction ``` Tool: URL Scraper URL: https://example.com/products Generate JSON: true ``` Returns structured product data automatically extracted by AI. #### Authenticated scraping ``` Tool: URL Scraper URL: https://example.com/dashboard Session ID: your-session-id-from-dashboard Use Browser: true ``` Scrapes pages that require login using your saved browser session. Learn more about [Browser Sessions](/docs/api-reference/browser-sessions). --- ### Processing times | Tool | Type | Typical Duration | |------|------|------------------| | URL Scraper | Async | 3–15 seconds | | Batch Scraper | Async | 5–30 seconds | | AI Search | **Sync** | Immediate | | Deep Research | Async | 1–5 minutes | | Custom Scraper | Async | 3–15 seconds | --- ### Troubleshooting | Code | Meaning | Action | |------|---------|--------| | 400 | Invalid parameters | Check your input | | 401 | Invalid API key | Verify your API key in plugin settings | | 402 | Plan upgrade required | Upgrade at [Pricing](/docs/documentation/pricing) | | 404 | Job not found | Job may have expired | | 429 | Rate limit exceeded | Wait and retry | | 5xx | Server error | Retry with backoff | --- ### Country codes Proxy routing supports [207 countries](/docs/api-reference/supported-countries). Common codes: | Code | Country | |------|---------| | `us` | United States (default) | | `gb` | United Kingdom | | `de` | Germany | | `fr` | France | | `jp` | Japan | | `au` | Australia | --- # Make (/docs/integrations/workflow/make) Official verified integration for Make (formerly Integromat) with native modules for web scraping, AI search, and deep research. | | | |---|---| | **Marketplace** | [make.com/integrations/anakin](https://www.make.com/en/integrations/anakin) | | **Status** | Verified, Official Vendor | | **Modules** | 4 (3 Actions, 1 Search) | --- ### Setup #### 1. Get your API key Sign up at [anakin.io/signup](/signup) and get your API key from the [Dashboard](/dashboard). #### 2. Add the Anakin module 1. In your Make scenario, click **+** to add a module 2. Search for **Anakin** 3. Select the module you need (UniversalDataExtractor, Search, etc.) #### 3. Connect your account 1. Click **Create a connection** 2. Enter your API key 3. Click **Save** --- ### Modules #### UniversalDataExtractor (Action) Extract data from any website. Submits a scrape job and returns the results including HTML, markdown, and structured data. Maps to the [URL Scraper API](/docs/api-reference/url-scraper). --- #### DataPoller (Action) Fetch the results for a previously submitted job using its job ID. Use this after **UniversalDataExtractor** or **AgenticSearch** if you need to poll for results separately. Maps to the scrape job status endpoint (`GET /v1/request/{id}`) or agentic search status endpoint (`GET /v1/agentic-search/{id}`). --- #### AgenticSearch (Action) Start an advanced AI search job that searches web sources and extracts structured data. Returns a job ID to check results later. Takes 1–5 minutes. Maps to the [Agentic Search API](/docs/api-reference/agentic-search). --- #### Search (Search) Perform an AI-powered web search. Returns instant results with answers, citations, and sources. This is synchronous — no polling needed. Maps to the [Search API](/docs/api-reference/search). --- ### Examples #### Scrape a URL and save to Google Sheets ``` Schedule → Anakin (UniversalDataExtractor) → Google Sheets (Add Row) ``` 1. **Schedule** — trigger on a daily/hourly schedule 2. **Anakin UniversalDataExtractor** — enter the URL to scrape 3. **Google Sheets** — map the markdown or structured data to columns #### Research and email a report ``` Webhook → Anakin (AgenticSearch) → Delay → Anakin (DataPoller) → Gmail (Send Email) ``` 1. **Webhook** — receive a research topic 2. **Anakin AgenticSearch** — start deep research, get a job ID 3. **Delay** — wait 3–5 minutes for research to complete 4. **Anakin DataPoller** — fetch the completed results using the job ID 5. **Gmail** — send the research report #### AI search to Notion ``` Webhook → Anakin (Search) → Notion (Create Page) ``` 1. **Webhook** — receive a search query 2. **Anakin Search** — get instant AI-powered results 3. **Notion** — create a page with the search results #### Popular connections Google Sheets, OpenAI, Gmail, Google Drive, Telegram Bot, Airtable, Notion, Google Docs, Slack, Shopify, HubSpot CRM, WordPress, and [more](https://www.make.com/en/integrations/anakin). --- ### Troubleshooting | Issue | Fix | |-------|-----| | Connection failed | Verify your API key is correct and has credits | | Scrape returns empty | Try enabling browser mode if the option is available, or check if the URL is accessible | | Agentic search still processing | Add a **Delay** module (3–5 min) before polling with DataPoller | | Rate limit error (429) | Add a delay between requests or reduce scenario frequency | --- # n8n (/docs/integrations/workflow/n8n) <ComingSoonBanner description="This is currently under active development. Stay tuned for updates." /> | | | |---|---| | **Platform** | [n8n](https://n8n.io) | | **Type** | Community Node | --- ### What to expect The n8n community node will provide: - **Scrape URL** — Extract content and structured data from any website with automatic polling - **AI Search** — Synchronous AI-powered web search with instant results - **Agentic Search** — Multi-stage deep research pipeline with structured data extraction --- ### In the meantime You can integrate AnakinScraper into your n8n workflows today using n8n's **HTTP Request** node with the [REST API](/docs/api-reference), or use the [Anakin CLI](/docs/sdks/cli) via the **Execute Command** node. --- ### Stay updated - **Discord** — [discord.gg/T57dHrdT8u](https://discord.gg/T57dHrdT8u) - **Email** — support@anakin.io --- # Zapier (/docs/integrations/workflow/zapier) Extract structured data from websites and perform AI-powered searches — connected to 8,000+ apps on Zapier. | | | |---|---| | **Marketplace** | [zapier.com/apps/anakin](https://zapier.com/apps/anakin/integrations) | | **Category** | AI Agents | | **Status** | Beta | --- ### Setup #### 1. Create a new Zap Go to [zapier.com/app/zaps](https://zapier.com/app/zaps) and click **Create Zap**. #### 2. Add a trigger Choose any trigger that produces data you want to scrape or research — Webhooks by Zapier, Schedule, Gmail, Slack, or any of 8,000+ apps. #### 3. Add an Anakin action 1. Click **+** to add an action 2. Search for **Anakin** 3. Choose your action (Extract Website Data, Perform AI Search, etc.) 4. Click **Sign in** and enter your API key (get one from the [Dashboard](/dashboard)) 5. Map fields from your trigger 6. Click **Test action**, then **Publish** --- ### Actions #### Extract Website Data Extracts structured data from any website including HTML, markdown, and generated JSON. Submits a scrape job and polls until complete. | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `url` | string | Yes | — | The full URL of the website to extract data from | | `country` | string | No | `us` | Country code for proxy routing (e.g., `us`, `gb`, `de`). See [supported countries](/docs/api-reference/supported-countries) | | `forceFresh` | boolean | No | `false` | Bypass cache and force fresh data extraction | | `maxWaitTime` | integer | No | `300` | Maximum seconds to wait for the job to complete | | `pollInterval` | integer | No | `3` | Seconds between status checks | **Output fields:** | Field | Type | Description | |-------|------|-------------| | `url` | string | The URL that was scraped | | `status` | string | Job status (`completed` or `failed`) | | `result` | text | Raw HTML content | | `markdown` | text | Clean markdown version of the page | | `generatedJson` | string | AI-extracted structured data | | `cached` | boolean | Whether the result came from cache | | `success` | boolean | Success flag | | `durationMs` | number | Processing time in milliseconds | --- #### Perform AI Search AI-powered search. Returns instant answers with citations and sources. This is synchronous — results are returned immediately. | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `searchQuery` | text | Yes | — | The search query or question (e.g., "What are the latest trends in AI?") | | `maxResults` | integer | No | `5` | Maximum number of search results to return | --- #### Start Agentic Search Starts an advanced AI search job that searches web sources and extracts structured data. Returns a job ID to check results later. Takes 1–5 minutes. | Field | Type | Required | Description | |-------|------|----------|-------------| | `searchPrompt` | text | Yes | The research question or topic | | `useBrowser` | boolean | No | Use headless browser for more reliable scraping | **Output:** Returns a `job_id` to use with **Get Agentic Search Results**. --- #### Get Agentic Search Results Fetches current status and results for an agentic search job. Returns immediately with the current state (`processing`, `completed`, or `failed`). | Field | Type | Required | Description | |-------|------|----------|-------------| | `jobId` | string | Yes | The job ID from the Start Agentic Search action | --- ### Examples #### Scrape a URL from a webhook 1. **Trigger:** Webhooks by Zapier — Catch Hook 2. **Action:** Anakin — Extract Website Data (map webhook URL to `url`) 3. **Action:** Google Sheets — Create Row (save markdown and structured data) Send data to your webhook: ```bash curl -X POST YOUR_WEBHOOK_URL \ -H "Content-Type: application/json" \ -d '{"url": "https://example.com"}' ``` #### Daily product monitoring 1. **Trigger:** Schedule by Zapier — Every Day 2. **Action:** Anakin — Extract Website Data (hardcode the product URL) 3. **Action:** Google Sheets — Create Row (append price data) 4. **Action:** Filter — Only continue if price changed 5. **Action:** Slack — Send Channel Message #### Research pipeline 1. **Trigger:** Webhook (receives a research topic) 2. **Action:** Anakin — Start Agentic Search 3. **Action:** Delay — Wait 3 minutes 4. **Action:** Anakin — Get Agentic Search Results 5. **Action:** Gmail — Send Email with the report #### AI search to Airtable 1. **Trigger:** Airtable — New Record (with a search query field) 2. **Action:** Anakin — Perform AI Search (map the query field) 3. **Action:** Airtable — Update Record (write results back) #### More ideas - **Price monitoring** — Scrape product pages on a schedule, save to Google Sheets, alert on changes - **Lead enrichment** — Scrape company websites from a CRM trigger, extract structured data - **News aggregation** — Scrape articles from RSS URLs, save cleaned markdown to Notion - **Competitor analysis** — Monitor competitor pricing pages, compare with previous data - **Market research** — Run agentic search on topics, email structured reports --- ### Troubleshooting | Issue | Fix | |-------|-----| | Scrape returns empty | Try setting `forceFresh` to true to bypass cache | | Timeout | Increase `maxWaitTime` (default is 300 seconds) | | Auth error | Re-enter your API key in the Zapier connection settings | | Agentic search still processing | Use a **Delay** action (3–5 min) between Start and Get Results | | Geo-restricted content | Set `country` to match the target site's region | --- # Releases (/docs/releases) A stable record of every released version across our four published packages. Each entry links to the npm or PyPI registry, the GitHub release, and the source commit it was built from. | Package | Latest | Source | |---|---|---| | [`@anakin-io/sdk`](https://www.npmjs.com/package/@anakin-io/sdk) | `0.1.0` | [github.com/Anakin-Inc/anakin-node](https://github.com/Anakin-Inc/anakin-node) | | [`@anakin-io/mcp`](https://www.npmjs.com/package/@anakin-io/mcp) | `0.1.0` | [github.com/Anakin-Inc/anakin-mcp](https://github.com/Anakin-Inc/anakin-mcp) | | [`anakin-sdk`](https://pypi.org/project/anakin-sdk/) (PyPI) | `0.1.0` | [github.com/Anakin-Inc/anakin-py](https://github.com/Anakin-Inc/anakin-py) | | [`anakin-cli`](https://pypi.org/project/anakin-cli/) (PyPI) | `0.2.0` | [github.com/Anakin-Inc/anakin-cli](https://github.com/Anakin-Inc/anakin-cli) | --- ## 2026-04-28 — Initial public release of three new packages Three packages went live to npm + PyPI on the same day. The fourth (`anakin-cli`) was already published earlier (last bumped to `0.2.0` on 2026-03-18) and is unchanged in this release window. ### `@anakin-io/sdk` v0.1.0 — Node.js / TypeScript SDK First public release. Wraps every Anakin endpoint in a single async call with built-in polling, real TypeScript types, and zero runtime dependencies. ```bash npm install @anakin-io/sdk ``` ```typescript import { Anakin } from "@anakin-io/sdk" const client = new Anakin({ apiKey: "ak-..." }) const doc = await client.scrape("https://example.com") console.log(doc.markdown) ``` **What's included:** `scrape`, `map`, `crawl`, `search`, `agenticSearch`, `wire`, plus a full `SessionsClient` for browser-session CRUD (`list`, `create`, `save`, `update`, `delete`). **Quality gates:** 25 unit tests, strict TypeScript (`exactOptionalPropertyTypes`), ESM + CJS dual build via tsup (~28 KB), zero runtime dependencies. Works on Node 18+. **Links:** [npm](https://www.npmjs.com/package/@anakin-io/sdk) · [GitHub release](https://github.com/Anakin-Inc/anakin-node/releases/tag/v0.1.0) · [docs](/docs/sdks/node) --- ### `@anakin-io/mcp` v0.1.0 — Model Context Protocol server First public release. Lets AI agents in **Claude Desktop, Claude Code, Cursor, Cline, Continue, Zed, Windsurf, and VS Code** call Anakin natively — no glue code required. ```bash npx -y @anakin-io/mcp init --all ``` That single command auto-configures every detected agent client on your machine. After it finishes, restart your client and Anakin's tools appear in the agent's tool list. **Tools exposed (6):** `scrape`, `search`, `map`, `crawl`, `agentic_search`, `wire_action` — each a thin wrapper over the corresponding Anakin REST endpoint with a typed JSON Schema for arguments. **Supported clients (8):** `claude-desktop`, `claude-code`, `cursor`, `cline`, `continue`, `zed`, `windsurf`, `vscode`. Each has its own config schema; all are auto-detected and configured by `init`. **Quality gates:** 27 unit tests across the tool registry shape and per-client config writers, strict TypeScript, ESM + CJS dual build. Only `@modelcontextprotocol/sdk` at runtime. **Links:** [npm](https://www.npmjs.com/package/@anakin-io/mcp) · [GitHub release](https://github.com/Anakin-Inc/anakin-mcp/releases/tag/v0.1.0) · [setup docs](/docs/integrations/ai-agents/mcp-server) --- ### `anakin-sdk` v0.1.0 — Python SDK First public release. Wraps every Anakin endpoint with a single sync call, typed Pydantic v2 response models, and built-in polling. ```bash pip install anakin-sdk ``` ```python from anakin import Anakin client = Anakin(api_key="ak-...") # or set ANAKIN_API_KEY doc = client.scrape("https://example.com") print(doc.markdown) ``` > **Distribution name** is `anakin-sdk` (PyPI). **Import name** is `anakin`. They differ because the unscoped `anakin` PyPI name is held by an unrelated package. **What's included:** `scrape`, `map`, `crawl`, `search`, `agentic_search`, `wire`, `countries`, plus a `SessionsClient` for browser-session CRUD. Built-in retries with exponential backoff and `Retry-After` honour on 429s. Hidden polling — calls return the final result. **Validated on:** Python **3.10, 3.11, 3.12, 3.13** — install + import + 25/25 tests pass on every version. Multi-version validation is automated via [`scripts/test-multiver.sh`](https://github.com/Anakin-Inc/anakin-py/blob/main/scripts/test-multiver.sh) in the repo. **Links:** [PyPI](https://pypi.org/project/anakin-sdk/) · [GitHub release](https://github.com/Anakin-Inc/anakin-py/releases/tag/v0.1.0) · [docs](/docs/sdks/python) --- ## Earlier releases ### `anakin-cli` v0.2.0 — 2026-03-18 The Python CLI for terminal-driven scraping. Installs `anakin` as a binary on `$PATH`. Already published before this release window. ```bash pip install anakin-cli anakin login --api-key "ak-..." anakin scrape "https://example.com" -o page.md ``` **Commands:** `search`, `scrape`, `scrape-batch`, `research`, `login`, `status`. **Links:** [PyPI](https://pypi.org/project/anakin-cli/) · [docs](/docs/sdks/cli) --- ## Versioning All four packages follow [Semantic Versioning](https://semver.org/): - **0.x.y** — alpha. Public API may change between minor versions until 1.0. - **x.0.0** (1.0+) — stable. Breaking changes require a major bump. - **Patch (z) bumps** — bug fixes, no API changes. Until 1.0, treat the SDKs as alpha and pin to a specific minor version (`@anakin-io/sdk@^0.1.0`, `anakin-sdk==0.1.*`) if you need stability across upgrades. ## Status badges If you'd like to embed real-time version badges on your own README: ```markdown [![npm](https://img.shields.io/npm/v/@anakin-io/sdk.svg)](https://www.npmjs.com/package/@anakin-io/sdk) [![npm](https://img.shields.io/npm/v/@anakin-io/mcp.svg)](https://www.npmjs.com/package/@anakin-io/mcp) [![PyPI](https://img.shields.io/pypi/v/anakin-sdk.svg)](https://pypi.org/project/anakin-sdk/) [![PyPI](https://img.shields.io/pypi/v/anakin-cli.svg)](https://pypi.org/project/anakin-cli/) ``` ## Reporting issues Each repo has its own issue tracker — file there for fastest response: - [`@anakin-io/sdk` issues](https://github.com/Anakin-Inc/anakin-node/issues) - [`@anakin-io/mcp` issues](https://github.com/Anakin-Inc/anakin-mcp/issues) - [`anakin-sdk` issues](https://github.com/Anakin-Inc/anakin-py/issues) - [`anakin-cli` issues](https://github.com/Anakin-Inc/anakin-cli/issues) For bugs that span multiple packages or are about the API itself, the [main repo's issue tracker](https://github.com/Anakin-Inc/blueprint-scribe-35/issues) is the catch-all. --- # Overview (/docs/geo-aeo) **GEO (Generative Engine Optimization)** and **AEO (Answer Engine Optimization)** come down to one thing: making sure the AI engines your buyers ask — ChatGPT, Gemini, Perplexity, Google AI Overviews — describe and cite your brand the way you want. Anakin gives you the raw signal to measure and improve it. Send a prompt to any engine and get its **real answer** back, with the **exact sources it cited** — 1:1 with the logged-in experience, from one API. ### Why it matters - Buyers increasingly ask an AI engine *before* they ever see a search result page. - Each engine answers differently and cites different sources — one engine isn't the whole picture. - You can't improve what you can't see. GEO/AEO starts with measuring the answers, verbatim. ### Explore <CardGrid> <ProductCard title="Engines" description="Every engine, its action_id, params, and response — start here" href="/docs/geo-aeo/engines" /> <ProductCard title="Pricing" description="Usage-based hits pricing from $100 to $5,000/mo" href="/docs/geo-aeo/pricing" /> <ProductCard title="Wire Task API" description="The POST /v1/wire/task + poll endpoints every engine uses" href="/docs/api-reference/wire/execute-task" /> <ProductCard title="AI Visibility API" description="Fan one prompt out to multiple engines in a single call" href="/docs/api-reference/ai-visibility" /> </CardGrid> ### How it works Each engine is a **Wire action** you call with one endpoint. Three steps: 1. **Get an API key** — from **Settings → API Keys** in the dashboard. Pass it as `X-API-Key` on every request. 2. **Submit a prompt** — `POST /v1/wire/task` with the engine's `action_id` and params. It returns a `job_id`. 3. **Poll for the answer** — `GET /v1/wire/jobs/{job_id}` until `status: "completed"`, or subscribe to the signed `wire.job.completed` [webhook](/docs/api-reference/webhooks). You get the full answer plus the citations behind it. ```bash curl -X POST https://api.anakin.io/v1/wire/task \ -H "X-API-Key: $ANAKIN_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "action_id": "perplexity", "params": { "query": "best CRM for small teams", "country": "us" } }' # → { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` Pick your engine on the [Engines](/docs/geo-aeo/engines) page — each has its exact params and response shape. Want one call across many engines at once? Use the [AI Visibility API](/docs/api-reference/ai-visibility). ### Related - **Guide** — [AI Visibility](/docs/documentation/ai-visibility): concepts, the dashboard flow, and outputs - **Product** — [LLM Visibility for SEO & GEO teams](/use-cases/llm-visibility) --- # All engines (/docs/geo-aeo/engines) Each AI engine is a **Wire action** you call with `POST /v1/wire/task`. Pass the engine's `action_id` and its params, then either poll `GET /v1/wire/jobs/{id}` or receive a webhook for the answer and the sources it cited. | Engine | Catalog | `action_id` | Credits | Cites sources | |--------|---------|-------------|:---:|:---:| | [ChatGPT](/docs/geo-aeo/engines/chatgpt) | [ai-assistants](https://anakin.io/wire/ai-assistants) | `chatgpt` | 2 | ✓ | | [Gemini](/docs/geo-aeo/engines/gemini) | [gemini](https://anakin.io/wire/gemini) | `gemini_prompt` | 2 | ✓ | | [Perplexity](/docs/geo-aeo/engines/perplexity) | [perplexity](https://anakin.io/wire/perplexity) | `perplexity` | 2 | ✓ | | [Google AI Overview](/docs/geo-aeo/engines/google-ai-overview) | [google_ai_mode](https://anakin.io/wire/google_ai_mode) | `google_ai_overview` | 2 | ✓ | | [Google AI Mode](/docs/geo-aeo/engines/google-ai-mode) | [google_ai_mode](https://anakin.io/wire/google_ai_mode) | `ai_mode` | 2 | ✓ | | [Naver](/docs/geo-aeo/engines/naver) | [naver](https://anakin.io/wire/naver) | `naver_prompt` | 2 | ✓ | | [Wenxin (ERNIE)](/docs/geo-aeo/engines/wenxin) | [wenxin](https://anakin.io/wire/wenxin) | `wenxin_prompt` | 2 | ✓ | ### The call, in brief **1. Submit** — `POST /v1/wire/task` returns a `job_id` immediately: ```bash curl -X POST https://api.anakin.io/v1/wire/task \ -H "X-API-Key: $ANAKIN_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "action_id": "perplexity", "params": { "query": "best CRM for small teams", "country": "us" } }' # → { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` **2. Get the result** — two ways: <Tabs items={["Poll", "Webhook"]}> <Tab value="Poll"> `GET /v1/wire/jobs/{job_id}` until `status: "completed"`: ```bash curl https://api.anakin.io/v1/wire/jobs/$JOB_ID -H "X-API-Key: $ANAKIN_API_KEY" # → { "status": "completed", "credits_used": 2, "data": { "status": "ok", "data": { … } } } ``` </Tab> <Tab value="Webhook"> Add a `webhook_url` to the submit body — then **don't poll**. We `POST` a signed `wire.job.completed` event to your URL the moment the job settles (within ~10s): ```json { "action_id": "perplexity", "params": { "query": "best CRM for small teams", "country": "us" }, "webhook_url": "https://your-app.com/hooks/anakin" } ``` Your endpoint receives: ```json { "id": "evt_5f2a9c…", "type": "wire.job.completed", "createdAt": "2026-07-13T10:00:05Z", "data": { "jobId": "…", "status": "completed", "creditsUsed": 2, "result_url": "https://api.anakin.io/v1/wire/jobs/…", "result": { "status": "ok", "data": { "…": "inlined when the envelope is ≤ 256 KB" } } } } ``` `result` is inlined for small answers; for large ones (`resultOmitted: "size"`) `GET` the always-present `result_url` with your API key — the same shape as polling. Verify the `X-Anakin-Signature` header (`sha256=HMAC_SHA256(secret, rawBody)`). See [Webhooks](/docs/api-reference/webhooks). </Tab> </Tabs> See any engine page for its exact params and response shape. --- # Pricing (/docs/geo-aeo/pricing) GEO / AI Visibility is priced on **hits** (engine answers). Buy a monthly volume and your per-1K rate drops as you scale. You only pay for **successful** calls — failed or retried sources are free. ### Volume bands | Purchase / mo | Price / 1K hits | Total hits | Credits (2 × hits) | |---------------|:---:|------------:|------------:| | $100 | $1.50 | ~67K | ~133K | | $200 – $900 | $1.00 | 200K – 900K | 400K – 1.8M | | $1,000 – $2,500 | $0.95 | ~1.05M – 2.63M | ~2.11M – 5.26M | | $3,000 – $4,500 | $0.90 | ~3.33M – 5.00M | ~6.67M – 10.00M | | $5,000+ | $0.85 | ~5.88M+ | ~11.76M+ | Each hit is **2 credits**, so your credit balance is always 2 × your hits. Pick any step from **$100 to $5,000/mo** — the [plan builder](/use-cases/llm-visibility) shows the exact hits and rate for each. Above $5,000/mo, [talk to sales](https://calendly.com/d/ctqw-64s-rgt/let-s-talk-about-your-use-case-anakin-io) for custom volume and terms. ### How billing works - **Monthly subscription.** Your hit allowance refills every billing cycle; unused plan hits don't roll over. - **Top-ups.** Need more mid-cycle? Buy on-demand hits in $10 blocks at your plan's rate — permanent (they never expire) and spent after your monthly allowance. - **One balance.** GEO hits are the same credits used across the rest of Anakin, so there's no separate wallet to manage. ### Credit cost per engine A "hit" is one engine answer, but engines cost different amounts of credits to serve (see each [engine page](/docs/geo-aeo/engines) for details): | Engine | Credits / request | |--------|:---:| | ChatGPT, Gemini, Perplexity, Google AI Overview, Google AI Mode, Naver, Wenxin | 2 | **Next:** [Build your plan](/use-cases/llm-visibility) · [Browse the engines](/docs/geo-aeo/engines) --- # ChatGPT (/docs/geo-aeo/engines/chatgpt) <EndpointBanner method="POST" path="/task" /> Run a prompt against ChatGPT and get the full answer with the web sources it cited when search is on — 1:1 with the real logged-in experience. Run it with the `chatgpt` action on [POST /v1/wire/task](/docs/api-reference/wire/execute-task). **Catalog:** [ai-assistants](https://anakin.io/wire/ai-assistants) · **`action_id`:** `chatgpt` · **Credits:** 2 / call · **Auth:** none --- ### Request Body ```json { "action_id": "chatgpt", "params": { "prompt": "best CRM for small teams", "web_search": true, "country": "us" } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | Set to `chatgpt` | | `params.prompt` **required** | string | The prompt to send to ChatGPT (max 4096 chars) | | `params.web_search` | boolean | Request live web search. The response reports whether search actually ran. Default `false` | | `params.additional_prompt` | string | Optional follow-up sent in the same conversation | | `params.include_html` | boolean | Also include `answer_html` (rendered DOM, ~0.5 MB). Default `false` | | `params.country` | string | 2-letter proxy country for the session. Default `us` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "chatgpt", "params": { "prompt": "best CRM for small teams", "web_search": true, "country": "us" } }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={ 'action_id': 'chatgpt', 'params': {'prompt': 'best CRM for small teams', 'web_search': True, 'country': 'us'}, }, ) job_id = res.json()['job_id'] ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'chatgpt', params: { prompt: 'best CRM for small teams', web_search: true, country: 'us' }, }), }); const { job_id } = await res.json(); ``` </Tab> </Tabs> --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` Poll [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) until `status: "completed"`: <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "credits_used": 2, "execution_ms": 17889, "data": { "status": "ok", "data": { "prompt": "best CRM for small teams", "title": "Best CRMs for small teams", "answer_text": "For small teams, the strongest options are …", "answer_text_markdown": "…", "citations": [ { "ref": 1, "url": "https://…", "title": "…", "domain": "…", "position": 1, "ref_type": "web", "description": "…", "result_source": "search", "date_published": "…", "cited": true } ], "search_sources": [], "web_search_queries": ["best CRM small teams 2026"], "web_search_triggered": true, "ads": [], "shopping": [], "entities": [], "recommendations": [], "business_locations": [], "model": "gpt-…", "country": "us", "timestamp": "…" }, "meta": { "action_id": "chatgpt", "catalog_slug": "ai-assistants", "envelope_version": "v1" } } } ``` | Field | Type | Description | |-------|------|-------------| | `data.data.answer_text` | string | The plain-text answer (also `answer_text_markdown`, and `answer_html` when `include_html`) | | `data.data.citations` | array | `{ ref, url, title, domain, position, ref_type, description, result_source, date_published, cited }` | | `data.data.search_sources` | array | Sources ChatGPT pulled during web search | | `data.data.web_search_queries` | string[] | The queries ChatGPT actually searched | | `data.data.web_search_triggered` | boolean | Whether live search ran | | `data.data.ads` | array | Ad slots when served | > Set `web_search: true` to force live sources — without it ChatGPT may answer from model knowledge with no citations. Billed **only on success**. --- # Gemini (/docs/geo-aeo/engines/gemini) <EndpointBanner method="POST" path="/task" /> Run a prompt against Gemini and get its answer with grounding sources and citations, plus multi-turn support via `conversation_id`. Run it with the `gemini_prompt` action on [POST /v1/wire/task](/docs/api-reference/wire/execute-task). **Catalog:** [gemini](https://anakin.io/wire/gemini) · **`action_id`:** `gemini_prompt` · **Credits:** 2 / call · **Auth:** none --- ### Request Body ```json { "action_id": "gemini_prompt", "params": { "prompt": "best CRM for small teams", "country": "us" } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | Set to `gemini_prompt` | | `params.prompt` **required** | string | Your question or instruction for Gemini (up to 8192 chars) | | `params.country` | string | Region to send the request from (2-letter code). Default `us` | | `params.additional_prompt` | string | Optional follow-up, asked in the same conversation | | `params.include_html` | boolean | Also return the answer as HTML under `answer_html`. Default `false` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "gemini_prompt", "params": { "prompt": "best CRM for small teams", "country": "us" } }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={'action_id': 'gemini_prompt', 'params': {'prompt': 'best CRM for small teams', 'country': 'us'}}, ) job_id = res.json()['job_id'] ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'gemini_prompt', params: { prompt: 'best CRM for small teams', country: 'us' }, }), }); const { job_id } = await res.json(); ``` </Tab> </Tabs> --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` Poll [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) until `status: "completed"`: <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "credits_used": 2, "execution_ms": 19948, "data": { "status": "ok", "data": { "answer_text": "…", "answer_text_markdown": "…", "citations": [ { "url": "https://…", "title": "…", "domain": "…", "position": 1, "ref_type": "web", "description": "…", "result_source": "search", "cited": true } ], "sources": [], "search_sources": [], "conversation_id": "…", "response_id": "…", "related_media": [], "detected_location": "…", "language": "en", "tools_used": ["search"], "model": "gemini-…" }, "meta": { "action_id": "gemini_prompt", "catalog_slug": "gemini", "envelope_version": "v1" } } } ``` | Field | Type | Description | |-------|------|-------------| | `data.data.answer_text` | string | The answer (also `answer_text_markdown`, and `answer_html` when `include_html`) | | `data.data.citations` | array | Grounding sources — `{ url, title, domain, position, ref_type, description, result_source, cited }` | | `data.data.conversation_id` | string | Reuse to continue the thread (multi-turn) | | `data.data.detected_location` | string | How Gemini localized the answer for `country` | | `data.data.tools_used` | string[] | Tools Gemini invoked (e.g. `search`) | > Billed **only on success**. --- # Google AI Mode (/docs/geo-aeo/engines/google-ai-mode) <EndpointBanner method="POST" path="/task" /> Google **AI Mode** is the dedicated conversational AI answer surface. Send a prompt and get the answer with its citations, plus any shopping/local modules Google attaches. Run it with the `ai_mode` action on [POST /v1/wire/task](/docs/api-reference/wire/execute-task). **Catalog:** [google_ai_mode](https://anakin.io/wire/google_ai_mode) · **`action_id`:** `ai_mode` · **Credits:** 2 / call · **Auth:** none --- ### Request Body ```json { "action_id": "ai_mode", "params": { "prompt": "best CRM for small teams", "country": "us" } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | Set to `ai_mode` | | `params.prompt` **required** | string | The prompt to send to Google AI Mode (max 4096 chars) | | `params.additional_prompt` | string | Optional follow-up in the same conversation | | `params.country` | string | 2-letter proxy country for the browser session | | `params.include_html` | boolean | Include `answer_html` (full stream body, ~0.3 MB). Default `false` | | `params.device` | string | User-agent profile (`desktop`, `desktop_chrome`, `mobile`, …). Default `desktop` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "ai_mode", "params": { "prompt": "best CRM for small teams", "country": "us" } }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={'action_id': 'ai_mode', 'params': {'prompt': 'best CRM for small teams', 'country': 'us'}}, ) job_id = res.json()['job_id'] ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'ai_mode', params: { prompt: 'best CRM for small teams', country: 'us' }, }), }); const { job_id } = await res.json(); ``` </Tab> </Tabs> --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` Poll [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) until `status: "completed"`: <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "credits_used": 2, "execution_ms": 21652, "data": { "status": "ok", "data": { "answer_text": "…", "answer_text_markdown": "…", "answer_html": "…", "answer_section_html": "…", "citations": [], "references": [], "search_sources": [], "products": [], "businesses": [], "ads": [], "engine": "ai_mode" }, "meta": { "action_id": "ai_mode", "catalog_slug": "google_ai_mode", "envelope_version": "v1" } } } ``` | Field | Type | Description | |-------|------|-------------| | `data.data.answer_text` | string | The answer (also `answer_text_markdown`, and `answer_html` when `include_html`) | | `data.data.citations` | array | Sources cited in the answer | | `data.data.search_sources` | array | Sources considered during search | | `data.data.products` | array | Shopping modules for commercial-intent queries | | `data.data.businesses` | array | Local results for local-intent queries | > AI Mode is geo-sensitive — availability and phrasing vary by `country`. Billed **only on success**. --- # Google AI Overview (/docs/geo-aeo/engines/google-ai-overview) <EndpointBanner method="POST" path="/task" /> Fetch the **AI Overview** Google shows above organic results, plus the surrounding SERP — organic results, People-Also-Ask, knowledge panel, and more. Highly geo-dependent. Run it with the `google_ai_overview` action on [POST /v1/wire/task](/docs/api-reference/wire/execute-task). **Catalog:** [google_ai_mode](https://anakin.io/wire/google_ai_mode) · **`action_id`:** `google_ai_overview` · **Credits:** 2 / call · **Auth:** none --- ### Request Body ```json { "action_id": "google_ai_overview", "params": { "query": "best CRM for small teams", "country": "us" } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | Set to `google_ai_overview` | | `params.query` **required** | string | The search query (max 2048 chars) | | `params.country` | string | 2-letter ISO country for the search locale (239 supported). Default `us` | | `params.location` | string | Location override: `lat,lng[,radius_m]` or a canonical place | | `params.uule` | string | Ready-made Google UULE cookie value (overrides `location`) | | `params.page` | integer | SERP page (1–10). Page 1 = AI Overview + organic. Default `1` | | `params.html` | boolean | Return a link to the full SERP snapshot instead of inline HTML. Default `false` | | `params.paa_aioverview` | boolean | Hydrate People-Also-Ask items with their own AI-Overview markdown. Default `false` | | `params.device` | string | User-agent profile (`desktop`, `desktop_chrome`, `mobile`, …). Default `desktop` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "google_ai_overview", "params": { "query": "best CRM for small teams", "country": "us" } }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={'action_id': 'google_ai_overview', 'params': {'query': 'best CRM for small teams', 'country': 'us'}}, ) job_id = res.json()['job_id'] ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'google_ai_overview', params: { query: 'best CRM for small teams', country: 'us' }, }), }); const { job_id } = await res.json(); ``` </Tab> </Tabs> --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` Poll [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) until `status: "completed"`: <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "credits_used": 2, "execution_ms": 16797, "data": { "status": "ok", "data": { "overview_text": "…", "overview_markdown": "…", "sources": [ { "url": "https://…", "title": "…", "domain": "…", "snippet": "…", "hostname": "…", "source_type": "web" } ], "claim_references": [], "organic": [], "featured_snippet": {}, "knowledge_panel": {}, "related_questions": [], "related_searches": [], "top_stories": [], "videos": [], "products": [], "discussions": [], "what_people_are_saying": [], "serp_layout": [], "device": "desktop", "page": 1 }, "meta": { "action_id": "google_ai_overview", "catalog_slug": "google_ai_mode", "envelope_version": "v1" } } } ``` | Field | Type | Description | |-------|------|-------------| | `data.data.overview_text` | string | The AI Overview text (also `overview_markdown`). `null` when Google serves no overview | | `data.data.sources` | array | AI Overview citations — `{ url, title, domain, snippet, hostname, source_type }` | | `data.data.claim_references` | array | Per-claim source mapping — the sharpest GEO signal | | `data.data.organic` | array | Organic SERP results | | `data.data.related_questions` | array | People-Also-Ask items | | `data.data.serp_layout` | array | Order of SERP modules on the page | > Google doesn't serve an AI Overview for every query/geo — check `overview_text` for `null` before relying on it (the SERP modules are still returned). Use `location`/`uule` for city-level targeting. Billed **only on success**. --- # Naver (/docs/geo-aeo/engines/naver) <EndpointBanner method="POST" path="/task" /> Naver **AI Briefing** is the AI answer surface on Korea's dominant search portal. Send a prompt (best in Korean) and get the answer with its citations and sources. Run it with the `naver_prompt` action on [POST /v1/wire/task](/docs/api-reference/wire/execute-task). **Catalog:** [naver](https://anakin.io/wire/naver) · **`action_id`:** `naver_prompt` · **Credits:** 2 / call · **Auth:** none --- ### Request Body ```json { "action_id": "naver_prompt", "params": { "prompt": "서울 커피 맛집 추천", "country": "kr" } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | Set to `naver_prompt` | | `params.prompt` **required** | string | Your question for Naver AI Briefing (best in Korean) | | `params.country` | string | Egress country (2-letter). Supported: `kr`, `us`, … Default `kr` | | `params.include_html` | boolean | Also return the answer as HTML under `answer_html`. Default `false` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "naver_prompt", "params": { "prompt": "서울 커피 맛집 추천", "country": "kr" } }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={'action_id': 'naver_prompt', 'params': {'prompt': '서울 커피 맛집 추천', 'country': 'kr'}}, ) job_id = res.json()['job_id'] ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'naver_prompt', params: { prompt: '서울 커피 맛집 추천', country: 'kr' }, }), }); const { job_id } = await res.json(); ``` </Tab> </Tabs> --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` Poll [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) until `status: "completed"`: <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "credits_used": 3, "data": { "status": "ok", "data": { "prompt": "…", "title": "…", "aib_available": true, "no_info": false, "answer_text": "…", "answer_text_markdown": "…", "answer_html": "…", "citations": [], "search_sources": [], "references": [], "related_questions": [], "recommendations": [], "shopping": [], "ads": [], "disclaimer": "…", "conversation_id": "…" }, "meta": { "action_id": "naver_prompt", "catalog_slug": "naver", "envelope_version": "v1" } } } ``` | Field | Type | Description | |-------|------|-------------| | `data.data.aib_available` | boolean | Whether an AI Briefing was served — check before relying on `answer_text` | | `data.data.answer_text` | string | The answer (also `answer_text_markdown`, and `answer_html` when `include_html`) | | `data.data.citations` | array | Sources cited in the answer | | `data.data.related_questions` | array | Follow-up questions Naver surfaces | > Answers are strongest for Korean-language, Korea-geo queries. Check `aib_available` before using `answer_text`. Billed **only on success**. --- # Perplexity (/docs/geo-aeo/engines/perplexity) <EndpointBanner method="POST" path="/task" /> Perplexity is an answer engine: it returns the answer with **inline `[n]` citation markers** and the full list of web sources behind them. Run it with the `perplexity` action on [POST /v1/wire/task](/docs/api-reference/wire/execute-task). **Catalog:** [perplexity](https://anakin.io/wire/perplexity) · **`action_id`:** `perplexity` · **Credits:** 2 / call · **Auth:** none --- ### Request Body ```json { "action_id": "perplexity", "params": { "query": "What is the capital of France?", "focus": "internet", "country": "us" } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | Set to `perplexity` | | `params.query` **required** | string | The question/prompt to ask Perplexity | | `params.focus` | string | `internet` (web search) or `writing` (no web). Default `internet` | | `params.country` | string | Residential exit country (2-letter ISO). Default `us` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "perplexity", "params": { "query": "best CRM for small teams", "focus": "internet", "country": "us" } }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={ 'action_id': 'perplexity', 'params': {'query': 'best CRM for small teams', 'focus': 'internet', 'country': 'us'}, }, ) job_id = res.json()['job_id'] ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'perplexity', params: { query: 'best CRM for small teams', focus: 'internet', country: 'us' }, }), }); const { job_id } = await res.json(); ``` </Tab> </Tabs> --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "c2916a5d-d4e8-49b4-bf24-f1e85321845a", "poll_url": "/v1/wire/jobs/c2916a5d-d4e8-49b4-bf24-f1e85321845a" } ``` Poll [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) until `status: "completed"`: <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "credits_used": 2, "execution_ms": 6061, "data": { "status": "ok", "data": { "query": "best CRM for small teams", "answer_text": "The strongest options are … [1][4]", "citations": [ { "title": "…", "url": "https://…", "position": 1, "snippet": "…", "domain": "…", "image": "https://…" } ], "search_sources": [], "web_search_triggered": true, "related_queries": ["best CRM for startups"], "reformulated_query": "top CRM software for small teams", "thread_title": "Best CRMs for small teams", "media": [], "shopping": [], "model": "…" }, "meta": { "action_id": "perplexity", "catalog_slug": "perplexity", "envelope_version": "v1" } } } ``` | Field | Type | Description | |-------|------|-------------| | `data.data.answer_text` | string | The answer, with inline `[n]` markers mapping to `citations[n-1].position` | | `data.data.citations` | array | `{ title, url, position, snippet, domain, image }` — the web sources cited | | `data.data.search_sources` | array | All sources considered (superset of citations) | | `data.data.related_queries` | string[] | Follow-up queries Perplexity suggests | | `data.data.reformulated_query` | string | How Perplexity rewrote your query — a useful GEO signal | | `data.data.web_search_triggered` | boolean | Whether live web search ran | > `focus: "writing"` skips web search — the answer comes from the model with no citations. Billed **only on success**. --- # Wenxin (ERNIE) (/docs/geo-aeo/engines/wenxin) <EndpointBanner method="POST" path="/task" /> Wenxin (文心一言 / ERNIE) is Baidu's AI assistant. Send a prompt in any language and get the answer with its citations and web sources. Run it with the `wenxin_prompt` action on [POST /v1/wire/task](/docs/api-reference/wire/execute-task). **Catalog:** [wenxin](https://anakin.io/wire/wenxin) · **`action_id`:** `wenxin_prompt` · **Credits:** 2 / call · **Auth:** none --- ### Request Body ```json { "action_id": "wenxin_prompt", "params": { "prompt": "What are the best running shoes in 2026?" } } ``` | Parameter | Type | Description | |-----------|------|-------------| | `action_id` **required** | string | Set to `wenxin_prompt` | | `params.prompt` **required** | string | Your question for ERNIE (up to 8192 chars, any language) | | `params.additional_prompt` | string | Optional follow-up, asked as a second turn | | `params.include_html` | boolean | Also return the answer as HTML under `answer_html`. Default `false` | --- ### Code Examples <Tabs items={["cURL", "Python", "JavaScript"]}> <Tab value="cURL"> ```bash curl -X POST {API_BASE}/task \ -H "X-API-Key: your_api_key" \ -H "Content-Type: application/json" \ -d '{ "action_id": "wenxin_prompt", "params": { "prompt": "What are the best running shoes in 2026?" } }' ``` </Tab> <Tab value="Python"> ```python import requests res = requests.post( '{API_BASE}/task', headers={'X-API-Key': 'your_api_key'}, json={'action_id': 'wenxin_prompt', 'params': {'prompt': 'What are the best running shoes in 2026?'}}, ) job_id = res.json()['job_id'] ``` </Tab> <Tab value="JavaScript"> ```javascript const res = await fetch('{API_BASE}/task', { method: 'POST', headers: { 'X-API-Key': 'your_api_key', 'Content-Type': 'application/json' }, body: JSON.stringify({ action_id: 'wenxin_prompt', params: { prompt: 'What are the best running shoes in 2026?' }, }), }); const { job_id } = await res.json(); ``` </Tab> </Tabs> --- ### Response <StatusBadge code={202} text="Accepted" /> ```json { "status": "processing", "job_id": "…", "poll_url": "/v1/wire/jobs/…" } ``` Poll [GET /v1/wire/jobs/\{id\}](/docs/api-reference/wire/get-job) until `status: "completed"`: <StatusBadge code={200} text="OK" /> ```json { "status": "completed", "credits_used": 2, "data": { "status": "ok", "data": { "prompt": "…", "title": "…", "answer_text": "…", "answer_text_markdown": "…", "answer_html": "…", "citations": [], "search_sources": [], "references": [], "related_media": [], "recommendations": [], "conversation_id": "…", "response_id": "…", "language": "zh", "detected_location": "…", "model": "…" }, "meta": { "action_id": "wenxin_prompt", "catalog_slug": "wenxin", "envelope_version": "v1" } } } ``` | Field | Type | Description | |-------|------|-------------| | `data.data.answer_text` | string | The answer (also `answer_text_markdown`, and `answer_html` when `include_html`) | | `data.data.citations` | array | Sources cited in the answer | | `data.data.search_sources` | array | Sources considered during search | | `data.data.language` | string | Detected answer language | > Handles any language, strongest on Chinese-language queries. `additional_prompt` runs a second conversational turn. Billed **only on success**. --- # End of anakin.io llms-full.txt