POST Scrape URL (Inline)

Scrape a single URL and get the result inline, without polling

POSThttps://api.anakin.io/v1/url-scraper/scrape

Submit a single URL and block until the scrape finishes, returning the full result in the same response — no polling required. Convenient for quick, interactive use where you'd rather wait than poll.

For batch jobs or long-running scrapes, prefer the asynchronous POST /v1/url-scraper + poll flow instead.

Note: the request holds the connection open until the scrape completes (up to ~90s). If the job doesn't finish in time, you get a 202 whose body has the same shape as a poll response (an id + a non-terminal status) to fall back to polling — see Timeout below. Set generous client/proxy read timeouts.

Because each request holds a connection open, this endpoint is rate limited more tightly than the async endpoint (20 requests/min per user). For high throughput, use the async submit + poll flow instead. Latency tracks the normal scrape pipeline, so it can rise when the platform is under heavy load.

Request Body

Identical to the asynchronous endpoint.

{
  "url": "https://example.com",
  "country": "us",
  "useBrowser": false,
  "generateJson": false
}

Parameter	Type	Description
`url` required	string	The URL to scrape. Must be valid HTTP/HTTPS.
`country`	string	Country code for proxy routing. Default `"us"`. See Supported Countries (207 locations).
`useBrowser`	boolean	Use headless Chrome with Playwright. Default `false`. Best for JS-heavy sites.
`generateJson`	boolean	AI-extract structured JSON from the content. Default `false`.
`sessionId`	string	Browser session ID for scraping authenticated pages. See Browser Sessions.

Response

200 OK

Returns the completed job with its content inline (same shape as GET /v1/url-scraper/{id}):

{
  "id": "job_abc123xyz",
  "status": "completed",
  "url": "https://example.com",
  "jobType": "url_scraper",
  "country": "us",
  "html": "<html>...</html>",
  "cleanedHtml": "<div>...</div>",
  "markdown": "# Page content...",
  "generatedJson": { "data": {} },
  "cached": false,
  "error": null,
  "createdAt": "2024-01-01T12:00:00Z",
  "completedAt": "2024-01-01T12:00:05Z",
  "durationMs": 5000
}

A scrape that fails also returns 200 with status: "failed" and a populated error field — check status, not just the HTTP code.

Timeout

202 Accepted

If the scrape doesn't finish within the wait budget, the response has the same shape as a poll response — an id and a non-terminal status — so you can poll GET /v1/url-scraper/{id} for the result. (The content fields are simply absent until the job completes.)

{
  "id": "job_abc123xyz",
  "status": "processing",
  "url": "https://example.com",
  "jobType": "url_scraper",
  "country": "us",
  "createdAt": "2024-01-01T12:00:00Z"
}

The id field is identical across the 200 result and this 202 — your client reads id either way; only status tells you whether to use the inline content or keep polling.

Code Examples

curl -X POST https://api.anakin.io/v1/url-scraper/scrape \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  --max-time 120 \
  -d '{
    "url": "https://example.com",
    "country": "us",
    "useBrowser": false,
    "generateJson": false
  }'

import requests

response = requests.post(
    'https://api.anakin.io/v1/url-scraper/scrape',
    headers={'X-API-Key': 'your_api_key'},
    json={
        'url': 'https://example.com',
        'country': 'us',
        'useBrowser': False,
        'generateJson': True,
    },
    timeout=120,
)

data = response.json()
if data.get('status') == 'completed':
    print(data['markdown'])
else:
    # 202 timeout fallback — poll GET /v1/url-scraper/{id}
    print(f"Still running: {data['id']}")

const response = await fetch('https://api.anakin.io/v1/url-scraper/scrape', {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    url: 'https://example.com',
    country: 'us',
    useBrowser: false,
    generateJson: true
  })
});

const data = await response.json();
if (data.status === 'completed') {
  console.log(data.markdown);
} else {
  // 202 timeout fallback — poll GET /v1/url-scraper/{id}
  console.log('Still running:', data.id);
}