Google Custom Search API Shutdown: What to Use Instead

Introduction

Google confirmed it on its own developer page: the Custom Search JSON API is closed to new customers, and existing customers have until January 1, 2027 to move off it. That's the API a lot of RAG pipelines and research agents have quietly leaned on since web search for LLMs became a real category instead of a side project.

This piece covers what's actually changing, why Google's own suggested replacement isn't a parameter swap, and what the migration looks like on Anakin.io: URL Scraper as the foundation, with Search API and Agentic Search as the layers built on top of it.

What's actually shutting down, and when

• No new signups, as of now. Google's own page states the API is closed to new customers today, not on some future date.
• Existing integrations stop working on January 1, 2027. There's no extended grace window documented beyond that date.
• Google's suggested migration path is the Gemini Enterprise Agent Platform, which shows up under three different names depending on where you're looking: Agent Search in the docs, Vertex AI Search in the console, and Discovery Engine in the API itself. Same product, three labels, which makes it harder to tell at a glance whether a given doc page or support thread is even talking about the thing you're evaluating.

None of this is a rumor or a changelog line easy to miss. It's stated plainly on the page every Custom Search JSON API integration already points to.

Why Google's own replacement isn't a drop-in swap

Cost is the first wall. Per Brave's breakdown of the shutdown, the Configurable subscription tier starts around $6,000 a month for a minimum of 1,000 queries per month, plus $5.00 per GB-month for indexed storage and $1.50 per 1,000 queries for corpus search on the General model. The default tier ships with a low rate quota that Google's own documentation says not to request an increase on.

The bigger structural change is what the product actually searches. Agent Search indexes content you feed it. It does not crawl the open web the way Custom Search JSON API did. A pipeline that called the old API to answer "what does this arbitrary URL say" or "find me pages about X across the web" is asking for a capability the replacement was not built to provide, regardless of budget.

That's the real reason this isn't a key-and-endpoint swap. The request shape changes, the pricing model changes, and the thing being searched changes.

The three jobs a pipeline actually needs, in order

Strip away the vendor names and a web-connected agent or RAG pipeline leans on three distinct capabilities, and most teams already have the first one built or are about to need it regardless of what happens to search:

• Fetch and read a page: given a URL, get its content back as clean text or structured data. Every pipeline needs this, search-related or not.
• Rank and describe: given a query, return a ranked set of URLs with enough context (title, snippet, date) to decide which ones matter.
• Search and read in one call: given a research question, find the right pages, fetch them, and hand back an answer with sources, without the pipeline orchestrating the first two steps itself.

Custom Search JSON API only ever did the second job. On Anakin.io, that maps to three products that build on each other: URL Scraper for the fetch, Search API for the rank-and-describe layer on top of it, and Agentic Search for when the call itself should do the finding and reading.

Layer 1: URL Scraper, the foundation

Before replacing search, most pipelines already need URL Scraper (POST /v1/url-scraper): the ability to turn any URL into clean markdown, HTML, or structured JSON. It's the layer everything else in this post sits on top of, since a search result is only useful once something fetches what's behind the link.

curl -X POST https://api.anakin.io/v1/url-scraper \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/article",
    "formats": ["markdown"]
  }' 

That returns a job to poll:

{
  "jobId": "job_abc123xyz",
  "status": "pending"
}

curl https://api.anakin.io/v1/url-scraper/job_abc123xyz \
  -H "X-API-Key: your_api_key"

# {
#   "id": "job_abc123xyz",
#   "status": "completed",
#   "url": "https://example.com/article",
#   "markdown": "# Article title...",
#   "cached": false,
#   "durationMs": 3200
# }

A basic scrape is 1 credit. Add useBrowser: true for JS-heavy pages at no extra cost, or outputSchema to get AI-extracted structured fields back instead of raw markdown. This is the piece worth building first, independent of whatever replaces search, because every downstream layer depends on it.

Layer 2: Search API, finding what to fetch

Once page-fetching is solved, the next layer is knowing which pages to fetch. Search API (POST /v1/search) handles that: one synchronous call, no polling.

curl -X POST https://api.anakin.io/v1/search \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "custom search api shutdown alternatives",
    "limit": 10
  }' 

The response:

{
  "id": "63385e99-3ef5-4667-84a7-e7b398ec8e06",
  "results": [
    {
      "url": "https://example.com/article",
      "title": "Migrating off Custom Search JSON API",
      "snippet": "Relevant excerpt from the page...",
      "date": "2026-07-01",
      "last_updated": "2026-07-03"
    }
  ]
}

Worth being precise about: that response is url, title, snippet, date, last_updated per result, not full page text. This is exactly where Layer 1 comes back in: feed the url field straight into URL Scraper for any result worth reading in full, instead of treating the snippet as the answer. limit defaults to 5 and caps at 20. The call costs 3 credits and is rate-limited at 60 requests per minute.

Layer 3: Agentic Search, when the call should search and read on its own

Layer 3 is Layers 1 and 2 run automatically as one pipeline. Agentic Search (POST /v1/agentic-search) runs four stages in order: query refinement, web search, citation scraping, and analysis. It's async, since actually reading the pages it finds takes longer than a single request-response round trip:

curl -X POST https://api.anakin.io/v1/agentic-search \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "What changed in the Custom Search JSON API shutdown and what are the real alternatives?"
  }' 

That returns a job to poll:

{
  "job_id": "3f8aa45d-6ea3-4107-88ce-7f39ecf48a84",
  "status": "pending",
  "message": "Agentic search job queued successfully",
  "created_at": "2026-07-01T12:00:00.000Z"
}

Poll GET /v1/agentic-search/{job_id} every 10 seconds or so until status settles. The citation-scraping stage is doing what Layers 1 and 2 would do by hand: searching, then fetching full page content for the sources it cites, not just returning links.

It costs more for a reason: 10 credits base plus 1 credit for every URL it scrapes during the research. Reach for it when a pipeline wants a researched answer with sources attached in one call, and reach for URL Scraper plus Search API separately when a pipeline wants to control each step itself.

What the search-call migration actually looks like

Before, calling Custom Search JSON API:

curl "https://www.googleapis.com/customsearch/v1?key=YOUR_KEY&cx=YOUR_CX&q=custom+search+api+shutdown"

After, calling Anakin's Search API, with URL Scraper already in place to read any result in full:

curl -X POST https://api.anakin.io/v1/search \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "custom search api shutdown", "limit": 10}' 

Two real differences to account for in the migration, not just a URL swap: there's no separate search-engine-ID parameter to manage, since a prompt is the whole query, and the response fields are named differently (snippet instead of Google's htmlSnippet, for one), so any code parsing the old response shape needs updating, not just the endpoint it calls.

What doesn't carry over, stated plainly

• Index breadth. Neither Google nor Anakin publishes an independently verified index size, and claiming parity either way isn't something this piece will do. Test against the actual queries a pipeline runs before committing.
• Parameter compatibility. This is a rewrite of the call and the response parsing, not a key-and-endpoint change. Budget for that in the migration, however small the pipeline.
• A fixed per-query price. Anakin bills in credits across all three endpoints (1 for URL Scraper, 3 for Search API, 10 plus 1 per scraped URL for Agentic Search), which is a different mental model than a flat per-1,000-queries rate.

The free tier is 300 credits, no card required, which is enough to run the actual migration test against real queries before deciding anything. Sign up and run a side-by-side on the queries that matter to the pipeline, rather than estimating from this post alone.

Conclusion

The deadline is real and the default Google path is priced for a different kind of customer than most agent and RAG teams calling a search API today. On Anakin.io, the migration starts with URL Scraper as the foundation every pipeline needs regardless of what replaces search, then adds Search API for ranking and describing results, and Agentic Search for the cases where a single call should search, read, and answer.

Start with Anakin.io, build the fetch layer first, and add search on top of it once that's solid.

Frequently Asked Questions

When does the Custom Search JSON API actually stop working?

January 1, 2027, per Google's own developer documentation. The API is already closed to new customers as of now, so there's no way to start a fresh integration on it today even as a stopgap.

Is Vertex AI Search the same thing as Google's Custom Search JSON API?

No. Vertex AI Search, also called Agent Search in the docs and Discovery Engine in the API, searches content fed into it, not the open web the way Custom Search JSON API did. It's a different product with a different pricing model, not a renamed continuation.

Why start with URL Scraper instead of jumping straight to a search API?

Because fetching page content is the capability nearly every pipeline needs regardless of how it finds URLs. Building that layer first means Search API and Agentic Search both have something solid to hand results to, instead of a pipeline that can rank URLs but has no reliable way to read what's behind them.

What does Anakin's Search API actually return?

Structured results: url, title, snippet, date, last_updated per result, up to 20 per call, returned synchronously. It does not return full page text; pass the url straight into URL Scraper, or use Agentic Search, when full text is required.

How much does switching cost?

URL Scraper costs 1 credit per page. Search API costs 3 credits per call. Agentic Search costs 10 credits base plus 1 credit per URL it scrapes during research. New accounts start with 300 free credits and no card requirement, enough to test the migration against real queries first.

Does a pipeline need Agentic Search, or is Search API plus URL Scraper enough?

If a pipeline wants to control ranking and fetching itself, Search API plus URL Scraper is cheaper and gives more control over which results get fetched. Reach for Agentic Search when the call itself should produce a researched, sourced answer in one request, since its citation-scraping stage already combines both steps.

Sources

• Custom Search JSON API overview and deprecation notice - developers.google.com
• Google API Shutdown breakdown and Agent Search pricing - brave.com