Back to blog
Tutorial·October 6, 2026·10 min read

Google Custom Search API Shutdown: What to Use Instead

Google's Custom Search JSON API closes to existing customers on January 1, 2027, and it's already closed to new signups. See how URL Scraper, Search API, and Agentic Search replace it as three layers instead of one swapped endpoint.

M

Mahendra Sreekumar

Anakin Team

Anakin platform shown as three layers: URL Scraper as the foundation, Search API and Agentic Search built on top, replacing Google Custom Search API

If a RAG pipeline or research agent calls Google's Custom Search JSON API, it's on a clock. Google's own developer page says the API is closed to new customers, and existing customers have until January 1, 2027 to move off it. A lot of pipelines have quietly leaned on it since web search for LLMs became a real category instead of a side project.

The short answer to the Google Custom Search API shutdown: Google's self-serve replacement searches a fixed set of up to 50 sites, not the open web, so this isn't a key-and-endpoint swap. On Anakin.io the migration is three layers: URL Scraper to fetch pages, Search API to find them, and Agentic Search when one call should do both. Below: what's changing, why Google's path doesn't fit open-web search, and the calls to make instead.

What's actually shutting down, and when

  • No new signups, as of now. Google's own page states the API is closed to new customers today, not on some future date.
  • Existing integrations stop working on January 1, 2027. There's no extended grace window documented beyond that date. The same date ends the "Search the entire web" option for existing Programmable Search Engines, and every engine created since January 20, 2026 is limited to 50 or fewer sites.
  • Google's notice points to Vertex AI Search for searching up to 50 domains, and asks anyone who needs full web search to contact Google about a separate full web search solution, with no published pricing. Vertex AI Search has since been renamed, and it now shows up under three names depending on where you're looking: Agent Search in the docs, Vertex AI Search in the console and on the Custom Search notice, and Discovery Engine in the API itself. Same product, three labels, which makes it harder to tell at a glance whether a given doc page or support thread is even talking about the thing you're evaluating.

None of this is rumor. It's on the same Google page every Custom Search JSON API integration already points to.

Timeline from closed-to-new-signups now to the January 1, 2027 Custom Search JSON API shutdown

Why Google's own replacement isn't a drop-in swap

Cost is the first wall. Per Google's Agent Search pricing page, the Configurable subscription has a minimum commitment of 1,000 queries per minute (QPM) and 50 GB of storage. At $6 per QPM a month, that's about $6,000 a month before add-ons. The pay-as-you-go General model starts cheaper, at $1.50 per 1,000 Standard queries plus index storage (about $5 per GB a month after 10 GiB free). But its default rate quota is low, and Google's quota docs say not to request an increase: switch to Configurable pricing instead. Brave's breakdown of the shutdown walks through the same numbers.

Cost comparison: Google replacement at $6,000 per month minimum versus Anakin credits per call

The bigger structural change is what the product actually searches. Agent Search searches content you index, including website data from a fixed list of sites. It doesn't search the open web the way Custom Search JSON API could, and Google's open-web option is still an interest form, not a self-serve product. A pipeline that called the old API to answer "what does this arbitrary URL say" or "find me pages about X across the web" is asking for a capability Agent Search wasn't built to provide, regardless of budget.

That's the real reason this isn't a key-and-endpoint swap. The request shape changes, the pricing model changes, and the thing being searched changes.

The three jobs a pipeline actually needs, in order

Strip away the vendor names and a web-connected agent or RAG pipeline leans on three distinct capabilities, and most teams already have the first one built or are about to need it regardless of what happens to search:

  • Fetch and read a page: given a URL, get its content back as clean text or structured data. Every pipeline needs this, search-related or not.
  • Rank and describe: given a query, return a ranked set of URLs with enough context (title, snippet, date) to decide which ones matter.
  • Search and read in one call: given a research question, find the right pages, fetch them, and hand back an answer with sources, without the pipeline orchestrating the first two steps itself.

Custom Search JSON API only ever did the second job. On Anakin.io, that maps to three products that build on each other: URL Scraper for the fetch, Search API for the rank-and-describe layer on top of it, and Agentic Search for when the call itself should do the finding and reading.

Layer 1: URL Scraper, the foundation

Before replacing search, most pipelines already need URL Scraper (POST /v1/url-scraper): the ability to turn any URL into clean markdown, HTML, or structured JSON. It's the layer everything else in this post sits on top of, since a search result is only useful once something fetches what's behind the link.

curl -X POST https://api.anakin.io/v1/url-scraper \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/article",
    "formats": ["markdown"]
  }' 

That returns a job to poll:

{
  "jobId": "job_abc123xyz",
  "status": "pending"
}
curl https://api.anakin.io/v1/url-scraper/job_abc123xyz \
  -H "X-API-Key: your_api_key"

# {
#   "id": "job_abc123xyz",
#   "status": "completed",
#   "url": "https://example.com/article",
#   "markdown": "# Article title...",
#   "cached": false,
#   "durationMs": 3200
# }

A basic scrape is 1 credit. Add useBrowser: true for JS-heavy pages at no extra cost, or outputSchema to get AI-extracted structured fields back instead of raw markdown (+2 credits, per the pricing docs). This is the piece worth building first, independent of whatever replaces search, because every downstream layer depends on it.

Layer 2: Search API, finding what to fetch

Once page-fetching is solved, the next layer is knowing which pages to fetch. Search API (POST /v1/search) handles that: one synchronous call, no polling.

curl -X POST https://api.anakin.io/v1/search \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Google Custom Search JSON API closed to new customers January 1 2027 transition",
    "limit": 5
  }'

We ran that query through Anakin's MCP search tool on 25 Oct 2026. Five results came back. Here's the top one, with the snippet shortened:

{
  "results": [
    {
      "url": "https://developers.google.com/custom-search/v1/overview",
      "title": "Custom Search JSON API  |  Google for Developers",
      "snippet": "...Note: The Custom Search JSON API is closed to new customers.\nVertex AI Search is a favorable alternative for searching up to 50 domains. ...",
      "date": "",
      "last_updated": ""
    }
  ]
}

Worth being precise about: that response is url, title, snippet, date, last_updated per result, not full page text. The date fields come back empty when a page doesn't publish them, as they did here. This is exactly where Layer 1 comes back in: feed the url field straight into URL Scraper for any result worth reading in full, instead of treating the snippet as the answer. limit defaults to 5 and caps at 20. The call costs 3 credits and is rate-limited at 60 requests per minute.

Anakin Search API response viewer showing url, title, snippet, date, and last_updated fields, with full_text noted as not included

Layer 3: Agentic Search, when the call should search and read on its own

Layer 3 is Layers 1 and 2 run automatically as one pipeline. Agentic Search (POST /v1/agentic-search) runs four stages in order: query refinement, web search, citation scraping, and analysis. It's async, since actually reading the pages it finds takes longer than a single request-response round trip:

curl -X POST https://api.anakin.io/v1/agentic-search \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "What changed in the Custom Search JSON API shutdown and what are the real alternatives?"
  }' 

That returns a job to poll:

{
  "job_id": "3f8aa45d-6ea3-4107-88ce-7f39ecf48a84",
  "status": "pending",
  "message": "Agentic search job queued successfully",
  "created_at": "2026-07-01T12:00:00.000Z"
}

Poll GET /v1/agentic-search/{job_id} every 10 seconds or so until status settles. The citation-scraping stage is doing what Layers 1 and 2 would do by hand: searching, then fetching full page content for the sources it cites, not just returning links.

It costs more for a reason: 10 credits base plus 1 credit for every URL it scrapes during the research. Reach for it when a pipeline wants a researched answer with sources attached in one call, and reach for URL Scraper plus Search API separately when a pipeline wants to control each step itself.

What the search-call migration actually looks like

Before, calling Custom Search JSON API:

curl "https://www.googleapis.com/customsearch/v1?key=YOUR_KEY&cx=YOUR_CX&q=custom+search+api+shutdown"

After, the lookup itself is the Layer 2 search call shown above, so there's no need to repeat it here. What changes is the step after it: the pipeline reads the results it cares about instead of stopping at snippets. Here's the whole chain in one Python script, search first, then URL Scraper on each result:

import os, time, requests

API = "https://api.anakin.io/v1"
HEADERS = {"X-API-Key": os.environ["ANAKIN_API_KEY"]}

# Layer 2: find pages (synchronous, 3 credits)
search = requests.post(f"{API}/search", headers=HEADERS, json={
    "prompt": "Programmable Search Engine search the entire web option ending January 2027",
    "limit": 3,
}).json()

for result in search["results"]:
    # Layer 1: read each page in full (1 credit each; useBrowser adds no cost)
    job = requests.post(f"{API}/url-scraper", headers=HEADERS, json={
        "url": result["url"],
        "formats": ["markdown"],
        "useBrowser": True,
    }).json()

    while True:
        page = requests.get(f"{API}/url-scraper/{job['jobId']}", headers=HEADERS).json()
        if page["status"] in ("completed", "failed"):
            break
        time.sleep(3)

    if page["status"] == "completed":
        print(result["url"])
        print(page["markdown"][:190])

We ran the same two steps through Anakin's MCP tools on 25 Oct 2026: a search with limit 3, then a browser-rendered scrape of the top result, which was Google's own January 2026 Programmable Search Engine post. The first lines of the markdown came back as:

https://programmablesearchengine.googleblog.com/2026/01/updates-to-our-web-search-products.html
Evolving Programmable Search Engine

Programmable Search Engine helps hundreds of partners – from academic institutions to retail websites – serve their users’ search needs on their sites.

Two real differences to account for in the migration, not just a URL swap: there's no separate search-engine-ID parameter to manage, since a prompt is the whole query, and the response fields are named differently (snippet instead of Google's htmlSnippet, for one), so any code parsing the old response shape needs updating, not just the endpoint it calls.

If the old integration leaned on the results-page shape of Custom Search (a ranked list you page through, scoped to a country), Anakin's Wire catalog also has a Bing Search action. It returns organic results with title, destination URL and snippet, takes country, count (1 to 50) and offset parameters, and costs 3 credits a call. Those are Bing's rankings, not Google's, so test them against your own queries too.

Request comparison: Custom Search JSON API needs key, cx, and q parameters versus Anakin Search API needing only prompt

What doesn't carry over, stated plainly

  • Index breadth. Neither Google nor Anakin publishes an independently verified index size, so this post doesn't claim parity either way. Test against the actual queries a pipeline runs before committing.
  • Parameter compatibility. This is a rewrite of the call and the response parsing, not a key-and-endpoint change. Budget for that in the migration, however small the pipeline.
  • A fixed per-query price. Anakin bills in credits across all three endpoints (1 for URL Scraper, 3 for Search API, 10 plus 1 per scraped URL for Agentic Search), which is a different mental model than a flat per-1,000-queries rate.

The free tier is 300 credits, no card required, which is enough to run the actual migration test against real queries before deciding anything. Sign up and run a side-by-side on the queries that matter to the pipeline, rather than estimating from this post alone.

Where to start before January 1, 2027

The deadline is real, and Google's own path is built for searching a fixed set of sites, priced for a different kind of customer than most agent and RAG teams calling a search API today. On Anakin.io, the migration starts with URL Scraper as the foundation every pipeline needs regardless of what replaces search, then adds Search API for ranking and describing results, and Agentic Search for the cases where a single call should search, read, and answer.

Start with Anakin.io, build the fetch layer first, and add search on top of it once that's solid.

Frequently Asked Questions

When does the Custom Search JSON API actually stop working?

January 1, 2027, per Google's own developer documentation. The API is already closed to new customers as of now, so there's no way to start a fresh integration on it today even as a stopgap.

Is Vertex AI Search the same thing as Google's Custom Search JSON API?

No. Vertex AI Search, now called Agent Search in Google's docs and Discovery Engine in the API, searches content you index or a fixed list of sites (Google positions it for up to 50 domains), not the open web the way Custom Search JSON API could. It's a different product with a different pricing model, not a renamed continuation.

Why start with URL Scraper instead of jumping straight to a search API?

Because fetching page content is the capability nearly every pipeline needs regardless of how it finds URLs. Building that layer first means Search API and Agentic Search both have something solid to hand results to, instead of a pipeline that can rank URLs but has no reliable way to read what's behind them.

What does Anakin's Search API actually return?

Structured results: url, title, snippet, date, last_updated per result, up to 20 per call, returned synchronously. It doesn't return full page text. Pass the url straight into URL Scraper, or use Agentic Search, when full text is required.

How much does switching cost?

URL Scraper costs 1 credit per page. Search API costs 3 credits per call. Agentic Search costs 10 credits base plus 1 credit per URL it scrapes during research. New accounts start with 300 free credits and no card requirement, enough to test the migration against real queries first.

Does a pipeline need Agentic Search, or is Search API plus URL Scraper enough?

If a pipeline wants to control ranking and fetching itself, Search API plus URL Scraper is cheaper and gives more control over which results get fetched. Reach for Agentic Search when the call itself should produce a researched, sourced answer in one request, since its citation-scraping stage already combines both steps.

Sources