How I Scrape Google Results with a Cheap SERP API (Python)

I used to keep a headless browser around just to read Google's search results page. It worked for a while, then one morning every request came back with a CAPTCHA page and I spent the next two days rebuilding my "scraper" with new fingerprints, proxies, and retries. The data I actually needed — titles, URLs, snippets, positions — is maybe 10% of what the page renders, and I was paying for the other 90% in breakage.

These days I just call an API that returns the results as JSON. In this post I'll show the shortest working Python example I have, walk through the response fields, and note what it costs.

The request

SerpBase is a Google SERP API: you POST a query to https://api.serpbase.dev/google/search with an X-API-Key header and get structured JSON back. All endpoints are POST + JSON, which keeps the code boring — that's a feature.

import os
import requests

API_KEY = os.environ["SERPBASE_API_KEY"]
BASE = "https://api.serpbase.dev/google/search"

resp = requests.post(
    BASE,
    headers={"X-API-Key": API_KEY, "Content-Type": "application/json"},
    json={
        "q": "cheap serp api",
        "hl": "en",
        "gl": "us",
        "device": "desktop",
    },
    timeout=30,
)
resp.raise_for_status()
data = resp.json()

for i, item in enumerate(data["organic_results"][:10], start=1):
    print(i, item["title"], item["link"])

That's the whole thing. No DOM parsing, no selector maintenance, no proxy pool.

What comes back

Each organic result carries the fields you'd scrape off the page anyway: title, link, snippet, plus things like position, favicon, and related metadata. Here's a trimmed example of one result (structure per the official docs):

{
  "organic_results": [
    {
      "position": 1,
      "title": "SerpBase - Cheap SERP API",
      "link": "https://serpbase.dev",
      "snippet": "Low-cost real-time Google Search, Maps, and SERP data API.",
      "favicon": "https://serpbase.dev/favicon.ico"
    }
  ]
}

You also get the query's search_metadata (processed time, engine, language) when you need it for logging.

Language, region, and what it costs

Two params do most of the localization work: hl (language) and gl (country). The service covers 200+ countries/regions, so switching a market is literally changing gl from "us" to "jp" — no new scraper, no new proxies. It also has session persistence, automatic rotation, and CAPTCHA recovery on the backend, which is the part my old headless-browser setup could never match.

On pricing (checked on the site today): search, news, and video requests use 1 credit; images and Maps endpoints use 2. Credits come as prepaid packs from $10 for 20,000 searches ($0.50/1k) down to $0.30/1k at the top tier, or a $3/month Starter Boost with 10,000 searches. New accounts get 100 free searches with no card — enough to run this script and see real results before paying anything. Since it's pay-as-you-go with no monthly contract, my cost is now proportional to what I actually query instead of a flat fee I half-use.

Next step

Copy the script, set SERPBASE_API_KEY, and run it against a query you actually care about. If you want the full endpoint list and field reference, the SerpBase /google/search endpoint docs have it — that's where I check parameter names before I write code. The best way to validate a data source is one real request, not ten comparisons.