Find Your Next Apartment with Search API: A London and San Francisco Guide

Day 27 of 30 Days of Search.

TL;DR

To build an apartment finder with the Valyu Python SDK, turn a renter's requirements into neighbourhood-specific searches with valyu.search(), retrieve rental pages with their source links, merge duplicate URLs, and review individual listings before shortlisting them. This tutorial builds a Python and Streamlit tool for London and San Francisco, including weekly-to-monthly rent conversion and a downloadable shortlist.

The finished tool lets you:

• Search selected rental websites from one brief.
• Read the returned evidence and open the original listing.
• Enter the advertised rent and confirm the details that matter.
• Download your research, notes and reviewed shortlist.

You'll need Python 3.11 or later, as required by the pinned Streamlit 1.66.1 release, and a Valyu API key. We use the official Valyu SDK for search; you don't need a separate language-model API key.

The apartment looks great. Then you read the listing.

You find a flat that seems to fit your budget. The photographs look good. The location looks promising.

Then you notice the price is per week, not per month.

Or you're looking in San Francisco, and the advertised price is an effective average after a move-in concession. The monthly base rent is higher. Another promising result is a room in a shared apartment, even though you need the whole place.

You now have tabs open across several websites and a notes app full of half-remembered addresses.

Let's build something that helps with that specific job: find rental pages worth investigating, keep the evidence together, and produce a shortlist you can use to arrange viewings.

We'll work through two searches. The budgets and neighbourhoods below are example inputs, not claims about market averages or guaranteed availability.

The app lets you change these inputs. It defaults the move-in date to 30 days from the day you run it.

How does an apartment discovery tool work?

An apartment discovery tool converts a rental brief into searches across relevant websites, then collects the returned pages for review. Valyu Search API supplies ranked results with source URLs and extracted text. Your application manages the renter's requirements, duplicate URLs, price calculations, review state and shortlist.

A search result starts the investigation. The reviewed listing determines whether you should contact the advertiser.

Our first version has a simple path:

Rental brief
  → One web search per neighbourhood
  → Source-linked candidate pages
  → Listing review and rent normalization
  → Downloadable shortlist

For London, the source configuration includes OpenRent, Rightmove and Zoopla. For San Francisco, it includes Zillow, Apartments.com and the San Francisco Bay Area Craigslist domain. These are example web-search filters, not direct inventory integrations or a promise that every website will return accessible results.

Keep the vocabulary local. A London query should include “flat to rent”. A San Francisco query should include “apartment for rent” and the city name. Country targeting adds context, but the query still needs the city and neighbourhood.

Step 1: Set up the Python apartment finder

Create a project and install the Valyu SDK and Streamlit:

mkdir apartment-finder
cd apartment-finder
python3 -m venv .venv
source .venv/bin/activate
python -m pip install valyu==2.13.2 streamlit==1.66.1

Add a .gitignore before storing credentials:

.env*
.venv/
__pycache__/
*.pem
*.key
credentials.*
secrets.*
shortlist-*.json

Set VALYU_API_KEY in the environment of the terminal that will run the app. In Bash or zsh on macOS or Linux, this reads the key without displaying it or putting its value in your shell history:

read -s VALYU_API_KEY
export VALYU_API_KEY

Paste the key at the prompt and press Enter. Keep it in the Python server process, not in browser JavaScript.

Step 2: Search rental websites with the Valyu SDK

The Valyu Python SDK reads VALYU_API_KEY automatically. Call valyu.search() with search_type="web" and the rental domains you want to investigate. The SDK handles the HTTP request and returns typed results.

Save this as first_search.py and run it with python first_search.py:

from valyu import Valyu

with Valyu() as valyu:
    response = valyu.search(
        "1 bedroom entire flat to rent in Stratford, London under GBP 2200 per month. "
        "Move in 2026-12-01. Individual long-term rental listings, not rooms.",
        search_type="web",
        country_code="GB",
        included_sources=["openrent.co.uk", "rightmove.co.uk", "zoopla.co.uk"],
        max_num_results=10,
    )
    if not response or not response.success:
        raise RuntimeError(response.error if response else "Search did not complete.")

    for listing in response.results:
        print(listing.title, listing.url)
        print(listing.content)

For San Francisco, use a query such as:

1 bedroom entire apartment for rent in Inner Sunset, San Francisco
under USD 3200 per month. Move in 2026-12-01.
Individual long-term rental listings, not rooms.

Change country_code to US and use the San Francisco domain list from the code below.

Putting a budget in a query is not the same as enforcing a numeric price filter. Search can return an area page, a building with multiple units, or a listing that doesn't meet every requirement. We apply the budget after reviewing a specific listing's price.

We also avoid a blanket start_date filter. It filters publication dates, which aren't the same as move-in dates or confirmed availability. A page's publication date can be missing, and an older listing may have been updated.

Step 3: Search rental websites and preserve the evidence

Save this as rental_search.py.

The two market profiles supply the local vocabulary, currency, example areas and source domains. discover() reuses one Valyu client for up to three neighbourhood searches, keeps source text and dates, and merges repeated URLs. It records failures separately so a failed search doesn't silently look like an empty neighbourhood.

import math
from datetime import datetime, timezone
from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit

from valyu import Valyu


MARKETS = {
    "London": {
        "country": "GB", "currency": "GBP", "budget": 2200,
        "areas": "Stratford, Lewisham, Finsbury Park",
        "domains": ["openrent.co.uk", "rightmove.co.uk", "zoopla.co.uk"],
        "home": "flat to rent", "destination": "King's Cross",
    },
    "San Francisco": {
        "country": "US", "currency": "USD", "budget": 3200,
        "areas": "Inner Sunset, Inner Richmond, Bernal Heights",
        "domains": ["zillow.com", "apartments.com", "sfbay.craigslist.org"],
        "home": "apartment for rent", "destination": "Embarcadero",
    },
}


def monthly_rent(amount, period):
    if not isinstance(amount, (int, float)) or isinstance(amount, bool):
        return None
    if not math.isfinite(amount) or amount <= 0 or period not in ("month", "week"):
        return None
    return amount * 52 / 12 if period == "week" else amount


def clean_url(url, domains):
    try:
        p = urlsplit(url)
        host = (p.hostname or "").lower()
        if p.scheme not in ("http", "https") or p.username or p.password:
            return None
        if not any(host == d or host.endswith("." + d) for d in domains):
            return None
        params = [(k, v) for k, v in parse_qsl(p.query, keep_blank_values=True)
                  if not k.lower().startswith("utm_") and k.lower() not in ("gclid", "fbclid")]
        return urlunsplit((p.scheme, p.netloc.lower(), p.path, urlencode(params), p.fragment))
    except (ValueError, TypeError):
        return None


def discover(brief):
    market = MARKETS[brief["city"]]
    rows, receipts = {}, []
    with Valyu() as valyu:
        for area in brief["areas"]:
            query = (f"{brief['bedrooms']} bedroom entire {market['home']} in {area}, "
                     f"{brief['city']} under {market['currency']} {brief['budget']} per month. "
                     f"Move in {brief['move_in']}. Individual long-term rental listings, not rooms.")
            observed_at = datetime.now(timezone.utc).isoformat()
            try:
                response = valyu.search(
                    query, search_type="web", max_num_results=5,
                    country_code=market["country"], included_sources=market["domains"],
                    response_length="short", include_image_links=False,
                )
            except Exception as exc:
                receipts.append({"query": query, "error": type(exc).__name__})
                continue
            if not response or not response.success:
                receipts.append({"query": query, "error": response.error if response else "Search did not complete.",
                                 "tx_id": response.tx_id if response else None})
                continue
            receipts.append({"query": query, "tx_id": response.tx_id, "warning": response.error,
                             "warnings": [w.model_dump(mode="json") for w in response.warnings or []],
                             "returned": len(response.results)})
            for hit in response.results:
                url = clean_url(hit.url, market["domains"])
                if not url:
                    continue
                if url in rows:
                    if area not in rows[url]["found_for"]:
                        rows[url]["found_for"].append(area)
                    continue
                rows[url] = {
                    "title": hit.title, "url": url, "original_url": hit.url,
                    "source": hit.source, "content": hit.content, "found_for": [area],
                    "publication_date": hit.publication_date or None,
                    "retrieved_at": observed_at, "rent": 0.0, "period": "unknown",
                    "whole_unit": False, "available": False, "fits_brief": False, "notes": "",
                }
    return {"brief": brief, "rows": list(rows.values()), "receipts": receipts}


def reviewed_shortlist(rows, budget):
    selected = []
    for row in rows:
        rent = monthly_rent(row["rent"], row["period"])
        if (rent is not None and rent <= budget and row["whole_unit"]
                and row["available"] and row["fits_brief"]):
            selected.append({**row, "monthly_base_rent": round(rent, 2)})
    return sorted(selected, key=lambda row: row["monthly_base_rent"])

There are two useful details here.

First, source can simply be "web". Keep the original URL as well, because that's how the renter gets back to the advertiser. The code also keeps tx_id values from successful responses for debugging.

Second, URL deduplication is deliberately modest. Removing utm_ tracking parameters can merge repeated links to the same page. It doesn't establish that two different URLs describe the same apartment. The code preserves other query parameters and URL fragments, such as #unit-2, because they may identify different units.

New candidates start with an unknown rent period and unchecked confirmations. We don't extract the first price from a multi-property page and attach it to an arbitrary apartment.

Step 4: Add the search and shortlist interface

Save this as app.py in the same folder.

The interface collects a rental brief, shows the returned source text, and provides a review table. Once you've checked a specific listing, enter its base rent and payment period, then confirm that it is the whole unit, available for your dates, and fits your other requirements.

import json
from datetime import date, timedelta

import streamlit as st

from rental_search import MARKETS, discover, reviewed_shortlist

st.set_page_config(page_title="Apartment discovery", layout="wide")
st.title("Find your next apartment")
st.caption("Search London or San Francisco. Review the source, then build your shortlist.")
city = st.selectbox("Rental market", list(MARKETS))
market = MARKETS[city]

with st.form(f"search-{city}"):
    areas = st.text_input("Neighbourhoods (up to three, separated by commas)", market["areas"])
    left, middle, right = st.columns(3)
    budget = left.number_input(f"Monthly base-rent limit ({market['currency']})",
                               min_value=1, value=market["budget"], step=100)
    bedrooms = middle.number_input("Bedrooms", min_value=1, max_value=5, value=1)
    move_in = right.date_input("Move-in date", date.today() + timedelta(days=30))
    needs = st.text_input("Other needs to check", f"Commute to {market['destination']}; pet policy")
    submitted = st.form_submit_button("Find candidate pages")

if submitted:
    selected_areas = list(dict.fromkeys(a.strip() for a in areas.split(",") if a.strip()))
    if not 1 <= len(selected_areas) <= 3:
        st.error("Enter one to three neighbourhoods.")
        st.stop()
    with st.spinner("Searching rental websites..."):
        st.session_state.discovery = discover({
            "city": city, "areas": selected_areas, "budget": budget,
            "bedrooms": bedrooms, "move_in": str(move_in), "needs": needs,
        })
    st.session_state.pop("rental-review", None)

result = st.session_state.get("discovery")
if not result:
    st.info("Start with a rental brief. Only the search button makes API calls.")
    st.stop()
brief = result["brief"]
if brief["city"] != city:
    st.info("Search this market to replace the previous market's results.")
    st.stop()

st.subheader("Candidate pages")
st.write(f"Saved brief: {brief['bedrooms']} bedroom(s), {brief['city']}, "
         f"{market['currency']} {brief['budget']:,}/month, move in {brief['move_in']}.")
st.text(f"Other needs: {brief['needs']}")
st.caption("Search terms express preferences; they are not verified property filters. "
           "Returned pages can include building or area pages. Availability is unconfirmed.")
failed = sum("error" in receipt for receipt in result["receipts"])
if failed:
    st.warning(f"{failed} neighbourhood search(es) failed. These results have incomplete coverage.")
with st.expander("Search receipts and source warnings"):
    st.json(result["receipts"])
if not result["rows"]:
    st.info("No candidate pages to review. Try different areas or inspect the source filters. "
            "This does not establish that no suitable apartments exist.")
    st.stop()

for number, row in enumerate(result["rows"], 1):
    with st.expander(f"{number}. {row['title']}"):
        st.link_button("Open source page", row["url"])
        st.caption(f"Retrieved: {row['retrieved_at']} | Publication: "
                   f"{row['publication_date'] or 'unknown'} | Source: {row['source']}")
        st.code(row["content"] or "No extracted text. Open the source page.", language=None)

st.subheader("Review the listings")
st.write(f"Open a specific listing, then enter its advertised base rent in {market['currency']}. "
         "Leave unknown details unchecked. 'Fits brief' means you checked bedrooms, "
         "move-in date, lease, commute and your other needs. Add when and where you checked in Notes.")
fields = ["title", "url", "rent", "period", "whole_unit", "available", "fits_brief", "notes"]
edited = st.data_editor(
    [{key: row[key] for key in fields} for row in result["rows"]],
    hide_index=True, disabled=["title", "url"], key="rental-review",
    column_config={
        "url": st.column_config.LinkColumn("Source"),
        "rent": st.column_config.NumberColumn("Base rent (0 = unknown)", min_value=0.0),
        "period": st.column_config.SelectboxColumn("Rent period", options=["unknown", "month", "week"]),
        "whole_unit": st.column_config.CheckboxColumn("Whole unit confirmed"),
        "available": st.column_config.CheckboxColumn("Availability checked"),
        "fits_brief": st.column_config.CheckboxColumn("Fits brief"),
    },
)
merged = [{**original, **review} for original, review in zip(result["rows"], edited)]
shortlist = reviewed_shortlist(merged, brief["budget"])
st.subheader(f"Your reviewed shortlist: {len(shortlist)}")
st.caption("Sorted by normalized monthly base rent. Extra fees and bills are not included. "
           "Review edits live in this browser session; download them before closing it.")
if shortlist:
    st.dataframe([{key: row[key] for key in ["title", "monthly_base_rent", "url", "notes"]}
                  for row in shortlist], hide_index=True)
st.download_button("Download research and review (JSON)",
                   json.dumps({**result, "rows": merged, "shortlist": shortlist}, indent=2),
                   file_name=f"shortlist-{city.lower().replace(' ', '-')}.json", mime="application/json")

Start it:

python -m streamlit run app.py

Open the local URL printed by Streamlit. Choose a market, edit your brief, and click Find candidate pages.

Three neighbourhoods mean three valyu.search() calls, requesting up to five results each. The SDK handles transport and may retry rate-limited requests. Returned results can overlap, and usage charges apply. Check current Search API pricing rather than assuming a fixed cost per app session.

The app stores results in the current Streamlit session. Editing the review table or downloading the JSON doesn't make another search request. Clicking the search button starts a new search and replaces the current review, so download your work first if you want to keep it.

London walkthrough: is £500 per week within a £2,200 budget?

Choose London, enter your preferred neighbourhoods, and set a £2,200 monthly base-rent ceiling. Keep King's Cross and your cat's requirements in Other needs to check. That field stays visible during review; this version doesn't turn it into an automated commute or pet-policy filter.

Open a candidate's source page. Check that you're looking at a particular flat rather than a search-results page or a room advertisement.

Suppose the individual listing advertises £500 per week. This is an illustrative calculation, not a live rental offer.

Monthly equivalent = weekly rent × 52 ÷ 12
                   = £500 × 52 ÷ 12
                   = £2,166.67

That is within the example base-rent limit. £500 × 4 = £2,000 would understate the monthly equivalent.

In the review table, enter 500 under base rent and choose week. Before checking Fits brief, inspect the bedroom count, move-in date and lease, confirm the cat policy, and check the journey from the actual address using TfL's journey planner.

“Near a station” isn't a door-to-door journey time. Include walking, transfers and the time you actually need to travel.

Next action: if the flat fits, use the original listing to enquire about a viewing. If the cat policy is unclear, put the question in Notes and leave the fit confirmation unchecked until you have an answer.

Outcome: the shortlist contains a source-linked candidate with a comparable rent and your review notes, rather than an attractive photograph you can't find again.

San Francisco walkthrough: is the advertised rent the base rent?

Switch to San Francisco and start a new search with a $3,200 monthly base-rent limit. Investigate the example areas or substitute your own. Keep your Embarcadero commute and pet-policy requirements visible in the brief.

For a candidate building, open the actual unit and lease terms. A “starting at” price on a building page may refer to a different floor plan or unit.

Consider this hypothetical offer:

Monthly base rent:             $3,200
Lease term:                   12 months
Concession:                   One month free
Effective monthly average:    $3,200 × 11 ÷ 12 = $2,933.33

The effective average is lower, but the base rent remains $3,200. The payment schedule depends on the offer's terms. Enter the base rent in this tool, keep the concession in Notes, and separately check additional charges and any conditions attached to the offer.

Compare like with like: monthly base rent in the same currency. Keep concessions, bills and other charges separate.

Check the trip from the actual address with SFMTA's trip planner. Confirm that the advertised unit is available for your move-in date before checking Availability checked.

Next action: contact the advertiser with the unit identifier, target move-in date and any outstanding questions about the lease or pet charges.

Outcome: you can compare the unit against other candidates without quietly comparing one property's base rent to another property's promotional average.

What belongs in the apartment shortlist?

A useful apartment shortlist includes the source link, comparable rent, availability check and unresolved questions. In this app, a row enters the reviewed shortlist only when its normalized base rent is within the saved budget and all three review confirmations are checked.

Download research and review (JSON) saves all candidate pages, your edits, search receipts and the reviewed shortlist. Notes and confirmations are your review record; they aren't certifications from a rental platform.

What should you add next?

Choose the next feature based on the part of apartment hunting that still wastes your time:

• Too many broad pages? Add structured extraction for individual listing pages. Preserve the supporting text for each field and leave missing details unknown. Valyu Contents can help read known URLs where extraction is available.
• Seeing the same property on different websites? Add property-level matching using the full address and unit identifier. Keep uncertain matches separate.
• Commute is the deciding factor? Integrate a routing service and evaluate the actual address at a specified travel time. This example only records the requirement for manual review.
• Want to check again tomorrow? Persist the results and schedule another search. A newly discovered URL is a new search result, not necessarily a newly listed or newly available home.

Search coverage can be incomplete, and some pages won't expose usable listing text. If reliable, comprehensive inventory is a product requirement, evaluate authorized listing feeds or partnerships alongside web discovery.

Verification note: You can decide to do this directly from the Valyu dashboard playground to check for the results directly from the search interface

Frequently asked questions

Can I build an apartment finder using a search API?

Yes. Use a search API to discover public rental pages from a brief, keep their URLs and source text, then review individual listings. Numeric budget enforcement, price normalization and shortlist rules belong in your application. Search retrieval alone doesn't confirm availability or every rental requirement.

Can the same rental search tool work in London and San Francisco?

Yes. Separate the retrieval and review logic from market-specific settings. This example changes the country code, currency, rental vocabulary, source domains and neighbourhoods while reusing the same application. It compares candidates within each market; it doesn't convert GBP to USD.

Does Search API return structured apartment inventory?

This example uses web search results, not a dedicated property-inventory dataset. Results include source URLs and extracted content, and can describe individual listings, buildings or area pages. The app keeps those as candidate pages until you review a specific rental.

How do I convert weekly rent to monthly rent?

For a monthly comparison, multiply the weekly rent by 52 and divide by 12. For example, £500 per week is approximately £2,166.67 per month. Keep the original advertised payment period and check the listing's contractual rent; the conversion is a comparison calculation.

Does a recent search result mean an apartment is still available?

No. Retrieval time, publication date and availability are different facts. Open the original listing and confirm the unit and move-in date with the advertiser. The app leaves availability unchecked until you review it.

Build something you would use during your own apartment search

Start with one city, a few neighbourhoods and the requirements you actually care about.

Use Search API to find candidate pages. Keep the source links. Put weekly and monthly rents on the same basis. Record the questions that still need answers.

The useful end product is a shortlist that helps you decide which advertiser to contact next.

Get a key at platform.valyu.ai, then follow the Search API quickstart to adapt the queries to your rental market.

What's the detail you always end up checking manually when looking for a place: the commute, pet policy, bills, or something else?