Your Service Search Should Never Recommend a Logo Idea

Your best keyword match can still be something nobody can hire.

Picture your service recommender answering “find someone to design a minimalist logo” with a logo inspiration page. The words fit. The destination doesn't. Somewhere between ingestion and retrieval, recordType: "logoMakerIdea" became “probably a service.”

Lab note 024: can we use directory vocabulary to improve recall without letting it become recommendation evidence? The catch comes first: sitemap gig records are URL-level references. They aren't enough to promise a price, delivery date, or availability.

Why does a logo idea match a hiring query?

An embedding can capture topical similarity without establishing what a page represents. A style page and a designer's service can both be excellent semantic neighbors of “minimalist logo.”

The failure happens when your application treats proximity as permission to recommend. A better embedding might preserve the same mistake with greater confidence. Lovely.

Directory vocabulary can expand a service query, but it cannot establish that a service is offered.

Our directory Actor makes this distinction concrete. Its sitemapDirectory mode can list gigs, sellers, categories, search tags, or Logo Maker ideas. Sharing a collection mechanism doesn't make those destinations interchangeable.

This collection cell requests the sources needed for the split and attaches the source dataset to each row:

import json
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
raw_rows = []

for sitemap_type in ("categories", "tags", "logomakerIdeas", "gigs"):
    run_input = {
        "mode": "sitemapDirectory",
        "sitemapType": sitemap_type,
        "maxShards": 1,
        "maxItems": 100,
    }
    run = client.actor(
        "vexaloom-labs/fiverr-gig-and-seller-sitemap-directory-scraper"
    ).call(run_input=run_input)
    if run["status"] != "SUCCEEDED":
        raise RuntimeError(f"Collection ended with {run['status']}")
    dataset_id = run["defaultDatasetId"]
    for item in client.dataset(dataset_id).iterate_items():
        raw_rows.append((dataset_id, item))

These inputs bound collection; they don't establish directory coverage.

Leaving searchKeyword unset is intentional: its directory filter matches URL or slug substrings. Applying a hiring phrase there would select the vocabulary before your retrieval experiment starts, making recall depend partly on a substring filter.

Give vocabulary a different job

The corpus boundary should express what each record is allowed to do in your application.

A vocabulary hit can suggest a term. It can't supply the final recommendation URL.

The split looks like this:

Hiring query
    -> vocabulary retrieval -> proposed search terms
    -> gig search or enriched gig retrieval
    -> relevant gig evidence -> recommendation

Gig sitemap references -> detail retrieval
Idea destinations      -> expansion context only

Keep the original hiring query attached to the search request. An expansion term should add vocabulary without replacing the user's intent to hire.

That also changes labeling. A category can be useful expansion material while being ineligible as a recommendation. Calling both judgments “relevant” hides the distinction your evaluator needs to enforce.

Keep category ancestry inside the unit

For embedding, a category's name travels with its available ancestry. A tag gets its name. An idea gets its name and ideaType, so a style concept keeps its role.

Splitting a category into separate name and parent fragments would discard context at retrieval time. Short vocabulary records usually need less chunking, not more.

The normalizer below creates an application envelope. Its unit_id, role, destination, and lineage keys belong to this corpus design.

def required_text(row, field):
    value = row.get(field)
    if not isinstance(value, str) or not value.strip():
        raise ValueError(f"Missing or invalid {field}")
    return value.strip()

def normalize(row):
    kind = required_text(row, "recordType")
    lineage = {}

    if kind == "category":
        level = required_text(row, "categoryLevel")
        paths = {
            "category": ("categorySlug",),
            "subCategory": ("categorySlug", "subCategorySlug"),
            "nestedSubCategory": (
                "categorySlug", "subCategorySlug", "nestedSubCategorySlug"
            ),
        }
        if level not in paths:
            raise ValueError("Unknown categoryLevel")
        lineage = {key: required_text(row, key) for key in paths[level]}
        identity = (kind, level, *lineage.values())
        destination = required_text(row, "categoryUrl")
        text = f"category: {required_text(row, 'name')}; " + " / ".join(
            lineage.values()
        )
    elif kind == "tag":
        identity = (kind, required_text(row, "tagSlug"))
        destination = required_text(row, "tagSearchUrl")
        text = f"search tag: {required_text(row, 'tagName')}"
    elif kind == "logoMakerIdea":
        idea_type = required_text(row, "ideaType")
        if idea_type not in {
            "tag", "industry", "monogram", "brandValue", "style", "symbolType"
        }:
            raise ValueError("Unknown ideaType")
        identity = (kind, idea_type, required_text(row, "ideaSlug"))
        destination = required_text(row, "ideaUrl")
        text = f"logo idea ({idea_type}): {required_text(row, 'ideaName')}"
    elif kind == "gigUrl":
        destination = required_text(row, "gigUrl")
        identity = (kind, str(row.get("gigId") or destination))
        text = ""
    else:
        raise ValueError("Unsupported recordType")

    return {
        "unit_id": json.dumps(identity, ensure_ascii=False),
        "role": "service_reference" if kind == "gigUrl" else "query_expansion",
        "record_type": kind,
        "text": " ".join(text.split()),
        "destination": destination,
        "lineage": lineage,
    }

Typed identities stop identical slugs from collapsing a search tag and an idea into the same concept. Category identities also include their ancestor path.

The category policy is deliberately strict: incomplete ancestry goes to review. We'd rather leave a unit out than silently move an ambiguous label to the root of the tree.

Ancestry describes membership. Matching words across a category, tag, and idea don't establish equivalence, and the normalizer creates no such relationship.

Gig references have empty embedding text because a URL slug is a locator, not a service description. They belong in a lookup for enrichment.

At the embedding boundary, measure the complete vocabulary text with your model's tokenizer, including ancestry and type prefixes. Reject oversized units for review rather than trimming the context that disambiguates them.

Deduplicate identities without erasing disagreements

Deduplicating by lowercased text would collapse distinctions the corpus needs to preserve.

Instead, group by typed identity and compare normalized content. Matching records share a unit while retaining their source observations; conflicting labels or destinations send the identity to review.

This cell applies that policy and writes the separate JSONL files:

groups = {}
review_rows = []

for ingestion_index, (dataset_id, row) in enumerate(raw_rows):
    source = {"dataset_id": dataset_id, "ingestion_index": ingestion_index}
    source.update({
        key: row[key]
        for key in ("sourceSitemapShard", "scrapedAt") if key in row
    })
    try:
        unit = normalize(row)
    except ValueError as error:
        review_rows.append({"source": source, "reason": str(error)})
        continue

    signature = json.dumps(unit, sort_keys=True, ensure_ascii=False)
    variants = groups.setdefault(unit["unit_id"], {})
    entry = variants.setdefault(signature, {"unit": unit, "sources": []})
    entry["sources"].append(source)

corpora = {"query_expansion": [], "service_reference": []}

for unit_id, variants in groups.items():
    if len(variants) != 1:
        review_rows.append({
            "unit_id": unit_id,
            "reason": "Conflicting normalized records",
            "variants": list(variants.values()),
        })
        continue

    entry = next(iter(variants.values()))
    document = {**entry["unit"], "sources": entry["sources"]}
    corpora[document["role"]].append(document)

for filename, documents in (
    ("query_expansion.jsonl", corpora["query_expansion"]),
    ("service_candidates.jsonl", corpora["service_reference"]),
    ("review.jsonl", review_rows),
):
    with open(filename, "w", encoding="utf-8") as stream:
        for document in documents:
            stream.write(json.dumps(document, ensure_ascii=False) + "\n")

Whitespace normalization removes formatting noise without deleting punctuation or changing label spelling. Aggressive cleaning would make distinct concepts harder to audit.

This implementation keeps collection and grouping in memory for inspection. At larger scale, persist the same identity and variant grouping in a database before exporting.

It also leaves an alias problem visible: a gig without gigId uses its exact URL as the fallback identity. Reconcile that URL with identified records before evaluation splits. Otherwise, a missing identifier can disguise a duplicate.

What still breaks after deduplication

A random row split can put the same concept into training and evaluation through destination aliases, related labels, or later snapshots.

Assign splits by resolved identity before generating paraphrases or chunks. If your experiment measures generalization to unfamiliar category branches, hold out branches together. If it measures generalization to unfamiliar sellers, group enriched gig evidence by seller identity.

Those are different experiments. A tidy JSONL file doesn't choose between them.

Query expansion can also leak evaluation information. If you learn expansion mappings from held-out relevance judgments, the vocabulary stage has already seen the answer. Freeze those mappings using only the data your evaluation permits.

Privacy needs its own boundary. The normalizer excludes seller identity fields, photos, image captions, and reviews from embedding text, but gig URLs can expose seller handles.

Keep destinations in a controlled lookup when preparing shareable training text. Source references also need a retention policy: a dataset identifier can reconnect a stripped corpus to richer records.

Dropping a username column while retaining a profile-bearing URL is privacy theater with better column names.

Make the recommendation boundary executable

Keep the vocabulary. Restrict what it can authorize.

The worked source is Fiverr Gig & Seller Sitemap Directory Scraper. Its directory and detail paths serve different jobs here: discovery supplies references; gig pages supply richer evidence on a best-effort basis.

The final stage should consume service records from trusted ingestion. A type guard establishes eligibility; relevance and current constraints still need checking.

Example fixtures below encode the eligibility boundary for an idea, a bare gig reference, and a service listing:

def service_evidence_url(record):
    kind = record.get("recordType")
    if kind not in {"gigListing", "gigDetail"}:
        return None
    title = record.get("title")
    url = record.get("gigUrl" if kind == "gigListing" else "sourceUrl")
    if not isinstance(title, str) or not title.strip():
        return None
    return url if isinstance(url, str) and url.strip() else None

example_idea = {
    "recordType": "logoMakerIdea",
    "ideaType": "style",
    "ideaName": "Example minimalist",
    "ideaUrl": "https://example.com/logo-idea",
}
example_reference = {
    "recordType": "gigUrl",
    "gigUrl": "https://example.com/service",
}
example_listing = {
    "recordType": "gigListing",
    "title": "Example logo design service",
    "gigUrl": "https://example.com/service",
}

assert service_evidence_url(example_idea) is None
assert service_evidence_url(example_reference) is None
assert service_evidence_url(example_listing) is not None

This guard doesn't validate arbitrary model-generated dictionaries. Its input must come from your ingestion path; otherwise, a model can simply invent an eligible recordType.

A passing record still doesn't establish budget fit, delivery suitability, or availability. It establishes that the evidence describes a service rather than a vocabulary page.

That smaller promise is useful. It's also enforceable.

Reproduce the corpus boundary

• Set APIFY_TOKEN in your environment and install apify-client.
• Execute the Python cells in order. The collection loop changes sitemapType for each source.
• Inspect review.jsonl and resolve identity aliases before embedding or assigning evaluation splits.
• Embed vocabulary text, keeping its destination and lineage as metadata.
• Resolve gig references through gigDetails, or retrieve service records through search, before applying the recommendation guard.

The exact collection input below is reused by the loop with only sitemapType changed:

{
  "mode": "sitemapDirectory",
  "sitemapType": "gigs",
  "maxShards": 1,
  "maxItems": 100
}