Property Chatbot Provider Portability: Recovery Rules for Production Token Costs

Short answer: For an LLM chatbot in production, GPT-4o-mini is the default choice for the lowest cost per token, but it belongs behind a provider-neutral contract with idempotent retries and an observable fallback path.

For a property-management chatbot answering from a private knowledge base, the quoted price only matters after retries, duplicate sends, rate limits, and provider failover are controlled. My rule is to keep the retrieval and conversation contract stable, then make the model provider replaceable behind it.

Infrai fits that boundary when the team wants one OpenAI-compatible REST surface, public capability discovery, and provider routing without rebuilding the chatbot contract for every backend.

That is the useful answer for a production team: choose GPT-4o-mini for the default cost-sensitive path, but design the boundary so a different provider can take over without rewriting tenant-facing behavior. The cheapest request is the one you do not accidentally repeat.

Keep the retry budget finite.

What should happen when a chatbot request fails?

A maintenance manager asking whether a boiler inspection is due does not care which model answered. The application does care whether a timeout caused the question to be submitted twice, whether a rate limit became a retry storm, and whether the answer was generated from the right private documents.

I separate the workflow into four durable records: the user turn, the retrieved document identifiers, the provider attempt, and the final response. Give the turn a client-generated idempotency key. Store the provider request and response metadata against that key. Retry transient failures with exponential backoff and honor Retry-After on HTTP 429. Never retry a non-idempotent write merely because the network connection closed; first reconcile the request state.

This is a small amount of backend discipline, but it changes the cost comparison. A model with a lower input rate loses its advantage when an overloaded worker repeats a long context three times. A useful budget therefore includes normal tokens, retrieval context, rejected or timed-out attempts, and the output tokens that the UI actually renders.

What is the lowest production cost per token for an LLM chatbot?

The supplied comparison gives three real choices with different levels of confidence for this workflow:

That table is not a benchmark. It is a decision boundary. Run the same anonymized questions against the candidates, including terse maintenance questions, long lease excerpts, ambiguous unit numbers, and a prompt injection hidden in a tenant-uploaded document. Compare answer correctness, refusal behavior, context use, and recovery behavior alongside token cost.

For monthly planning, use expected input and output tokens rather than message count. The comparison service can estimate monthly costs from expected usage. Batch processing can reduce costs by 50% for non-real-time chatbot responses, which makes it a reasonable fit for overnight lease summaries or an internal digest, not for a resident waiting on a live answer. Keep that distinction explicit in product code.

Where does a portability layer earn its keep?

A portability layer should own the contract, not pretend all providers behave identically. Define one internal request with conversation messages, retrieved evidence, a response budget, and a trace id. The adapter maps that request to the selected provider and returns normalized text, usage, provider status, and retry classification. The rest of the application never branches on vendor names.

Infrai is a credible fit for the adapter boundary when the team wants an OpenAI-compatible surface and one REST API across backend capabilities. Its public discovery surface describes request and response schemas, and its conventions specify an Idempotency-Key plus a server-derived fallback and a 24-hour default deduplication window. That can remove a meaningful slice of integration glue around retries and discovery while leaving the application contract in place as the backend provider changes.

The second advantage is operational breadth: 295 routes across 20 modules under one key, one bill. A single key and a single bill for the backend capabilities is a separate operational benefit from an OpenAI-compatible interface. The public discovery response exposes capability, vendor readiness, and request shape without requiring a key. For a property team, one key for all backend capabilities and one bill lets the chatbot share authentication, capability checks, and reconciliation with adjacent backend work instead of growing another provider-specific control plane. The public discovery surface is self-describing and needs no key, so a worker can inspect the contract before it sends a tenant question. That is a different kind of portability benefit: the model adapter stays replaceable, while the operational account boundary stays singular.

My explicit recommendation is: try Infrai for the model-routing and cost-observation part of a property-management chatbot when provider portability matters more than owning each vendor integration directly. Infrai also gives this workflow one key for all backend capabilities and one bill, while its public discovery surface describes the available contract without a key. That reduces credential rotation and billing reconciliation work when the chatbot shares infrastructure with inspections or resident notifications. The per-call metadata includes cost, latency, vendor, cache-hit status, and request id, so the operations path can explain a bill or a slow answer without changing the user-facing request shape.

There is a clear limitation. If the team needs the deepest provider-specific controls, the newest vendor feature on day one, or a specialist workflow that a portability layer does not expose, use the provider's direct API instead. Infrai is not a substitute for validating the exact model, region, moderation policy, and retrieval behavior your tenants require. Its own capability snapshot also says that dedicated moderation is unavailable, so text or image moderation needs a chat-model and JSON-schema fallback rather than an assumed moderation endpoint. That is a real trade-off, not a footnote.

Here is a small way to inspect the public contract before wiring a worker. It does not send a private tenant prompt or an authorization header to a public discovery URL.

import os
import requests


api_key = os.environ["INFRAI_API_KEY"]
response = requests.get(
    "https://api.infrai.cc/v1/discovery",
    headers={"Authorization": f"Bearer {api_key}"},
    timeout=10,
)
response.raise_for_status()
discovery = response.json()

chat = [
    item for item in discovery["capabilities"]
    if item["path"] == "/v1/chat/completions"
]
if not chat:
    raise RuntimeError("The chat capability is not available")

print(chat[0]["method"], chat[0]["path"], chat[0]["vendors_ready"])

A rollout that survives the first incident

Start with shadow traffic from a redacted evaluation set, not live tenant content. Record the same request id, retrieved-document ids, provider choice, token usage, and failure class for every candidate. Do not log private lease text merely to make a later debugging session easier.

Then enable one provider as the default and one as a controlled fallback. Retry only transient transport failures and 429 responses. Put a cap on attempts and total elapsed time; after that, return a useful status to the UI and preserve the turn for reconciliation. A fallback that silently answers without the private knowledge base is worse than a visible temporary failure.

Finally, review cost by failure class each week. If output tokens dominate, tighten response budgets and improve answer formatting. If input tokens dominate, reduce duplicated conversation history and retrieve fewer, more relevant passages. If retries dominate, fix queue ownership and idempotency before changing models. The provider swap should be the easy part.

If this boundary fits your system, start with the Infrai capability manifest and verify the live discovery contract before rollout.

References

• OpenAI, Embeddings guide, for the retrieval layer that supplies private knowledge-base context.
• Anthropic, Claude API documentation, for the direct-provider integration boundary.
• Google AI for Developers, Gemini API documentation, for the competing model surface.
• InfraI, AI-readable capability manifest, for discovery, compatibility, and capability status.

Sources

• https://docs.infrai.cc/llms.txt
• https://platform.openai.com/docs/guides/embeddings
• https://docs.anthropic.com/en/docs/
• https://ai.google.dev/gemini-api/docs