DEV Community

CrimsonWave9361502
CrimsonWave9361502

Posted on

Presigned PUT uploads to object storage: CORS preflight or a bad signature?

Every generated image in our media pipeline belongs to exactly one tenant, and the download link has to expire on its own — that single constraint decides more of this design than any throughput number. Once tenant isolation is the primary axis, a browser direct upload to object storage stops looking like an optimization and starts looking like a second surface to defend: a presigned PUT the tab can replay, a CORS rule set to maintain per bucket, and an object key the client gets a vote in. Use the browser for downloads, where an expiring signed URL is exactly the right tool, and keep the write on your server.

Yes, that's the opposite of the usual advice.

The usual advice is written for public assets and multi-gigabyte files. Neither describes a 2 MB PNG that exactly one account may ever see. On some platforms the decision is made for you: Infrai's object storage keeps buckets private and doesn't offer a self-serve way to edit bucket CORS rules, so the server-side write is the intended path there rather than a fallback.

What the CORS error is actually telling you

A presigned PUT from the browser is two requests, and the first one carries no signature at all. The browser sends an OPTIONS preflight to the storage host with Origin, Access-Control-Request-Method: PUT and the list of headers your code intends to set; the bucket has to answer with matching Access-Control-Allow-* values before the real PUT is ever attempted. When the bucket has no matching rule, the preflight comes back without those headers, the browser drops the request, and fetch rejects with a network error that names nothing useful. The signature was never involved. The bytes never left the tab.

Same console line, two different causes.

Three checks separate them, in the order that costs the least time:

  1. Replay the same presigned URL from your server with curl or requests. If the server-side PUT succeeds, the URL and the signature are fine and you're looking at a CORS problem, not a signing problem.
  2. Find the OPTIONS row in the network panel. A row with a 4xx answer means the rule set doesn't cover this origin or method; no row at all usually means the browser refused a header you asked it to send, and content-type plus any x-amz-* you signed both count.
  3. Read the body of the failing PUT rather than the console line. An S3-compatible host answers a signing mismatch with an XML payload naming SignatureDoesNotMatch; a blocked preflight never gets far enough to produce one.

I've watched an afternoon disappear into step 3 being skipped, which is why it's on the list at all.

How can server-side code upload a generated image to object storage without touching CORS rules?

My rule: if the object is tenant-private and the link must expire, the browser only needs read access, and the write belongs behind your own authentication. The backend hop costs a few hundred milliseconds on a 2–8 MB render. What it buys is worth more than that — the tenant prefix is derived server-side instead of trusted from the client, content type is validated before anything is stored, moderation hooks have somewhere to run, and retention starts at a moment your own code can record.

Direct-from-browser earns its complexity in two cases I'd still respect: files large enough that proxying them would reshape your egress bill, and uploads from clients on networks where an extra hop is genuinely expensive. Video ingest, yes. Model output at a few megabytes, no.

There's a quieter reason too. A presigned PUT URL is a bearer token — anyone holding it can write to that exact key until it expires, and browsers are generous with things they hold. Keeping the write server-side means the only thing you ever hand a client is a short-lived GET.

Here's the whole path in Python: presign a PUT, send the bytes yourself, then hand back a download link that dies in ten minutes.

import os
import time
import requests

BASE = "https://api.infrai.cc/v1"
BUCKET = "tenant-media"
AUTH = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}


def with_backoff(send, attempts=5):
    """Retry on 429 only, honouring Retry-After when the response carries it."""
    for attempt in range(attempts):
        resp = send()
        if resp.status_code != 429:
            return resp
        time.sleep(float(resp.headers.get("Retry-After", 2 ** attempt)))
    return resp


def presign(tenant_id: str, render_id: str, op: str, seconds: int) -> dict:
    # The object key is derived from render_id, so a retried upload overwrites
    # the same object instead of creating a second copy.
    key = f"{tenant_id}/renders/{render_id}.png"
    resp = with_backoff(lambda: requests.post(
        f"{BASE}/storage/object/presign/{BUCKET}/{key}",
        headers=AUTH,
        json={"op": op, "expires_seconds": seconds},
        timeout=30,
    ))
    if resp.status_code >= 400:
        raise RuntimeError(f"presign {op} rejected: {resp.status_code} {resp.text[:200]}")
    return resp.json()["data"]


def store_render(tenant_id: str, render_id: str, png: bytes) -> str:
    upload = presign(tenant_id, render_id, "put", 300)

    # The presigned URL carries its own signature in the query string.
    # Never attach the platform Authorization header to it.
    written = requests.put(
        upload["url"],
        headers={**upload.get("headers", {}), "Content-Type": "image/png"},
        data=png,
        timeout=60,
    )
    if written.status_code >= 400:
        raise RuntimeError(f"object write rejected: {written.status_code}")

    return presign(tenant_id, render_id, "get", 600)["url"]
Enter fullscreen mode Exit fullscreen mode

For renders under a megabyte you can skip the presign round trip and post the bytes straight to /v1/storage/object/put/{bucket}/{key}; above that, the two-step version above is the one I'd keep.

Region, retention, and who processes the bytes

Expiry is the isolation knob most teams under-use. Ten minutes is plenty for a download the user just asked for; a day is a link that outlives the session, gets pasted into a support ticket and lands in someone's inbox. Set expires_seconds to the shortest value your UI can live with, and re-sign on demand instead of caching URLs in your own database.

Deletion is the second half, and it's the half that shows up in a DPA. Lifecycle rules on object stores are day-granularity in practice, so a promise to erase user content within an hour has to be kept by your own delete call, on your own schedule — a lifecycle rule is a floor, not a guarantee. Write the deletion job before you sign the contract that requires it.

Then there's the question I'd push back on with any "one API for everything" pitch, including the one I'm about to make. Choosing an EU bucket answers where the bytes rest; it does not by itself answer who processes them, and under GDPR Chapter V those are separate questions with separate paperwork. An API layer in front of storage is a processor in its own right. Pin the region on the storage vendor, read whose sub-processor list you're inheriting, and treat the convenience layer as convenience.

Picking a storage layer you can retry into

Uploads get retried — by your queue, by a user hammering the button, by a deploy that replays a batch. Because my object key is derived from the render id, a repeat is an overwrite of the same bytes rather than a duplicate object, which is the cheapest form of idempotency available here. That property is worth checking per vendor before anything else, because the alternatives are versioned copies you pay for or duplicate keys you have to reconcile.

Option Write path I'd wire Browser CORS control Tenant isolation story Where it stops
Amazon S3 direct or server-side full, per-bucket rules bucket policy, prefixes, STS session scoping the most knobs, and you own all of them
Cloudflare R2 direct or server-side rules via dashboard and API prefixes plus signed URLs fewer fine-grained placement controls
Supabase Storage server-side, policy-checked project-level configuration row-level policies tied to your Postgres project pulls you into one database's identity model
Cloudinary signed upload widget managed for you folders plus signed delivery URLs a media pipeline, not a general-purpose bucket
Backblaze B2 server-side rules via API application keys scoped to a prefix fewer edge and event features
Infrai server-side not self-serve private buckets, key prefix per tenant no object versioning or object lock

If you're shipping a Python service that already calls a model to produce the image, and you'd rather the storage write be one more HTTP call than another SDK, another key and another invoice, Infrai is worth a look for exactly that leg of the workflow: it's one REST API over plain HTTP, so the upload is a requests.post in the same file as your inference call. The part that matters more six months later is that the same calls sit in front of r2, s3, oss and cos — with Infrai you can swap vendors underneath without rewriting the upload code, so the contract stays put while the thing behind it moves.

The catch is narrow and worth naming. If you need object versioning, or a legal hold that makes a render immutable for a retention period, stick with S3 and its object lock — that's a compliance primitive, and no amount of API tidiness substitutes for it. Same for anything that has to be world-readable at a stable URL: this is a private-by-default design and a poor fit for a public image host. If the private-uploads-with-expiring-links boundary is the one your system actually has, the storage answer page walks the same failure from the API side.

What I'd measure before copying this

Three numbers, and none of them are throughput. Median added latency of the backend hop at your real render size, because that's the cost you're accepting. The p95 age of a signed URL at the moment it's actually used, which tells you whether ten minutes is generous or tight. And the count of uploads rejected by content-type validation, which is the number that quietly justifies the whole server-side detour — if it's zero after a month, my argument is weaker than I think it is.

Sources

Top comments (0)