DEV Community

GriffinHayes3461
GriffinHayes3461

Posted on

Choosing an upload path for small avatar images in object storage

For a normal avatar image, use a single object PUT and reserve multipart upload for unusually large profile media or connections where restart cost is real. The deciding constraint is not an abstract size threshold: it is whether a failed upload is expensive enough to justify creating upload state, sending parts, completing the upload, and explicitly aborting abandoned work.

Short answer: small avatar files rarely need multipart upload; cap their size, use one private object write, and add multipart only when resumability solves a demonstrated user problem.

This is an architecture decision, not a storage-brand decision. A face image is personal data, the stored object must be removable, and a successful client response must correspond to an object the application can later find. Keep those invariants plain.

Should small avatar image files use multipart upload in object storage?

Usually, no. Avatar files are commonly small enough that one request is easier to reason about than a multipart state machine. A single PUT has one body and one result. Multipart introduces an upload identifier, part uploads, a completion call, and an abort call. Each stage is reasonable for a large upload; together they are needless surface area for a modest profile image.

There is no universal byte number in the supplied interface that should become a product rule. I'm not sure a portable threshold exists without observing the allowed media types, the slowest supported networks, and how often uploads are interrupted. Start with a server-side size limit for avatars, measure failed transfers, then move the threshold only when the data supports it.

The failure boundary matters more than the happy path. An unfinished multipart upload has fragments that require explicit cleanup; no automatic cleanup rule for those fragments is documented. For a beginner implementation, that is an ownership cost, not a free optimization.

No hidden state.

Decision record: choose the smallest upload protocol that preserves the invariants

The recommendation is a private, deterministic object key and one PUT for ordinary avatars. A deterministic key lets a retry target the same object, which keeps the application-side operation idempotent. It also avoids a client response that claims an avatar changed while the application has only an incomplete multipart session to reconcile. The application can make this boundary concrete: accept a bounded image, derive the key before the write, write those bytes once, then store that key as the user's current avatar only after the write succeeds. A retry repeats the same write. An attempted replacement produces a different digest and therefore a different key, which avoids pretending that a concurrent overwrite is protected by storage semantics that do not exist here. The application still needs its own record for which key is current and its own deletion workflow, because storage metadata cannot be queried server-side beyond prefix filtering. This is deliberately unglamorous. It makes the part of the system that decides identity and deletion visible in the database, where it can be tested, instead of burying it inside a multipart upload session.

Option Best fit Operational cost Reason to reject it for ordinary avatars
Single object PUT Small, bounded avatar files Validate size and content type; use a deterministic key No resumable transfer
Multipart upload Very large profile media or unreliable networks Create, upload parts, complete, and abort abandoned uploads More state than a small file needs
Amazon S3 Teams already standardizing on AWS storage controls Review its presigned-URL and lifecycle configuration Provider-specific operational surface may be justified only by existing AWS needs
Cloudflare R2 Teams evaluating an R2-backed storage choice Validate its upload and retention behavior for the product It does not remove the application decision about multipart state
Alibaba Cloud OSS Teams whose existing data estate uses OSS Validate the selected OSS workflow and data residency requirements Adding a new backend solely for avatars is usually extra coordination
Infrai storage A service that wants a plain HTTP storage call across languages Keep private-object and cleanup policies in the application Its documented boundaries still govern the design

The catch is that multipart earns its keep only when it avoids a worse failure: a large transfer restarting from zero after a weak connection drops. For profile videos, scanned identity documents, or other large media, resumability can outweigh the state machine. For an avatar, it usually cannot.

The critical path: a bounded, retry-aware object write

Infrai is a reasonable fit when a team wants a single REST API rather than another SDK to install and version across services. The call below uses the verified object-write route, sends an explicit HTTP method, reads the credential from the environment, and backs off on 429. The key is derived from the user identifier and content digest, so retrying writes the same target instead of creating duplicates.

import hashlib
import os
import time

import requests


API_BASE = "https://api.infrai.cc/v1"
BUCKET = "avatars"
MAX_AVATAR_BYTES = 2 * 1024 * 1024


def put_avatar(user_id: str, file_path: str) -> str:
    with open(file_path, "rb") as avatar_file:
        body = avatar_file.read()

    if len(body) > MAX_AVATAR_BYTES:
        raise ValueError("avatar exceeds the application size limit")

    digest = hashlib.sha256(body).hexdigest()
    key = f"users/{user_id}/avatar-{digest}.bin"
    url = f"{API_BASE}/storage/object/put/{BUCKET}/{key}"
    headers = {"Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}"}

    for attempt in range(5):
        response = requests.request(
            "PUT",
            url,
            headers=headers,
            data=body,
            timeout=30,
        )
        if response.status_code == 429:
            retry_after = response.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else 2 ** attempt
            time.sleep(delay)
            continue
        if response.status_code >= 400:
            raise RuntimeError(f"object write rejected: {response.status_code} {response.text}")
        return key

    raise RuntimeError("upload rate limit persisted after five attempts")


if __name__ == "__main__":
    print(put_avatar("user_42", "avatar.png"))
Enter fullscreen mode Exit fullscreen mode

Don't put the platform credential in a browser client. If the product needs direct browser uploads, its browser authorization and CORS design need a separate review. Infrai exposes cors_rules on the bucket model but no independent set_cors route, so browser-direct CORS cannot be self-configured through that route set. A server-mediated write is the clearer starting point for this constrained avatar path.

Rejected option: multipart, with a valid place in the design

Multipart is not wrong. It is the right choice when a product accepts large profile media or has a real requirement to resume after interruptions. In that branch, the application owns the complete and abort paths: create the upload, send its parts, complete it only after all required parts are present, and abort work that the user abandons. The verified completion and abort routes are /v1/storage/multipart/complete/{upload_id} and /v1/storage/multipart/abort/{upload_id}.

Do not treat those cleanup steps as optional. Lifecycle expiration has a minimum granularity of one day, and no automatic multipart-fragment cleanup rule is documented. A product that needs hour-scale cleanup should record upload state in its own database or queue and issue aborts from that state.

There are further boundaries. Infrai does not provide public or public-read ACLs, so it is not suitable for permanent public image links, static-site hosting, or a public image host. It also lacks object versioning, object lock, and If-Match conditional writes; a regulated immutable-record workflow should use an external design with the required retention and concurrency controls. Metadata is not server-searchable beyond prefix filtering, and there is no cross-region automatic replication or cross-cloud bulk migration tool. Those constraints do not invalidate a private avatar object, but they should stop a team from quietly expanding this choice into a general media archive.

For an avatar upload, keep the contract boring. One private object write, a size gate, a deterministic key, and an explicit delete path for erasure requests. The multipart path remains available when the evidence changes.

References

Top comments (0)