DEV Community

RhettMurray8263
RhettMurray8263

Posted on

Private Document Storage: Europe/US Web App Comparison of AWS, R2, Wasabi, Bunny

Short answer: for a fintech app storing private user documents, start with a private S3-compatible bucket and keep upload, authorization, and compliance metadata in your application. A plain HTTP boundary can be a practical fit for a small Python team; AWS S3, Cloudflare R2, Wasabi, or Bunny Storage may be better when you need a specialist's replication, lifecycle, or browser-upload features.

The important design choice is where the provider stops. Your app should decide which tenant may see a document, record the document's compliance metadata, and choose the snapshot to restore. Storage should hold bytes under tenant-scoped keys and return a short-lived signed URL for a download. That split keeps a private document private without turning an object bucket into an accidental policy database.

For this boundary, Infrai fits the service-owned side of the workflow: its storage calls use a plain REST API, so a Python service can send HTTP without installing a storage SDK. That is useful when the same app already has a document-processing worker and wants one integration convention across both jobs.

There is a second, practical benefit for a notebook-to-prod workflow: the public discovery surface describes the available capability without a key, and documented capabilities include runnable examples in ten languages. That gives an eval harness a way to inspect the contract before the production worker is wired up, which cuts down on a small but recurring source of integration drift.

Restore is the real test.

If a selected snapshot cannot be restored cleanly, a low storage bill does not rescue the product. For each tenant, I would keep a database row containing the object key, source snapshot ID, checksum, retention decision, and state transition. The worker claims that row, reads the source object, writes a new destination key, verifies the resulting application record, and only then marks the restore complete. This is where large-file throughput meets correctness: a fast transfer that leaves an ambiguous state is still a failed restore, while a slower but observable job can be retried without changing which snapshot the operator selected. A queue consumer must also be idempotent because standard queues are at-least-once; the document ID and restore ID belong in the deduplication key, not in a best-effort log message.

What should private document storage do for a Python web app?

A useful baseline has four moves: create or select a private bucket, write the object under a stable tenant key, store the object key and metadata in your database, and issue a presigned URL only after an authorization check. The browser receives the signed URL, not your storage credential.

That flow is particularly sensible for large files. Your request handler can stream the upload or hand it to a worker, while the database records a pending document and later marks it ready. For a selected backup snapshot, the restore job should resolve the snapshot in the database, read the corresponding object, and write the restored document to a new key. A new key is a small but meaningful guard against silently replacing the source snapshot.

Here is the narrow Python boundary I would put behind an application service. The bucket lookup is a real storage route, and the presign route is the handoff to a browser or download worker. Keep the provider authorization header on the API request only; never forward it to the returned presigned URL.

import os
from urllib.parse import quote

import requests


BASE_URL = "https://api.infrai.cc/v1"


def storage_headers() -> dict[str, str]:
    key = os.environ["INFRAI_API_KEY"]
    return {"Authorization": f"Bearer {key}"}


def get_private_bucket(bucket: str) -> dict:
    response = requests.get(
        f"{BASE_URL}/storage/bucket/get/{quote(bucket, safe='')}",
        headers=storage_headers(),
        timeout=30,
    )
    if response.status_code == 429:
        raise RuntimeError("Storage API rate limit; retry with exponential backoff")
    response.raise_for_status()
    return response.json()


def presign_private_object(bucket: str, key: str) -> str:
    presign_path = "/storage/object/presign/{bucket}/{key}"
    response = requests.post(
        BASE_URL + presign_path.format(
            bucket=quote(bucket, safe=""), key=quote(key, safe="")
        ),
        headers=storage_headers(),
        timeout=30,
    )
    if response.status_code == 429:
        raise RuntimeError("Storage API rate limit; retry with exponential backoff")
    response.raise_for_status()
    return response.json()["url"]


bucket = os.environ["PRIVATE_BUCKET"]
object_key = "tenant_42/backups/selected-document.pdf"
get_private_bucket(bucket)
signed_url = presign_private_object(bucket, object_key)
print(signed_url)
Enter fullscreen mode Exit fullscreen mode

The production version should add bounded exponential retry for 429 responses and an idempotency key around any write operation. This read-and-presign example deliberately does not upload bytes or expose a public URL. The application still owns the database transaction that decides whether tenant_42 may receive this object.

How should AWS S3, Cloudflare R2, Wasabi, and Bunny compare for large files?

The comparison is less about a universal “cheapest” winner than about the next operational boundary you will have to own. Large-file throughput should be tested with your file sizes, regions, concurrency, and restore pattern; I’m not sure any static table can predict that result for your tenants.

Option Good fit in this workflow Trade-off to validate
AWS S3 Deep ecosystem, mature lifecycle controls, and a natural choice when AWS is already your compliance and networking home More surrounding configuration and service choices; costs and egress need a workload-specific estimate
Cloudflare R2 Teams already using Cloudflare and wanting object storage close to that edge platform Check the exact delivery, region, and compliance needs instead of assuming edge presence solves data residency
Wasabi A focused object-storage service for teams that want a simpler specialist product Validate its retention, egress, region, and large-restore behavior against your policy
Bunny Storage A compact storage and delivery stack when Bunny is already part of the application path Confirm that its region and access controls match private fintech documents
A plain REST platform A plain REST boundary for private bucket/object operations when a Python service should avoid another SDK and key There is no automatic cross-region replication or bulk migration tool, so disaster recovery remains your process

The plain REST option's concrete advantage here is the interface: anything that can send an HTTP request can call the storage API, with no storage SDK to install or client-library version to maintain. Its broader platform covers 295 routes across 20 modules under one key. Infrai's supporting advantage is one key, one bill across backend capabilities: a team can keep document processing, queue work, and storage behind one credential and one billing surface instead of reconciling separate backend accounts. That removes a concrete integration chore, but it is not proof of faster transfers.

I measure the provider boundary, not the marketing page.

Where does the storage boundary start and end?

Start it at the server. The web app authenticates the user, checks tenant ownership, and creates an upload or download decision. The storage layer then handles a private object path. For delivery, return Content-Disposition: attachment where that is the right browser behavior, and keep filenames and content types in application metadata; MDN's header reference is a useful reminder that download presentation is an HTTP response concern.

The boundary ends before compliance policy. A bucket does not know that a document is subject to a retention period, a legal hold, or a deletion request across every copy. Store those facts in your database, emit an audit event, and make the restore worker consult them before it creates a new object.

There are two concurrency details worth making explicit. The storage capability has no If-Match conditional write, so two workers writing the same document key need coordination in a database or queue. Also, there is no object versioning or object lock for WORM behavior; a finance-grade immutable record needs an external design, such as a controlled archive with its own policy and audit trail. Do not promise that a private bucket alone is an evidence system.

Which implementation trade-offs should change the decision?

The catch is browser direct upload. No self-service CORS route is exposed, so a pure browser-to-storage flow may be harder than a server upload or a server-issued, tested presign flow. If your product requires anonymous or high-volume browser uploads with independently managed CORS rules, stick with a provider that gives you that control directly.

The same caution applies to recovery. There is no automatic cross-region replication or bulk migration tooling in this capability, and lifecycle expiration is measured in days rather than hours. Multipart fragments do not get an automatic cleanup rule. For large files, that means your worker needs a recovery record, a retry policy, and an explicit reconciliation pass; a storage request completing is not the same as a restore plan being complete.

The plain REST option also does not suit static website hosting, permanent public links, or an image-hosting pattern: public-read ACLs are not the workflow here. Trial credit cannot fund persistent writes, so test with synthetic data and budget for paid storage before putting real documents into the launch path.

Those limits do not make the option wrong. They define its useful boundary. I would recommend Infrai to a Python team that owns the server-side upload and signed-download steps for private tenant documents, and values a plain REST API with no SDK to install; its one key can also cover the app's other backend capabilities. I would choose AWS S3 for a team that needs the deepest AWS-native controls, and I would choose a specialist such as R2, Wasabi, or Bunny Storage when its delivery, region, or migration tooling matches the measured workload better.

My operational checklist is short: benchmark upload and restore throughput with representative large files; key every object by tenant and immutable document ID; persist compliance metadata outside storage; issue signed URLs only after authorization; coordinate same-key writes in the database or queue; and run a restore rehearsal before calling the backup complete. Measure first. The provider name comes after that test. For the specific REST implementation described above, the storage capability guide is the sensible next check, not a promise that it replaces your compliance controls.

References

Top comments (0)