Never Trust File Uploads: Building a Secure S3 Pipeline

File uploads look simple.

A user selects a file, your backend receives it, saves it to storage, and returns a URL.

Done, right?

Not really.

What if someone uploads a malicious file disguised as an image? What if hundreds of users upload large files at the same time? Or what if an infected PDF becomes available for everyone to download?

A simple upload feature can quickly become a security problem.

Let's look at how to build a safer file upload pipeline using S3, magic bytes, and ClamAV.

1. Stop sending files through your API server

A common approach is to receive files through your Node.js backend and then upload them to S3.

This works, but your API becomes a middleman.

Every upload uses server bandwidth and connections. If you're buffering files in memory, large uploads can also consume a lot of RAM.

A better approach is to use S3 presigned URLs.

Here's how it works:

1. The client requests an upload URL from your API.
2. Your API checks permission and generates a short-lived URL.
3. The client uploads the file directly to S3.
4. S3 stores the file in a private quarantine location.

Your API handles permissions, not the actual file transfer.

For example, using AWS SDK v3:

import { randomUUID } from "node:crypto";
import { S3Client } from "@aws-sdk/client-s3";
import { createPresignedPost } from "@aws-sdk/s3-presigned-post";

const s3 = new S3Client({ region: "us-east-1" });

export async function createUpload(userId: string) {
  // userId must come from the authenticated session
  const key = `quarantine/${userId}/${randomUUID()}`;

  const { url, fields } = await createPresignedPost(s3, {
    Bucket: "my-private-uploads",
    Key: key,
    Expires: 300,
    Conditions: [
      ["content-length-range", 1, 10 * 1024 * 1024],
    ],
  });

  return { url, fields };
}

This creates a presigned POST that allows uploads up to 10 MB and expires after five minutes.

The client sends the returned fields and file as multipart/form-data directly to S3.

The bucket stays private, and the backend should record the generated key against the authenticated user.

One important thing: a successful upload doesn't mean the file is safe.

We still need to inspect it.

2. Never trust file extensions or MIME types

Imagine someone uploads a file named profile.png.

Your backend checks:

if (file.mimetype === "image/png") {
  // Accept file
}

Looks fine, right?

The problem is that the MIME type usually comes from the client.

Anyone can rename a file or send a fake Content-Type header.

Instead, check the actual file content.

Check the magic bytes

Many file formats have recognizable binary signatures called magic bytes.

For example:

In Node.js, the file-type package makes this easier.

import { fileTypeFromBuffer } from "file-type";

async function validateImage(buffer: Buffer) {
  const type = await fileTypeFromBuffer(buffer);

  const allowed = ["image/png", "image/jpeg"];

  if (!type || !allowed.includes(type.mime)) {
    throw new Error("Unsupported file format");
  }

  return type;
}

You don't need to download the entire file just to check its signature.

For supported formats, you can read a small part of the object using an S3 range request. The number of bytes needed depends on the format and detection library.

But remember, valid magic bytes don't mean a file is safe.

A valid PDF or image can still contain harmful content or exploit a vulnerable file parser.

That's why we need another layer.

3. Scan files before making them available

This is the part I wouldn't skip.

Never make an uploaded file available to users before it passes validation and scanning.

Instead, use a quarantine system.

Here's the flow:

Client
  |
  | Request upload URL
  v
Node.js API
  |
  | Generate presigned URL
  v
Client
  |
  | Upload directly
  v
Private S3 Quarantine
  |
  | Storage event
  v
Background Worker
  |
  | Check file size
  | Check magic bytes
  | Scan with ClamAV
  |
  +-- Failed --> Reject
  |
  +-- Passed --> Approved Storage
                       |
                       v
                Available to User

For malware scanning, we can use ClamAV, a free, open-source antivirus engine.

A background worker can download or stream the quarantined object to the scanner.

Here's a simplified example using the clamscan package:

import NodeClam from "clamscan";

const clam = await new NodeClam().init({
  clamdscan: {
    host: "clamav-service",
    port: 3310,
  },
});

export async function scanFile(
  fileStream: NodeJS.ReadableStream,
) {
  const result = await clam.scanStream(fileStream);

  if (result.isInfected !== false) {
    throw new Error("File rejected");
  }

  return true;
}

This example only shows the scanning step. The worker still needs to retrieve the correct S3 object, validate it, handle failures, and update the database.

If the scan fails, times out, or returns an unknown result, keep the file quarantined.

Don't approve it automatically.

Also, ClamAV reduces malware risk, but it cannot detect every possible threat.

4. Let background workers handle the heavy work

Virus scanning can take time.

You don't want users waiting for it inside an HTTP request.

Instead, use a background queue like BullMQ or AWS SQS.

When S3 receives a file, a storage event can trigger a job.

The worker validates the file, scans it, and updates its status.

A simple database lifecycle could look like this:

PENDING_UPLOAD
      |
      v
 QUARANTINED
      |
      v
   SCANNING
    /    \
   v      v
APPROVED REJECTED

The frontend can show "Processing your file" while scanning is in progress.

Make sure workers can safely retry jobs and handle duplicate events.

There's another important detail here.

Scan and approve the exact same object.

Otherwise, a file could be replaced after scanning but before approval.

Use unique storage keys and object versioning or other controls that prevent an unscanned replacement from being published.

5. A few security rules worth following

Even with these three layers, I'd still add a few protections:

• Limit file sizes. Don't allow unlimited uploads.
• Use random file names. Generate UUID-based storage keys instead of trusting user-provided paths.
• Keep storage private. Approved files don't always need public URLs. Use authorized downloads when appropriate.
• Limit upload requests. Add authentication, rate limits, and storage quotas.
• Protect against ZIP bombs. Limit archive extraction size, file count, and nesting depth.
• Serve files safely. Use correct response headers and avoid serving risky user content from your application's main origin.
• Monitor failures. Track rejected uploads, scanner errors, and unusual upload activity.

And one underrated point: don't expose your ClamAV service directly to the internet. Keep it accessible only to trusted workers.

Final thoughts

File uploads are easy to build, but securing them takes a little more thought.

You don't need a complicated system from day one.

Start with three things:

1. Presign: Let clients upload directly to private object storage.

2. Validate: Check file size, format, and actual content instead of trusting filenames.

3. Scan: Run malware checks in the background before making files available.

These steps won't eliminate every security risk, but they'll give you a much safer foundation.

The main thing I've learned is simple:

Uploading a file and trusting a file should be two different steps.

How do you handle file uploads in your applications? Do you scan files before making them available, or use a different approach?

PS: A reader pointed out a gap I missed: abandoned uploads.

If a client's upload expires on a slow connection, it retries with a new key. The first file can stay in quarantine forever, because no scan ever runs for it.

The fix is simple. Add an S3 lifecycle rule on the quarantine prefix that deletes anything older than a day. Also run a small scheduled job to expire old PENDING_UPLOAD rows, so your database matches what's in the bucket.

Lifecycle rules run in the background, so deletion can happen a little after the one-day mark.

References

• AWS S3 Presigned URLs
• AWS S3 Presigned POST
• ClamAV Documentation
• OWASP File Upload Cheat Sheet