Here is the disk layout of one mobile mapping survey — a vehicle with a LiDAR scanner and a panoramic camera, driven along a road:
data/001_MMS/ 507 MB point cloud
orbit/oblak/ 566 MB spherical photos
trajectory/*.gpkg 108 KB the path the vehicle drove
Just over a gigabyte. The database this feeds holds 2.3 GB in total — for 2.7 million road features across a hundred layers. Two more surveys and the binary data outweighs everything the database has ever stored.
So the question isn't how to put a point cloud in Postgres. It's what you put in Postgres instead.
The trajectory is the index
Of that gigabyte, one file goes into the database: the 108 KB trajectory, a GeoPackage holding the line the vehicle drove.
That line is what makes the survey findable. It draws on the map with everything else. You can ask which surveys cover a junction, which are newest, whether a stretch of road has been captured since the resurfacing. All the questions people actually ask are questions about where and when, and the trajectory answers every one of them at 0.01% of the storage.
The heavy files never enter the database. The row holds paths:
class Cloud(models.Model):
name = models.CharField(max_length=120, db_index=True)
path_name = models.CharField(max_length=120) # -> octree metadata JSON
orbit_url = models.CharField(max_length=255) # -> spherical photo index
spherical_photo = models.BooleanField(default=False)
recording_date = models.DateField(null=True)
source_srid = models.IntegerField(null=True, choices=SOURCE_SRID_CHOICES)
available = models.BooleanField(default=True)
Metadata, geometry, and pointers. That's the whole trick, and it isn't clever — it's just the discipline to not reach for a bytea column.
Why not in the database
Postgres will happily store a gigabyte. It's the access pattern that kills you.
A browser point cloud viewer doesn't fetch a point cloud. It fetches an octree: a tree of small files, and as the user moves the camera it pulls the nodes covering what's in view at the detail level that's visible. Zoom in, it fetches deeper nodes. Pan away, it drops them. A single session issues hundreds of small ranged reads driven by mouse movement.
That is precisely the workload a static file server is built for, and precisely the one a database connection pool is not. Serving it through Django would mean an application worker occupied for every node fetch, connection pool pressure from mouse movement, and no benefit whatsoever — there is no query, no join, no permission decision per node beyond the one already made when the survey was opened.
nginx serves the directory directly:
location /media/ {
alias /app/media/;
# CORS - needed for local frontend dev servers (different origin/port)
# fetching point cloud (Potree octree/hierarchy) and other media files
# directly via fetch()/XHR. No credentials involved, so a wildcard is safe.
add_header 'Access-Control-Allow-Origin' '*' always;
}
That wildcard needs the justification written next to it, which is why the comment is there. It's safe because no credentials ride along: the octree nodes are opaque binary that mean nothing without the metadata, and the metadata comes from the authenticated API. Change either of those facts and the wildcard becomes a mistake.
Where the size actually goes
27 GB orthophotos
24 GB prepared point clouds
74 MB symbology (icons for signs, poles, cameras)
1.5 MB project thumbnails
The orthophotos are the bigger half, and they follow a different path — served as Cloud-Optimised GeoTIFFs through a raster tile server, cached at nginx for a day. Different data, different access pattern, different tool. What they share is that neither one is in Postgres.
Note the shape of the tail: two entries measured in gigabytes, everything else in megabytes. That's typical, and it's the argument for treating "large binary" as its own tier rather than a column type. The 74 MB of symbology icons could live in the database without anyone noticing. The 24 GB could not.
The escape hatch, and what it costs
Not all data can be copied. Some surveys are enormous and already sitting on a storage array, and duplicating them to bring them into the system is not worth 500 GB.
So there's a symlink path: point the system at data that lives elsewhere, and it appears under media/external/<hash>/ as if it had been uploaded.
This works, and it has a cost that must be paid explicitly. A symlink is a reference the database doesn't own, so deleting the row has to clean up the link too — otherwise media/external slowly fills with pointers to nothing:
@receiver(post_delete, sender=Cloud)
def delete_cloud_symlinks(sender, instance, **kwargs):
"""Remove the symlink folders under media/external when a Cloud is deleted."""
Every "just point at the existing files" shortcut buys you disk and sells you a lifecycle problem. Worth it here — media/external is 116 KB of links standing in for far more — but the cleanup is not optional, and a signal is the cheapest place to guarantee it runs.
The field that has to be asked for
One more field in that model earns its place: source_srid.
The trajectory arrives in whatever coordinate system the surveyor worked in, and — as with shapefiles that ship without a .prj — the file often doesn't say. So the model offers the choices, and the pipeline prefers the file's declared CRS, falls back to the user's selection, and refuses to guess.
That dropdown is small and easy to get wrong. I found one of its options labelled with an EPSG code from the wrong country, which would have put a trajectory about 5,000 km from the road it was recorded on.
The rule
Put in the database what you'll query. Put on disk what you'll stream.
For survey data that means: the trajectory, the recording date, the coordinate system, the paths, and a flag for whether the thing is ready to view. Not the octree, not the panoramas, not the orthophoto.
The test I'd apply to any large asset: is there a question someone will ask that requires this to be in a table? For a point cloud the honest answer is no — every question is about the trajectory, which is 0.01% of the bytes and answers all of them.
Top comments (2)
This is a very solid architecture decision, especially the principle:
“Put in the database what you’ll query. Put on disk what you’ll stream.”
For LiDAR, panoramic imagery, orthophotos, and other large geospatial assets, the important design question isn't whether PostgreSQL can store the binary data—it can. The real question is whether the database should participate in every read.
I particularly like treating the trajectory as the spatial index. It keeps the database responsible for metadata, spatial relationships, temporal filtering, permissions, and asset discovery, while object/file storage handles the high-volume streaming workload.
For a production system, I'd extend this architecture with a few additional considerations:
Object storage over local disk where possible (S3/Azure Blob/MinIO), with immutable asset paths and lifecycle policies.
Signed URLs or short-lived access tokens instead of exposing /media/ publicly when datasets become sensitive.
CDN/cache layers for octree nodes and frequently accessed imagery.
Asset manifests + checksums so ingestion can be idempotent and corrupted/missing files can be detected.
Explicit ingestion states such as pending → processing → ready → failed, rather than relying only on available.
Spatial indexes on trajectories (PostGIS) so coverage queries remain fast as the survey catalog grows.
Background processing for generating Potree/LOD structures, COGs, thumbnails, and metadata rather than tying those operations to web requests.
The CRS point is also critical. Never silently guess a coordinate reference system. A technically valid geometry in the wrong CRS is often more dangerous than an obvious failure because it can look correct at first glance while being geographically meaningless.
One additional pattern I'd consider is separating the logical asset identity from its physical storage location. That makes migrations from local filesystem → NAS → S3 → cloud archive much easier without changing the application domain model.
The same architectural principle applies well beyond mapping: video, medical imaging, satellite imagery, CAD files, ML datasets, and other large immutable assets.
The interesting engineering problem isn't “How do I store 500 GB?”
It's “What information belongs in the database so that I never need to move 500 GB to answer a 50-byte question?”
That's where good architecture starts. I'd be very interested in discussing how you're approaching the next stage of this system—especially storage abstraction, ingestion pipelines, and scaling the survey catalog.
Thanks - you picked out the two things I left implicit, so let me answer them directly.
Object storage and signed URLs. Both right, neither available to me here. This runs on-prem, and that's also why the symlink hatch exists: half a terabyte already sitting on someone's array isn't getting copied anywhere, least of all to S3. And when the data does become sensitive, I don't think per-node signed URLs are the answer. Potree issues hundreds of node fetches per session, and signing each one is a lot of machinery for a single permission decision. A short-lived cookie scoped to the survey directory, checked with nginx auth_request in front of the alias, puts the decision back where it already happens: once, at open time. The wildcard CORS header goes out with the same change.
Logical identity vs physical location is the sharpest thing you said, and it's the actual debt in that model. path_name and orbit_url are filesystem paths living in database columns, precisely the coupling that makes local to NAS to object storage painful.
media/external// is a half-step toward the indirection you're describing: the application already pretends everything sits under media/, it just does it with symlinks instead of a storage backend. Swapping that for a key plus a pluggable backend is the migration I'd do before anything about scale.
On ingestion states - agreed, available is a boolean doing a state machine's job, and the failure mode is specific: a conversion that dies partway leaves a directory that looks complete from the outside. On checksums, I'd scope them to the octree hierarchy rather than the whole asset; hashing 500 GB you deliberately chose not to copy gives most of the win back.
Spatial indexes on the trajectories aren't the bottleneck yet. The catalog is small and per-node latency dominates. Ask me again at ten thousand surveys.
Which brings me to the number I don't have: have you served an octree straight off object storage rather than a filesystem? Specifically what per-node latency looks like under a few hundred ranged reads driven by mouse movement, with and without a CDN in front.
That's the measurement that decides whether the storage abstraction is a refactor or a rewrite