Contract-Guided 3D Scene Compilation: Eliminating Geometric Hallucinations in Embodied AI

Training autonomous robots and embodied agents requires vast, structurally diverse, and physically grounded 3D environments. While visual diffusion models and Large Language Models (LLMs) can dream up plausible semantic layouts, they fail miserably at strict spatial arithmetic. Meshes clip through walls, bounding boxes intersect, and floating assets break simulation engines.

In this article, we examine the architecture behind SceneBloom (Liu et al., 2026), an agentic framework that bridges the gap between probabilistic layout generation and deterministic physical compilation.

The Failure Mode: Why Generative Geometry Breaks Simulators

Most current pipeline architectures attempt to solve scene generation in one of two suboptimal ways:

1. Pure LLM Prompting: Models generate bounding boxes or programmatic scene graphs (e.g., in Python or Blender API). Lacking continuous spatial vision, LLMs routinely misjudge clearances, generating narrow corridors that navigation agents cannot traverse.
2. End-to-End Diffusion: Generates visually stunning rooms, but converts into messy non-manifold geometry, mesh interpenetrations, and ungrounded physics.

When navigation policies (such as ObjectNav or PointNav) train inside these defective worlds, reinforcement learning agents exploit collision glitches. Transferring these policies to real hardware leads directly to catastrophic failure.

The Architecture: Split Proposal from Compilation

SceneBloom decomposes the problem into an explicit two-stage pipeline: probabilistic proposal sampling followed by contract-guided constraint compilation.

[ Functional Zoning Prior ] 
           │
           ▼
[ Hierarchical Diffusion Sampler ] ── (Proposes 2D/2.5D Layout & Spatial Priors)
           │
           ▼
[ Relational Contract Extractor ]  ── (Derives topological constraints: on, next_to, clear_path)
           │
           ▼
[ Contract-Guided 3D Compiler ]    ── (Non-linear optimization: solves geometry without collisions)
           │
           ▼
[ Simulation-Ready 3D Twin ]       ── (0 mesh collisions, 0 out-of-bounds assets)

1. Hierarchical Layout Sampling

Rather than treating an entire room as a single flat distribution, the sampler first partitions space into functional zones (e.g., sleeping area, workstation, storage corridor). A visual diffusion model seeds object placements within these functional boundaries, retaining realistic arrangement priors.

2. Contract-Guided Geometric Compilation

Once candidate assets are retrieved from a CAD repository, the system extracts a set of spatial relational contracts:

• Containment Contracts: All oriented bounding boxes (OBBs) must strictly remain inside the room polyhedron.
• Disjointness Contracts: Intersection volume between distinct asset meshes must equal zero.
• Support and Clearance Contracts: Functional pathways must preserve passable Voronoi corridors for robot kinematic footprints.

The compiler solves these contracts via local pose optimization and iterative projection, ensuring that the final output satisfies mathematical guarantees before entering the simulation engine.

Concrete Engineering Takeaways

1. Enforce Deterministic Compilers Over Raw Generation: Never feed raw outputs of generative models directly into execution or simulation engines. Generative layers should propose distributions; mathematical constraint solvers must compile them.
2. Synthetic Data Can Outperform Real Scans: Policies trained on SceneBloom achieved higher navigation success on held-out physical scans than policies trained on equal numbers of real Matterport3D scans. Clean synthetic variety with enforced physical stability eliminates capture artifacts and sensor noise.
3. Decouple Semantics from Kinematics: By separating semantic zoning (LLM/diffusion) from collision resolution (geometric optimization), engineering teams can debug failure modes independently.

Building dependable cyber-physical systems demands zero tolerance for spatial hallucinations. SceneBloom demonstrates that contract-driven compilation is the missing link between generative models and production robotics.

Canonical source: bohdanhordii.dev

Author: Bohdan Hordii | Website: bohdanhordii.dev | Contact: @bohdan_hordii