SceneBloom: An Agent Framework for Generating Realistic 3D Indoor Scenes

Training embodied agents requires large amounts of realistic, diverse indoor environments that can be used directly for simulation. However, manually building such scenes is costly, and existing generative methods struggle to balance visual plausibility with physical validity. SceneBloom proposes an agent framework that combines visual layout sampling with contract-guided 3D compilation to automatically generate high-quality, collision-free indoor scenes, providing a new data source for embodied learning.

Why Do We Need Automatic Indoor Scene Generation?

Embodied agents (such as home service robots) need to navigate and manipulate objects in various indoor environments. To train agents with strong generalization capabilities, researchers need massive, diverse, and physically plausible 3D scenes. Real scanned data, while realistic, is limited in quantity and uneven in scene type coverage; manually building simulation environments is time-consuming and difficult to scale.

Language-guided methods can generate scene layouts from text descriptions, supporting semantic planning and object-level reasoning, but they still struggle with precise spatial localization. Visual generative models provide rich layout priors, but their outputs require further geometric reasoning to be converted into physically valid 3D scenes. SceneBloom is designed precisely to bridge this gap.

Embodied agents refer to intelligent systems that can perceive, make decisions, and execute actions in physical or simulated environments, such as home service robots, autonomous vehicles, or virtual characters. Unlike models trained solely on text data, the capabilities of embodied agents depend heavily on interaction experience with environments, making "environment diversity" and "physical realism" of training data crucial. Real-world data collection is costly and limited in coverage, while simulated environments can be generated infinitely and annotated precisely—but manual construction is time-consuming and difficult to scale. SceneBloom targets exactly this data bottleneck: using generative models to automatically produce 3D indoor scenes that can be used directly for simulation, thereby providing a scalable data engine for embodied learning.

SceneBloom: A Two-Stage Agent Framework

The core of SceneBloom is a two-stage pipeline: first, a hierarchical sampler generates furniture layout proposals, and then a compiler transforms them into physically plausible 3D scenes.

Hierarchical Sampler: Diffusion-Prior Layout Generation

The hierarchical sampler leverages diffusion model priors to generate furniture arrangements within functional zones. It divides rooms into different functional areas (such as rest areas and work areas), then samples furniture positions and orientations within these areas. This hierarchical strategy ensures both layout diversity and functional plausibility—for example, sofas typically face the TV, and desks are near windows.

Diffusion models are a class of generative models that learn a gradual denoising process to recover data distributions from random noise. In scene generation tasks, diffusion models can learn statistical patterns from large amounts of real indoor layouts, thereby sampling furniture arrangements that conform to human habits given room outlines and functional zones. The so-called "prior" refers to the layout distribution knowledge the model learns from training data, which enables the sampler to generate diverse and plausible proposals without explicit rules. However, diffusion model outputs are inherently probabilistic 2D or abstract layouts and do not guarantee non-overlapping objects or physical constraint compliance—which is precisely why the subsequent compiler needs to intervene.

Contract-Guided 3D Compiler: From Proposals to Physically Valid Scenes

The sampler outputs only 2D or abstract layout proposals, and the compiler is responsible for transforming them into actual 3D scenes. The compiler extracts inter-object relationships from each proposal (such as "chair next to table" or "bed against wall") and uses them as guidance for geometric fitting and local pose correction. In this way, SceneBloom ensures that objects in the generated scenes do not penetrate each other, do not exceed room boundaries, and have physically sensible poses.

Contracts here refer to spatial relationship constraints between objects extracted from layout proposals, such as "distance between chair and table is between 0.3 and 0.8 meters," "one side of the bed is flush against the wall," or "sofa directly faces the TV." These constraints transform abstract 2D layouts into executable geometric conditions, based on which the compiler retrieves and fits models from a 3D asset library, then applies local pose corrections to satisfy collision detection and boundary constraints. The contract-guided approach is similar to specification-driven generation in program synthesis: first declare "what needs to be satisfied," then solve "how to place it," thereby ensuring physical validity while maintaining generation diversity.

Experimental Results: Leading in Visual Quality and Physical Stability

The researchers compared SceneBloom with existing methods across five room types. Results show that SceneBloom achieves the highest levels in both visual quality and physical stability, and is the only method with no detected mesh collisions or out-of-bounds objects. Moreover, its generated layout distributions are closer to real room distributions than all baseline methods.

More importantly, object navigation policies trained in these generated scenes achieved higher success rates on real scanned data not involved in training than those trained with an equivalent amount of real training scans. This demonstrates that SceneBloom-generated scenes not only look realistic but can also be effectively used to train embodied agents, potentially even compensating for the scarcity of real data.

Significance and Outlook for Embodied Learning

The value of SceneBloom lies in providing a path to automatically generate high-quality simulation environments. Compared to traditional methods that rely on manual construction or limited real scans, SceneBloom can generate diverse scenes on demand while guaranteeing physical validity. This provides a scalable data engine for embodied intelligence research.

Of course, current methods still have some limitations. For example, they mainly focus on furniture layout and may not yet cover finer-grained objects (such as small items on desktops); the diversity of generated scenes is limited by the diffusion model's prior distribution; and the contract extraction and compilation process may also introduce errors. Future work can explore richer scene elements, more precise physical simulation, and interaction with other modalities (such as text and speech).

SceneBloom's code is open-sourced on GitHub, providing a reproducible foundation for future research. With the rise of embodied intelligence, such scene generation tools will become increasingly important.

One-line Summary

SceneBloom uses a two-stage framework of diffusion sampling and contract compilation to automatically generate physically valid 3D indoor scenes.

Paragraph Summary

SceneBloom is an agent framework aimed at embodied intelligence data needs, designed to automatically generate high-quality, collision-free 3D indoor scenes. It employs a two-stage pipeline: a hierarchical sampler leverages diffusion model priors to generate furniture layout proposals within functional zones; a contract-guided 3D compiler then extracts spatial relationships between objects from the proposals, performs geometric fitting and pose correction to ensure physical validity. Experiments comparing with existing methods across five room types show that SceneBloom is optimal in both visual quality and physical stability, and is the only method with no mesh collisions or out-of-bounds objects, with layout distributions also closer to real rooms. More importantly, navigation policies trained in its generated scenes achieved higher success rates on real scanned data than policies trained with an equivalent amount of real data, demonstrating that generated scenes can serve as an effective data source for embodied learning.