3DCodeEdit Benchmark: Instruction-Driven 3D Program Editing, Best Model Success Rate Below 60%

Language models are increasingly generating 3D content through code, but in real-world workflows, editing existing programs is often more common than regenerating from scratch—the former is faster and offers more precise control. However, such editing requires models to not only understand what a program builds, but also to anticipate how a single change will reshape the final result. Existing benchmarks mostly evaluate generation capability or simple edits against reference results, lacking systematic measurement of complex editing tasks. To address this, researchers introduce 3DCodeEdit, a benchmark containing 2,592 instruction-guided editing tasks across 108 Blender programs that build objects entirely with code.

Task Design: From Parameter Tweaks to Scene Reconstruction

3DCodeEdit tasks span a wide range, including parameter modification, structural changes, style redesign, and scene composition. Each task requires the model to modify existing code based on natural language instructions, with final evaluation based on rendered outputs checked against human-reviewed checklists. This evaluation approach is closer to real creative scenarios: users care not about the code itself, but whether the rendered 3D object meets expectations.

Compared to previous benchmarks, 3DCodeEdit emphasizes editing rather than generation, and the complexity of edits is higher—not something that can be completed by simply replacing a single value.

Blender procedural modeling refers to creating and manipulating 3D objects through scripts using Blender's built-in Python API (bpy), rather than relying on manual modeling. Scripts can parametrically define geometry, materials, lighting, and cameras, so the same piece of code can generate different variants by changing values. The advantage of this approach is reproducibility, batch generation capability, and it is naturally suited as an output target for language models—the model generates executable code, and the renderer transforms it into images. The 108 programs in 3DCodeEdit are exactly such Blender scripts that build objects entirely from code, so editing tasks become modifications to Python code rather than direct manipulation of mesh vertices.

17 Models Average Success Rate Only 37%

Researchers tested 17 models, with an average success rate of only 37%. Even the best-performing models achieved less than 60% completion on structural and style edits. This indicates that current models still have significant shortcomings in understanding the geometric structures built by code and predicting the consequences of modifications.

Interestingly, as model capabilities improve, failure modes shift. Earlier models often failed to generate runnable code, while stronger models' failures concentrate more on "geometric foresight": the code runs, but the resulting shape is wrong. In other words, models have learned to write syntactically correct code, but have not yet learned to simulate the results of geometric transformations in their minds.

Geometric foresight refers to the ability of a model to internally reason, before actually executing code or rendering, about how a code modification will change 3D shapes, spatial layouts, and visual results. It differs from code syntactic correctness: a piece of code can run without errors yet produce a model with distorted proportions, misaligned parts, or mismatched style. For language models, this requires mapping symbolic code to continuous 3D geometric transformations, and such mapping signals are relatively sparse in training data. Therefore, while more capable models rarely write crashing code, they more often exhibit "runs but wrong shape" failures—a manifestation of the geometric foresight gap.

Multi-Turn Feedback: Compensating for Geometric Foresight Gaps

Since single-pass generation struggles to guarantee geometric correctness, researchers explored whether multi-turn feedback can compensate for this deficiency. They compared various feedback strategies, including increasing reasoning effort, using agent frameworks, and introducing independent critic models. Results showed that feedback from an independent critic model—describing differences between rendered results and requests—worked best.

This feedback not only outperformed maximizing reasoning effort but also outperformed agent frameworks. More strikingly, a 27B-parameter model with such feedback achieved a single-pass success rate exceeding that of a 2.8T-parameter model. This means that through reasonable external feedback mechanisms, smaller models can match or even surpass the one-shot output level of giant models on complex editing tasks.

Independent critic model feedback refers to using a model separate from the generation model to inspect rendered results, describe differences between the rendered image and user requests in natural language, and feed these differences back to the generation model for correction. Compared to having the same model extend its reasoning chain or self-reflect, an independent critic model can access rendered image information that the generation model cannot see, thereby providing more objective and targeted signals. In experiments, this feedback outperformed increased reasoning effort and agent frameworks, indicating that external, render-based evaluation is more effective at compensating for geometric foresight deficiencies than simply increasing computation.

Implications for 3D Creative Workflows

The introduction of 3DCodeEdit provides a more realistic evaluation standard for 3D program editing. For developers relying on code to generate 3D content, this research suggests that simply pursuing larger models or longer reasoning times may not solve the geometric foresight problem; introducing iterative feedback on rendered results may be a more effective path.

At the same time, the benchmark itself exposes common shortcomings of current models—structural and style editing remain difficult. In the future, how to enable models to "see" geometric results before code execution, or design more efficient feedback loops, will be key directions for improving 3D program editing capabilities.

One-line Summary

3DCodeEdit benchmark reveals insufficient geometric foresight in models editing 3D code; independent critic feedback significantly improves success rates.

Paragraph Summary

Researchers introduce the 3DCodeEdit benchmark, containing 2,592 instruction-guided editing tasks across 108 Blender programs that build objects entirely with code, with tasks ranging from parameter tweaks to structural changes, style redesign, and scene composition. Testing 17 models shows an average success rate of only 37%, with the best models achieving under 60% on structural and style edits. As model capabilities improve, failure modes shift from inability to generate runnable code to insufficient geometric foresight—code runs but shapes are wrong. The research further finds that render-difference feedback from an independent critic model is most effective, enabling a 27B model's single-pass success rate to exceed that of a 2.8T-parameter model, outperforming maximized reasoning effort and agent frameworks.