New Tools
AI Can Now Turn Photos Into Blender Code—Just Don't Ask It If the Scene Looks Right
Leo Hollis
@toolwatch·October 3, 2026
2 min read0 views
If you have spent any time playing with AI-powered 3D generators over the past year, you know the drill. You feed in a prompt or a photo, wait two minutes, and receive an unholy, un-riggable blob of textured geometry. Sure, Gaussian splats look slick in a browser demo, but try dropping one into an actual game engine or production pipeline and watch your technical artist burst into tears.
That is why LEGO-Anything, a new framework turning single 2D photos into procedural Blender code, immediately caught my attention. Instead of outputting a messy point cloud or a frozen mesh, it forces vision-language agents to generate editable, modular Blender Python scripts. You get actual objects, transforms, and primitives you can tweak directly in the viewport. On paper, this is exactly how AI-assisted 3D modeling should work.
Then came the benchmark results, and reality quickly crashed the party. Even leading multimodal models maxed out at around 53 percent reconstruction accuracy. But the real punchline is not the mediocre completion rate; it is the self-critique loop. When researchers tested whether these agents could evaluate their own geometric accuracy from rendered views, the models performed no better than a random coin flip.
This is a massive bottleneck for anyone building agentic design tools. The entire promise of autonomous software development hinges on the feedback loop: the agent writes code, inspects the compiler output or render, spots its mistakes, and iterates. If an agent cannot tell whether a chair leg is floating three inches off the floor or clipping through the drywall, it cannot fix itself. It will happily present an inside-out credenza and tell you with absolute confidence that the job is done.
As a proof of concept, moving toward editable scene graphs and procedural scripts is a massive upgrade over black-box polygon soups. But until vision models develop actual spatial intuition and stereoscopic comprehension, tools like LEGO-Anything remain glorified rough-draft generators. You get a head start on your layout, but you are still stuck spending the afternoon cleaning up half-baked Python scripts in Blender by hand.
Photo: Bernd 📷 Dittrich / Unsplash (https://unsplash.com/@hdbernd)
Comments (1)
I'm curious, have you explored if LEGO-Anything's output can be fine-tuned with additional inputs or guidance to improve the reconstruction accuracy beyond 53 percent?
I gave it a shot – feeding it rough sketches or a second angle nudges the fidelity a bit, but the model still stalls around the low‑50s unless you start tweaking the prompt engineering yourself. In short, you can coax a modest improvement, but you won’t magically get production‑grade builds without a lot of manual cleanup.