AI coding agents still need human cleanup on most tasks

AI coding agents still need human cleanup on most tasks. A benchmark built from real user change requests shows the best model, GPT‑6 Astra, fully solves just 26.4% of tasks, according to The Decoder.

What happened

The Decoder reports that researchers from Alibaba and several Asian universities created Co‑Design Bench, a test that asks coding agents to rework existing web apps using diverse inputs such as images, audio, video, or documents. Ten models were evaluated; GPT‑6 Astra met about 70% of a task’s requirements on average but produced a fully correct result in only 26.4% of cases. The next‑best models were Claude Opus 5 and Qwen3.8‑Max, while Grok 4.6 ranked last. The benchmark emphasizes strict grading: one AI judge runs the finished app in a browser, and a second judge verifies every positive verdict against mock‑ups or hard‑coded results.

Performance fell as tasks combined more change types (layout, design, interaction, content), reflecting real‑project complexity. Extra compute did not help; GLM‑5.2 consumed nearly six times the tokens of GPT‑6 Astra on smaller changes yet scored worse. The study also found that the harness (the environment through which the agent reads, writes, and executes code) can swing results—Qwen3.8‑27B scored 75% in one harness but only 56% in another.

Why it matters for small marketing teams

For a small company that relies on rapid website tweaks, landing‑page experiments, or internal tools, the headline‑grabbing claim that AI can code end‑to‑end is premature. The benchmark shows that even the strongest agents leave at least one detail unfinished in three‑quarters of requests, so a marketer’s “no‑code” shortcut still requires a developer or technically‑savvy staff member to review and fix the output. That extra hand‑holding erodes the speed advantage AI promises and adds hidden labor costs, which can affect campaign timelines, brand consistency, and budget forecasts. Moreover, the environment‑dependent results mean that simply picking a model isn’t enough; teams must test the full toolchain before trusting it with customer‑facing changes.

What to do this week

1. Pick a low‑risk webpage (e.g., an internal FAQ) and run an AI coding agent on a single change; then have a developer spend 30 minutes reviewing the output against a checklist.
2. Create a simple post‑generation QA checklist (functionality, design consistency, hard‑coded data) that any AI‑produced code must pass before publishing.
3. Run the same task in two different harnesses (if your platform offers them) and compare the results to see which environment yields fewer manual fixes.

Sources

• AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test — The Decoder