Palette Project Teaches Small Open-Weight Models to Build Decks

Frontier models are great at generating slide decks, but they inevitably drift back to their default house styles and require sending sensitive business data to external servers. To solve this without sacrificing quality, IBM Research has introduced Palette, a technique for teaching small, open-weight models to build presentations in a specific organizational design language. According to the Hugging Face Blog, researchers fine-tuned a 20B model using synthetic data and a verification harness, allowing teams to keep ownership of their brand styles and data.

What happened

As detailed in the Hugging Face Blog, slide generation typically forces a model to tackle editorial judgment and precise geometry simultaneously, a task where small models often fail—resulting in overlapping shapes, missing titles, and text running off slides. Palette tackles this by splitting the work into two distinct roles served by a single LoRA adapter: a designer that handles editorial planning, and a coder that draws each slide via code placement.

Because there is no public corpus of slide creation reasoning, the team generated around 1,000 decks (roughly 10,000 slides) using a large open teacher model. Each synthetic training example includes counterfactual reasoning explaining why certain design choices—such as chart type or font sizing—were made or rejected. Furthermore, every generated slide passes through a deterministic harness that renders the output, checks for layout overflows or geometry errors, and invokes a separate editor model to fix broken code while ensuring the fix actually improved the slide.

When evaluated on PresentBench using plans derived from source documents, a fine-tuned 20B model equipped with the harness scored competitively against frontier systems, ranking first on two of the three measured domains.

Why it matters for small teams

For small companies and product teams, this architecture represents a major step forward in self-hosted, domain-specific AI automation. When you are building client-facing materials, internal roadmaps, or financial reviews, you cannot afford data leakage to external LLM providers, nor can you waste billable hours fixing generic, off-brand AI templates.

Palette demonstrates how to build production-grade agentic loops for constrained output formats. By decoupling editorial planning from programmatic rendering—and wrapping the output in a deterministic validation harness—you can reliably deploy smaller, cheaper open-weights models that adhere strictly to internal brand guidelines. The inclusion of bracketed natural-language directives (e.g., [[make this a timeline]]) also gives teams a lightweight way to steer agent outputs without complex prompt engineering.

What to do this week

1. Audit your internal document templates and design guidelines to identify repetitive formatting tasks that currently require manual developer or designer intervention.

1. Experiment with splitting your agent workflows into explicit planning and execution phases, separating high-level structural logic from code generation or API execution.

1. Implement a post-generation validation step or harness in your local pipelines to programmatically catch layout, syntax, or data-faithfulness errors before outputs reach your team.

Sources

• Palette: teaching a small model to build decks in your design language — Hugging Face Blog