local-suno

Constrained decoding broke lyrics generation. This is the most useful counter-example I have measured. Forcing structured output was meant to make the format reliable; the result was the opposite: 8,192 tokens, 140 seconds, truncated JSON. Unconstrained: 708 tokens, 13.8 seconds, valid JSON.

The cause was not the tool but the schema — unbounded, open objects and lists. Under a hard constraint the model has no reason to stop and loops to the ceiling. Rule kept and locked by a test: constrained decoding does not compensate for an unbounded schema, it amplifies it.

Durations quoted warm. The pipeline answered in 5 to 7 seconds — with the model already loaded. On the first call after eviction, expect roughly two minutes. Quoting the warm figure was the surest way to disappoint: timeouts and UI must be sized on the cold one.