Output Layer: Keep the Model's Mouth Shut
Explicit negative constraints cut ~30% fluff; polish with Diff, don't rewrite the whole passage; use stop sequences as a hard cut
THE QUESTION THIS PAGE ANSWERS
ANSWER FIRSTWhat is the key idea behind “Output Layer: Keep the Model's Mouth Shut”?
Explicit negative constraints cut ~30% fluff; polish with Diff, don't rewrite the whole passage; use stop sequences as a hard cut
Read cost as a shape, not a single number. Break a request into input, output, retries, tools, and waiting time. The shape of usage usually tells you which design choice is expensive and where a smaller change can help.
Measure one real request before you optimize an imagined average.
A cheaper call that quietly creates more retries, latency, or review work.
Lots of people know to ask for “please answer concisely”—but that fails—“concise” is too abstract for the model; it doesn't know how concise you want. What works is spelling out “don't do X”:
In practice, Agentic scenarios cut about 30% of the fluff. Plain talk: tell it—skip the extras, just deliver the result.
Biggest cost trap in text polish: the user says “make this sentence smoother,” and the model reprints the whole 2,000-word article. Even if you constrain it to one paragraph, Tokens still burn fast.
Better idea: make the model output only what changed—mark “original → revised” in Diff form, or return a regex/replace instruction for the program to apply. Touch two or three words, done in a second: cost differs by tens of times, and the UX is better—faster, and users see exactly what changed without diffing by hand.
Writing “please output only 3 items” in the Prompt is mysticism—maybe 3 items, maybe 5 plus a summary. But stop sequences are a hard cut; they don't care whether the model “obeys”: the API kills the generation stream when it sees the string; the cut part isn't billed and never enters history context.
Simple. Blunt. Effective. Controlling output is controlling cost and UX.
Negative constraints must be concrete: “no small talk, no summary, no pleasantries” beats “please be concise”; Agentic scenarios cut ~30% fluff.
Polish with Diff or replace instructions—don't let the model rewrite the whole passage: cost differs by tens of times, UX gets better.
Stop sequences are a physical switch: cut lists on “4.”, single-line answers on newline, JSON on “}”, self-Q&A on “User:”.
Source: Adapted from the author's internal team share “AI Token Cost Engineering Strategy” hands-on section “04｜Output.” Stop-sequence parameters: see each vendor's API docs (OpenAI-compatible interfaces use the stop field, up to 4 sequences).
The complete interaction cost of “Instruction layer · Write explicit negative constraints”
“Lots of people know to ask for “please answer concisely”—but that fails— “concise” is too abstract for the model;” is a reminder that AI cost is not one price multiplied by one call. Input, output, retries, tools, waiting time, and human cleanup together decide what a task really costs.
Find what the bill repeats
The key variables behind “In practice, Agentic scenarios cut about 30% of the fluff .” are usually repeated context, oversized output, retries after failure, and calls that do not produce useful progress. Removing wasted Tokens can reduce cost, latency, and concurrency pressure at the same time.
A cheaper call can make the whole workflow more expensive
Start with “Stop sequences are a physical switch: cut lists on “4.”, single-line answers on newline, JSON on “}”, self-Q&A on “User:”” and keep a small table for input, output, retries, tools, and human review. Compare quality before and after optimizing instead of looking at one price in isolation.
From “Instruction layer · Write explicit negative constraints” to “Code layer · Polish with Diff, don't rewrite the whole passage”
“Instruction layer · Write explicit negative constraints” grounds the problem in “Lots of people know to ask for “please answer concisely”—but that fails— “concise” is too abstract for the model; it doesn't know how concise you want . What works is spelling out “don't do X””. “Code layer · Polish with Diff, don't rewrite the whole passage” then moves it toward “Biggest cost trap in text polish: the user says “make this sentence smoother,” and the model reprints the whole 2,000-word article. Even if you constrain it to one paragraph, Tokens still burn fast”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.
Carry the judgment into the next situation
When analyzing cost, map the complete interaction first, then find repeated input, wasted output, and retries. A cheap individual call does not make the whole task cheap.
- “Instruction layer · Write explicit negative constraints”: Lots of people know to ask for “please answer concisely”—but that fails— “concise” is too abstract for the model; it doesn't know how concise you want . What works is spelling out “don't do X”
- “Code layer · Polish with Diff, don't rewrite the whole passage”: Biggest cost trap in text polish: the user says “make this sentence smoother,” and the model reprints the whole 2,000-word article. Even if you constrain it to one paragraph, Tokens still burn fast
- “The closing point”: Polish with Diff or replace instructions— don't let the model rewrite the whole passage: cost differs by tens of times, UX gets better
The final “The closing point” brings the discussion to “Polish with Diff or replace instructions— don't let the model rewrite the whole passage: cost differs by tens of times, UX gets better”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.
I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.
After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.
When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.
No discussion on this article yet.