The thing I say most often to Codex may not be the request. It may be the patch that comes after it.
“One more thing: don't touch the backend.”
“Wait. Give me a plan first. Don't edit yet.”
“And leave the database alone.”
The longer a spoken request gets, the easier it is for those boundaries to disappear between fillers, repetitions, and late corrections. A person can tell which sentence is noise and which one is a hard constraint. A model may flatten both, then hand back a task brief with cleaner grammar and much broader authority.
Local Prompt Refiner grew out of that narrow problem. It has Web, CLI, and macOS menu-bar entry points. It takes pasted or recorded Chinese and turns it into an execution brief, a plan, a /goal, or JSON for tools such as Codex, Claude Code, and Cursor Agent.
But the more I worked on it, the smaller its definition became: its job is not to make a request sound smarter. Its job is to keep explicit boundaries from falling out during the rewrite.
The model interprets. The code is not allowed to forget
Spoken-request cleanup looks like a pure text problem: remove repetition, restore punctuation, add headings. A model alone can produce a convincing demo.
The gap between “convincing” and “ready to execute” is authority.
Remove “only” from “only change the UI” and the scope changes. Turn “plan first” into “plan and implement” and the action starts too early. Drop “do not touch the database” from the summary and the coding agent has no way to know it just crossed a line. That is not a slightly worse summary. It is unauthorized work disguised as a reasonable next step.
So I stopped trying to solve the whole thing with a more persuasive system prompt. The pipeline has two jobs instead: the model interprets and restructures; the program protects obligations it can detect explicitly.
After generation, the service normalizes sections, scrubs leaked thinking, compacts Plan output, then checks the original input again for hard constraints such as “do not change the backend,” “UI only,” “plan first,” “do not touch the database,” and “no architectural rewrite.” If the model dropped one, deterministic code restores it.
Ambiguity handling is much less magical. The current code recognizes three patterns: scope contradictions, “plan first” combined with “execute now,” and subjective UI wording that needs a more concrete target. Markdown and Plan put detected cases in Ambiguities / Need attention; /goal uses Stop if; JSON writes them to the ambiguities array. Long, messy fixtures now check these recognized patterns too, but the mechanism is still regex- and fixture-based. This is not a semantic judge, and it will not catch every fuzzy request.
This guardrail is not intelligent. It is pattern-based and limited. That is also why it is useful: it does not wake up with a different opinion on the second generation. The model can answer, “What does this person probably want?” Code can enforce, “They said this line. We do not get to pretend they didn't.”
JSON mode made the split even more obvious. In a real evaluation, the model returned a string where the schema required an array. The content looked reasonable. The contract was still broken.
I do not negotiate with that output for long. A non-empty scalar in an array field becomes a one-item array; an empty string becomes an empty array. Missing fields, invalid JSON, placeholders, or more serious shape errors get one strict-schema repair retry. If that fails, the tool returns the error and raw output. Stop. A mechanically verifiable shape error can be repaired. Inventing content until the schema passes only hides the failure.
The four modes encode permission in different shapes. Execute means action is authorized. Plan means inspect and propose first. /goal gives a longer task explicit stop conditions. JSON is for software. They are not four skins over the same answer.
The 8B model was not a compromise. The task had finally become small enough
I took the obvious detour at first: surely a larger model would be safer.
The recorded tests were less cooperative. On one targeted Plan fixture, qwen3.6:35b-a3b timed out. Qwen3-8B-MLX-4bit passed the same fixture in about 8.1 seconds on average. A later full run covered ten Chinese spoken-request fixtures across three output modes, thirty cases in total, and reported 30/30 passing.
I do not read that as “the small model beat the large model.” It proves something narrower: on this machine, for these fixtures and this transformation task, an 8B model that returns reliably and accepts deterministic repair was more useful than a 35B request that failed on latency.
As soon as the suite expanded toward longer and messier inputs, the edge showed up. Three fast Plan cases kept their required headings and guardrails and leaked no thinking, but all crossed the 1,400-character warning line. A contradictory-scope fixture also passed the hard checks while remaining visibly too long. The model was good at what the test required and verbose where the test was still soft. Very model-like. Also useful evidence.
That is why I would rather expand the fixture set before reaching for a larger model. A narrow task produces nameable failures. Once a failure has a name, it can become the next test. Model size is one control, not the conclusion.
The voice input is part of the contract
When I added local whisper.cpp transcription to the menu-bar app, I expected Chinese speech recognition to be the difficult part. The actual failure was more basic: it recorded the wrong microphone. I spent time tuning transcription while feeding Whisper digital silence. In other words, I was doing NLP on air.
The system default on this Mac happened to be BlackHole 2ch, a virtual audio device. The early recording path followed the system default. The format looked valid, builds passed, tests passed, and the result could still be digital silence. The fix was not another Whisper prompt. The app now enumerates inputs, persists the user's choice, prefers physical devices, and binds recording to a specific device UID. If the selection looks like BlackHole, Loopback, or another routing device, the UI warns about it directly.
The code and thirty Swift tests pass, but I will not call the microphone problem solved. In an independent acoustic loopback, the built-in microphone still produced peak 0 and RMS 0. Another Continuity input produced non-zero audio but did not reliably capture the sentence being played. Whisper transcribed a directly generated version of the same sentence correctly, which isolates the remaining failure to the physical input path, but the full end-to-end acceptance check is still incomplete.
Voice is not just another input widget here. It is the source of the contract. If the device never captures “do not touch the backend,” no guardrail can preserve it later. The sentence never entered the system. So the voice-path test has to begin with “did we record the right sound?” rather than “does this transcript look vaguely Chinese?”
The narrow gate has one job
Local Prompt Refiner stores no history and does not bundle a cloud model service. Its default path is a local front end or menu-bar app calling a local Node service, which then connects to an OpenAI-compatible endpoint you configure. If that endpoint also runs locally, the text path can stay on the machine. Point it at a remote host and it is obviously no longer “fully offline.”
I like that shape because the tool does not pretend to understand every request. It catches one repeated action: turn speech into something an agent can execute, then check the places where permission is easiest to lose.
Now, after I hand it a rambling request, I do not inspect the headings first. I look for the three lines from the opening: “UI only.” “Plan first.” “Leave the database alone.”
If all three are still there, the gate worked. If one is missing, nine polished headings will not rescue it.