Speech recognition makes a particular kind of mistake in Cantonese: homophones. Many characters share a sound, so a transcript can be phonetically perfect and still use the wrong words. Gongje has an optional second pass for this — a small language model running locally with MLX that reads the transcript and fixes those errors.
The hard part was not getting the model to correct text. It was stopping it from doing anything else.
What Can Go Wrong
Ask a chat model to “fix this sentence” and it is eager to help. It rephrases. It answers a question you dictated instead of transcribing it. It adds an explanation, a heading, a polite introduction. It translates. For a dictation tool every one of those is a bug: the user said specific words, and the text that gets pasted should be those words.
Gongje supports several local models — Qwen3 at 1.7B, 4B or 8B parameters, and a Cantonese-tuned Qwen2 7B — so the safeguards could not depend on any one model behaving well. They had to work around the model, not inside it.
Layer One: A Narrow Prompt
The system prompt gives the model one job and three rules:
- Only fix homophone or near-homophone errors; keep the original when uncertain.
- Preserve punctuation, line breaks, numbers and proper nouns.
- Keep the word count the same, and output only the corrected text.
It includes examples of exactly the kind of fix we want (“went too the store” → “went to the store”), and it is localized, so the default prompt matches the app's language. A good prompt makes good output likely. It does not make bad output impossible, so the next two layers assume the prompt failed.
Layer Two: Clean the Input and the Output
Before the transcript reaches the model, markup-like content is stripped from it. Text spoken by the user is data, and it should never be able to look like instructions to the model.
After generation, the output is rejected outright if:
- it contains angle brackets — the model has started producing markup;
- it contains phrases from a list of known chat-assistant boilerplate;
- it shows signs of the model talking about itself;
- it is empty once a leading “corrected:” label is removed.
Rejected output is not retried. Gongje falls back to the original transcript, which is always a safe answer.
Layer Three: Measure the Edit
The last check does not look at what the model wrote, only at how much it changed. A homophone fix swaps characters one for one; it does not add sentences or rewrite clauses. So a correction is accepted only if:
- the number of lines is unchanged;
- the length changed by no more than an eighth of the original (at least two characters of slack for short phrases);
- the Levenshtein edit distance between input and output is within a limit — a third of the input length, with a floor of four.
That limit is scaled by which speech model produced the transcript. Transcripts from the small Whisper models get the full allowance, medium gets 85% of it and the large models 70%. A larger speech model makes fewer mistakes in the first place, so a large edit to its output is more likely to be the LLM drifting than the LLM fixing.
let maxLengthGap = max(2, input.count / 8)
if abs(input.count - output.count) > maxLengthGap { return false }
let maxDistance = max(4, Int(Double(input.count) / 3 * whisperModel.distanceScale))
return levenshtein(input, output) <= maxDistanceWhat We Learned
- Treat model output as untrusted input. Validate it the way you would validate anything arriving over a network.
- Always have a safe fallback. For a correction step, the uncorrected text is it. Users get a slightly worse transcript rather than a wrong one.
- Check the shape of the change, not just its content. Line count, length and edit distance are cheap, model-independent and catch the failures that matter most.
- Use what you know about upstream quality. Scaling the allowance by speech model size let one rule fit very different transcripts.
The correction step is an optional toggle in Gongje's settings, and both prompts can be edited under Advanced. The code, including these checks, is on GitHub.