Google's new Gemini 3.5 Transcribe reported a 2.6% word error rate on recorded audio this week, an independently measured, genuine improvement. But 2.6% is not zero. On a 1,000-word meeting transcript, that's roughly 26 wrong words -- and AssemblyAI's own published breakdown of a modern speech-recognition model's errors found that the large majority, 81%, are substitutions: a confident wrong word standing in for the right one, not an obvious gap. Those are exactly the errors a skim doesn't catch.
Why the failures are invisible, not obvious
The errors worth worrying about are homophones and dropped words, not garbled nonsense. AssemblyAI's own writeup gives the starkest example: a missing negation turns "the patient is allergic to penicillin" into its dangerous opposite -- the sentence still reads fine, it just says the wrong thing. Names and technical terms fail the same way: a person's name spelled a plausible-but-wrong way, a term autocorrected into a similar-sounding common word. None of these look like errors on a skim, which is exactly why skimming isn't the check.
Check an AI transcript in this order
- A run of odd substitutions -- swapped words, garbled proper nouns -- usually means a language or accent mismatch, not a one-off mistake.
- This is the highest-frequency error category and the fastest to verify against a participant list, an agenda, or a company directory.
- Money amounts, percentages, dates, and measurements are the category most likely to be silently wrong in a way that still reads as confident and correct.
- Negation words are short, often unstressed in speech, and are exactly the words a model is most likely to drop without leaving any sign something's missing.
- Sample across the recording rather than relistening to all of it -- if the same error type shows up in more than one sample, treat it as happening throughout, not just in the spots you checked.
The order matters more than the thoroughness. Fixing punctuation before checking whether a dollar figure is right just means the highest-stakes error sits uncaught the longest.
Three ways this check gets skipped
None of this makes an AI transcript untrustworthy -- 2.6% is a genuine improvement over what shipped even a year ago. It just means the number on the announcement and the number that matters for your specific recording are two different things, and only one of them is checkable from a chair.
- A low word-error rate still means real errors -- 2.6% is roughly one wrong word every 40.
- Most transcription errors are substitutions, not deletions: a wrong word, not a missing one.
- Check names, numbers, and negations first -- they change meaning, not just spelling.
- Spot-check a few risk-based samples against the audio instead of relistening to everything.
- Caveat: the same error type showing up twice usually means it's happening throughout, not just there.