How Whisper runs offline on a Mac, explained

You hold a hotkey, say a sentence, and text appears at your cursor with Wi-Fi off. This post explains what happens in between: the model, the runtime that puts it on Apple Silicon, and the one trade off you actually choose. It is the same stack Akova uses to turn speech into prompts for Claude Code, Cursor and Codex.

The model: OpenAI Whisper

Whisper is a speech recognition model from OpenAI. Its README calls it “a general-purpose speech recognition model” trained on a large and diverse audio dataset. One model handles multilingual transcription, translation and language identification. The code and the weights are released under the MIT License, which is why so many Mac apps can ship it.

Whisper comes in sizes. OpenAI’s repository lists these:

Size	Parameters	English-only variant
tiny	39 M	tiny.en
base	74 M	base.en
small	244 M	small.en
medium	769 M	medium.en
large	1550 M	none
turbo	809 M	none

OpenAI describes turbo as “an optimized version of large-v3 that offers faster transcription speed with a minimal degradation in accuracy.” It also notes that the English-only models tend to do better on English, most of all at the tiny and base sizes.

The runtime: WhisperKit and Core ML

The original Whisper code is Python on PyTorch. That suits a server, not a menu bar app that should answer the moment you let go of a key.

WhisperKit is an open source Swift framework from Argmax, also MIT licensed, that runs Whisper on Apple devices. It uses Core ML, Apple’s framework for running machine learning models on the device itself. Argmax converts the Whisper weights to Core ML and hosts them on Hugging Face (argmaxinc/whisperkit-coreml). An app downloads a model once, keeps the files on disk, and from then on every transcription runs locally.

That is all “offline” means here. The network is used once, to fetch the model. Speaking, decoding and typing the text need no connection, and your audio is never uploaded.

You can see this for yourself with Argmax’s command line tool. It needs Xcode and git-lfs, and the commands come straight from the project’s README:

git clone https://github.com/argmaxinc/argmax-oss-swift.git
cd argmax-oss-swift
make setup
make download-model MODEL=large-v3-v20240930_626MB
swift run argmax-cli transcribe \
  --model-path "Models/whisperkit-coreml/openai_whisper-large-v3-v20240930_626MB" \
  --stream

The last command transcribes from your microphone. Once the model is downloaded, turn Wi-Fi off and run it again.

Which model size should you pick?

Bigger models are more accurate, slower and use more memory. That is the whole trade off. Two cautions before you read too much into the table above:

• OpenAI measured its relative speeds on an A100 GPU, transcribing English. They are not Mac numbers, and real speed depends on your chip, the language and how fast you talk.
• Argmax publishes its own advice for WhisperKit. It recommends large-v3-v20240930_626MB, a compressed Large v3 Turbo, for maximum multilingual accuracy, and the uncompressed turbo variant on macOS for “maximum speed and accuracy.” It calls tiny “only recommended for development & debugging.”

In practice: if you dictate English and want the fastest response, try a small English-only model. If you speak another language, switch languages mid sentence, or say a lot of technical words, a Large v3 Turbo variant earns its extra size.

No size is immune to one quirk: Whisper can write a stray “Thank you” into a clip where you said nothing. We cover the cause and the fixes in why Whisper types “Thank you” when you say nothing.

Checked on 10 October 2026 against the Whisper and WhisperKit READMEs.

What this means for prompts to coding agents

Prompts for a coding agent are full of identifiers: repo names, function names, flags, file paths. Whisper has never seen your codebase, so even a large model will guess at useAuthStore or pgbouncer. Two things help. A larger model guesses better, and a custom vocabulary tells the app the words you use. We explain the mechanics in why Whisper mishears code words. In Akova you teach it your repo names, APIs and jargon once, and they come out spelled right.

Transcription is also only half the job. A faithful transcript of you thinking out loud is not a good prompt: it has false starts, “no wait”, and the important constraint buried at the end. Akova transcribes with WhisperKit on your Mac, then Prompt mode turns the transcript into a structured prompt. That step runs locally too, on Apple Intelligence on macOS 26 or a local Ollama model on macOS 14 and 15. The whole pipeline stays on your machine.

If you want the wider comparison of local and server transcription, read on-device vs cloud dictation. For the setup in a real agent session, see how to dictate prompts into Claude Code.

Skip the build step

The command line tool is a good way to understand the stack, not a good way to dictate all day. Akova wraps it in a menu bar app: hold the hotkey, talk through the task, and the prompt lands at your cursor in any terminal, editor or chat box. It needs macOS 14 Sonoma or later on an Apple Silicon Mac.

The free tier includes custom vocabulary and file transcription, with no account. Prompt mode is a Lifetime feature, free for 7 days after your first launch. Compare Free and Lifetime on the pricing section, then pick the model size that suits the way you talk.