Use this file to discover all available pages before exploring further.
Audio and voice notes
Audio / Voice Notes (2026-01-17)
What works
- Media understanding (audio): If audio understanding is enabled (or auto‑detected), OpenClaw:
- Locates the first audio attachment (local path or URL) and downloads it if needed.
- Enforces before sending to each model entry.
- Runs the first eligible model entry in order (provider or CLI).
- If it fails or skips (size/timeout), it tries the next entry.
- On success, it replaces with an block and sets .
- Command parsing: When transcription succeeds, / are set to the transcript so slash commands still work.
- Verbose logging: In , we log when transcription runs and when it replaces the body.
Auto-detection (default)
If you don’t configure models and
tools.media.audio.enabled
is
not set to
,
OpenClaw auto-detects in this order and stops at the first working option:
- Active reply model when its provider supports audio understanding.
- Local CLIs (if installed)
- (requires with encoder/decoder/joiner/tokens)
- (from ; uses or the bundled tiny model)
- (Python CLI; downloads models automatically)
- Gemini CLI () using
- Provider auth
- Configured entries that support audio are tried first
- Bundled fallback order: OpenAI → Groq → xAI → Deepgram → Google → SenseAudio → ElevenLabs → Mistral
To disable auto-detection, set
tools.media.audio.enabled: false
.
To customize, set
.
Note: Binary detection is best-effort across macOS/Linux/Windows; ensure the CLI is on
(we expand
), or set an explicit CLI model with a full command path.
Config examples
Provider + CLI fallback (OpenAI + Whisper CLI)
{
tools: {
media: {
audio: {
enabled: true,
maxBytes: 20971520,
models: [
{ provider: "openai", model: "gpt-4o-mini-transcribe" },
{
type: "cli",
command: "whisper",
args: ["--model", "base", "{{MediaPath}}"],
timeoutSeconds: 45,
},
],
},
},
},
}
Provider-only with scope gating
{
tools: {
media: {
audio: {
enabled: true,
scope: {
default: "allow",
rules: [{ action: "deny", match: { chatType: "group" } }],
},
models: [{ provider: "openai", model: "gpt-4o-mini-transcribe" }],
},
},
},
}
Provider-only (Deepgram)
{
tools: {
media: {
audio: {
enabled: true,
models: [{ provider: "deepgram", model: "nova-3" }],
},
},
},
}
Provider-only (Mistral Voxtral)
{
tools: {
media: {
audio: {
enabled: true,
models: [{ provider: "mistral", model: "voxtral-mini-latest" }],
},
},
},
}
Provider-only (SenseAudio)
{
tools: {
media: {
audio: {
enabled: true,
models: [{ provider: "senseaudio", model: "senseaudio-asr-pro-1.5-260319" }],
},
},
},
}
Echo transcript to chat (opt-in)
{
tools: {
media: {
audio: {
enabled: true,
echoTranscript: true, // default is false
echoFormat: '📝 "{transcript}"', // optional, supports {transcript}
models: [{ provider: "openai", model: "gpt-4o-mini-transcribe" }],
},
},
},
}
Notes & limits
- Provider auth follows the standard model auth order (auth profiles, env vars,
models.providers.*.apiKey
- Groq setup details: Groq.
- Deepgram picks up when is used.
- Deepgram setup details: Deepgram (audio transcription).
- Mistral setup details: Mistral.
- SenseAudio picks up when is used.
- SenseAudio setup details: SenseAudio.
- Audio providers can override , , and via .
- Default size cap is 20MB (
tools.media.audio.maxBytes
- Tiny/empty audio files below 1024 bytes are skipped before provider/CLI transcription.
- Default for audio is unset (full transcript). Set
tools.media.audio.maxChars
- OpenAI auto default is ; set
model: "gpt-4o-transcribe"
- Use
tools.media.audio.attachments
- Transcript is available to templates as .
tools.media.audio.echoTranscript
tools.media.audio.echoFormat
- CLI stdout is capped (5MB); keep CLI output concise.
- CLI should use for the local audio file path. Run to migrate deprecated placeholders from older
audio.transcription.command
Proxy environment support
Provider-based audio transcription honors standard outbound proxy env vars:
If no proxy env vars are set, direct egress is used. If proxy config is malformed, OpenClaw logs a warning and falls back to direct fetch.
Mention detection in groups
When
is set for a group chat, OpenClaw now transcribes audio
before checking for mentions. This allows voice notes to be processed even when they contain mentions.
How it works:
- If a voice message has no text body and the group requires mentions, OpenClaw performs a "preflight" transcription.
- The transcript is checked for mention patterns (e.g., , emoji triggers).
- If a mention is found, the message proceeds through the full reply pipeline.
- The transcript is used for mention detection so voice notes can pass the mention gate.
Fallback behavior:
- If transcription fails during preflight (timeout, API error, etc.), the message is processed based on text-only mention detection.
- This ensures that mixed messages (text + audio) are never incorrectly dropped.
Opt-out per Telegram group/topic:
- Set
channels.telegram.groups.<chatId>.disableAudioPreflight: true
- Set
channels.telegram.groups.<chatId>.topics.<threadId>.disableAudioPreflight
- Default is (preflight enabled when mention-gated conditions match).
Example: A user sends a voice note saying "Hey @Claude, what's the weather?" in a Telegram group with
. The voice note is transcribed, the mention is detected, and the agent replies.
Gotchas
- Scope rules use first-match wins. is normalized to , , or .
- Ensure your CLI exits 0 and prints plain text; JSON needs to be massaged via .
- For , if you pass , OpenClaw reads
<output-dir>/<media-basename>.txt
- Keep timeouts reasonable (, default 60s) to avoid blocking the reply queue.
- Preflight transcription only processes the first audio attachment for mention detection. Additional audio is processed during the main media understanding phase.
Related