Music generation

The

For session-backed agent runs, OpenClaw starts music generation as a background task, tracks it in the task ledger, then wakes the agent again when the track is ready so the agent can post the finished audio back into the original channel.

Quick start

text
Copy
  <Step title="Pick a default model (optional)">
    ```json5}
    {
      agents: {
        defaults: {
          musicGenerationModel: {
            primary: "google/lyria-3-clip-preview",
          },
        },
      },
    }
    ```
  </Step>

  <Step title="Ask the agent">
    *"Generate an upbeat synthpop track about a night drive through a
    neon city."*

    The agent calls `music_generate` automatically. No tool
    allow-listing needed.
  </Step>
</Steps>

For direct synchronous contexts without a session-backed agent run,
the built-in tool still falls back to inline generation and returns
the final media path in the tool result.

text
Copy
  <Step title="Cloud auth (optional)">
    For Comfy Cloud, set `COMFY_API_KEY` or `COMFY_CLOUD_API_KEY`.
  </Step>

  <Step title="Call the tool">
    ```text}
    /tool music_generate prompt="Warm ambient synth loop with soft tape texture"
    ```
  </Step>
</Steps>

Example prompts:

text
Copy
Generate a cinematic piano track with soft strings and no vocals.

text
Copy
Generate an energetic chiptune loop about launching a rocket at sunrise.

Supported providers

Capability matrix

The explicit mode contract used by

Use

text
Copy
/tool music_generate action=list

Use

text
Copy
/tool music_generate action=status

Direct generation example:

text
Copy
/tool music_generate prompt="Dreamy lo-fi hip hop with vinyl texture and gentle rain" instrumental=true

Tool parameters

Output filename hint. Optional provider request timeout in milliseconds.

Async behavior

Session-backed music generation runs as a background task:

• Background task:
  text

  Copy

  music_generate
  creates a background task, returns a started/task response immediately, and posts the finished track later in a follow-up agent message.
• Duplicate prevention: while a task is
  text

  Copy

  queued
  or
  text

  Copy

  running
  , later
  text

  Copy

  music_generate
  calls in the same session return task status instead of starting another generation. Use
  text

  Copy

  action: "status"
  to check explicitly.
• Status lookup:
  text

  Copy

  openclaw tasks list
  or
  text

  Copy

  openclaw tasks show <taskId>
  inspects queued, running, and terminal status.
• Completion wake: OpenClaw injects an internal completion event back into the same session so the model can write the user-facing follow-up itself.
• Prompt hint: later user/manual turns in the same session get a small runtime hint when a music task is already in flight, so the model does not blindly call
  text

  Copy

  music_generate
  again.
• No-session fallback: direct/local contexts without a real agent session run inline and return the final audio result in the same turn.

Task lifecycle

Check status from the CLI:

bash
Copy
openclaw tasks list
openclaw tasks show <taskId>
openclaw tasks cancel <taskId>

Configuration

Model selection

json5
Copy
{
  agents: {
    defaults: {
      musicGenerationModel: {
        primary: "google/lyria-3-clip-preview",
        fallbacks: ["minimax/music-2.6"],
      },
    },
  },
}

Provider selection order

OpenClaw tries providers in this order:

1. text

   Copy

   model
   parameter from the tool call (if the agent specifies one).
2. text

   Copy

   musicGenerationModel.primary
   from config.
3. text

   Copy

   musicGenerationModel.fallbacks
   in order.
4. Auto-detection using auth-backed provider defaults only:
   • current default provider first;
   • remaining registered music-generation providers in provider-id order.

If a provider fails, the next candidate is tried automatically. If all fail, the error includes details from each attempt.

Set

Provider notes

Choosing the right path

• Shared provider-backed when you want model selection, provider failover, and the built-in async task/status flow.
• Plugin path (ComfyUI) when you need a custom workflow graph or a provider that is not part of the shared bundled music capability.

If you are debugging ComfyUI-specific behavior, see ComfyUI. If you are debugging shared provider behavior, start with Google (Gemini) or MiniMax.

Provider capability modes

The shared music-generation contract supports explicit mode declarations:

• text

  Copy

  generate
  for prompt-only generation.
• text

  Copy

  edit
  when the request includes one or more reference images.

New provider implementations should prefer explicit mode blocks:

typescript
Copy
capabilities: {
  generate: {
    maxTracks: 1,
    supportsLyrics: true,
    supportsFormat: true,
  },
  edit: {
    enabled: true,
    maxTracks: 1,
    maxInputImages: 1,
    supportsFormat: true,
  },
}

Legacy flat fields such as

Live tests

Opt-in live coverage for the shared bundled providers:

bash
Copy
OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts

Repo wrapper:

bash
Copy
pnpm test:live:media music

This live file loads missing provider env vars from

• text

  Copy

  google
  :
  text

  Copy

  generate
  plus
  text

  Copy

  edit
• text

  Copy

  minimax
  :
  text

  Copy

  generate
  only
• text

  Copy

  comfy
  : separate Comfy live coverage, not the shared provider sweep

Opt-in live coverage for the bundled ComfyUI music path:

bash
Copy
OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts

The Comfy live file also covers comfy image and video workflows when those sections are configured.

Related

• Background tasks — task tracking for detached
  text

  Copy

  music_generate
  runs
• ComfyUI
• Configuration reference —
  text

  Copy

  musicGenerationModel
  config
• Google (Gemini)
• MiniMax
• Models — model configuration and failover
• Tools overview