Audio MCP Server

audio-mcp

Audio perception for AI agents. Language models can't hear. This MCP server turns sound into things they can reason about: measurements, plain-language problem reports, spectrogram images, note lists, event timelines, transcripts and before/after diffs. It can also edit and synthesize audio, so an agent can work in a loop: make or fix a sound → measure it → look at it → verify the change.

Everything runs locally. No GPU, no API keys. Core dependencies: numpy, scipy, soundfile, matplotlib.

Real output (JSON abbreviated) from the bundled demo, scripts/demo.py:

> audio_analyze voice_raw.wav          # speech recorded with mains hum and hiss
  levels: lufs_integrated -32.16, true_peak_dbtp -11.76, clipped_samples 0
  issues: [warning] Mains hum at 60 Hz (peaks at 60, 180 Hz). Fix: notch each with
                    {"op":"eq","hz":60,"db":-30,"q":30}, {"op":"eq","hz":180,"db":-30,"q":30};
                    for voice, highpass at ~80-100 Hz also removes the fundamental.
          [info]    Noise floor around -33.4 dBFS in the quietest parts; hiss or background noise may be audible.

> audio_events sfx.wav                 # a zap, a noise hit, another hit, a rising chirp
  0.09-0.36s | -4dB  | tonal | balanced          | glide G#6->B3 (-33st) | sharp attack
  0.59-0.69s | -16dB | noisy | very bright/harsh | sharp attack
  ...

> audio_melody melody.wav              # "Mary Had a Little Lamb", transcribed blind
  E4(0.23) D4(0.24) C4(0.23) D4(0.24) E4(0.25) E4(0.25) E4(0.5) D4(0.24) D4(0.26) D4(0.49) E4(0.25) G4(0.26) G4(0.47)

Tools

Reads WAV, FLAC, OGG/Vorbis, Opus, MP3, AIFF and CAF natively (via libsndfile). With FFmpeg installed it also reads M4A/AAC (phones, Zoom, voice memos), WMA and the audio of video files (MP4, MOV, WebM, MKV): winget install Gyan.FFmpeg on Windows, brew install ffmpeg on macOS, sudo apt install ffmpeg on Debian/Ubuntu, or set AUDIO_MCP_FFMPEG to the binary. Each file is decoded once per session. Edited files are written as WAV/FLAC/OGG/AIFF/MP3.

Delivery checks

audio_check_delivery tells you whether a file meets a platform's spec and how to fix it if it doesn't. Every criterion shows what's required, what was measured and whether it passes. The suggested fix is an ordered audio_process chain (noise handling before gain, true-peak limiting last) that has already been simulated on your file, and the tool says plainly when something can't be fixed automatically, or when heavy background noise needs a person's judgment. Spec numbers cite their official sources and the date they were checked.

Game projects (Godot 4)

audio_check_game_project(project_dir) audits every .wav/.ogg/.mp3 in a Godot 4 project. It infers categories (UI, SFX, voice, music, ambience) from folder and file names, or takes your own mapping. It reads the .import settings to find music that won't loop, sound effects that loop forever, and loop seams that click. It compares loudness within each category (short sounds by their loudest moment, music by integrated LUFS) and reports outliers with fixes you can run directly. It also scans scenes and scripts for broken, wrongly cased or unused res:// audio references. It never modifies the project.

Install

pip install audio-mcp              # or: uv tool install audio-mcp
pip install "audio-mcp[speech]"    # adds audio_transcribe (faster-whisper; downloads a ~145 MB model on first use)

Connect it to your MCP client

Claude Code

claude mcp add audio -- uvx audio-mcp
# or, if installed with pip:
claude mcp add audio -- audio-mcp

Claude Desktop, Cursor, and other clients (JSON config):

{
  "mcpServers": {
    "audio": { "command": "uvx", "args": ["audio-mcp"] }
  }
}

Then ask things like "check the sound effects in ./sfx for problems", "clean up interview.wav for a podcast" or "design a short UI confirm sound and check it isn't harsh".

Restricting file access

By default the server can read and write any file your user account can. To confine it, set AUDIO_MCP_ALLOWED_DIRS to a list of directories (separated by ; on Windows, : elsewhere):

{ "command": "uvx", "args": ["audio-mcp"], "env": { "AUDIO_MCP_ALLOWED_DIRS": "/home/me/audio" } }

Paths are checked after resolving .. and symlinks. Other settings: AUDIO_MCP_MAX_SECONDS (largest section loaded into memory per call, default 1800), AUDIO_MCP_WHISPER_MODEL (default base), AUDIO_MCP_ROOT (base for relative paths).

How accuracy is checked

The tests mostly use synthetic signals whose correct answer is known by construction, so they check correctness rather than just "no crash":

• Loudness: our streaming BS.1770 meter matches pyloudnorm's exact-filter mode within 0.01 LU from 8 to 96 kHz, and reads the EBU reference tone (997 Hz, 0 dBFS) at −3.01 LUFS.
• Limiter: a +15 dBTP input comes out at exactly −1.0 dBTP with no samples touched by the safety clip.
• Pitch and melody: sines, chirps and synthesized melodies are recovered note for note, including repeated notes; pitch is within 1 cent from 82 Hz to 1760 Hz.
• Tempo, onsets, events, silence, stereo phase, hum, clipping, alignment offsets: each is checked against a constructed signal.
• A contract test runs every tool on every content type (silence, DC, noise, full-scale square), length (1 sample to 3 s), format (16/24/float WAV, FLAC, Vorbis), channel count (1, 2, 6) and sample rate (8–96 kHz). Each call must either succeed or fail with an AudioError that explains itself.
• Memory: analysis is chunked. A 20-minute stereo 48 kHz file analyzes in about 19 s with about 1.2 GB peak memory, 0.7 GB of which is the audio itself.

Design principles

• Never destructive by default. Edits write <name>_processed.wav; overwriting needs overwrite=true, and batch output can't target the input folder.
• Errors say what to do next. An unknown op lists every op with its parameters; an undecodable file names the supported formats and how to install FFmpeg for the others. Unexpected errors are reported as bugs instead of being hidden.
• Honest about uncertainty. Chords set polyphony_likely instead of reporting a confident wrong pitch, weak tempos carry a low-confidence note, Whisper's uncertain words are listed, and LUFS is null for clips under 0.4 s.
• Compact output. Rounded numbers, sparklines and one-line event summaries keep responses small enough for an agent's context.

Limitations

• Pitch, melody and per-event pitch assume one note at a time. Chords and full mixes are flagged, not transcribed.
• Denoise is a spectral gate for steady noise (hiss, hum, fans, room tone). It doesn't remove clicks, music or other voices. Too much sensitivity dulls quiet consonants, so verify speech edits with audio_transcribe.
• Tempo is a classic autocorrelation estimate; half- and double-tempo errors happen on complex music.
• The tools describe sound, but they don't judge taste ("is this satisfying?").

Reference signals

audio-mcp-assets is a public-domain pack of test and calibration audio (tones, sweeps, calibrated noise, loudness, stereo and phase tests, impulses, problem files) that an AI agent built using only these tools. Every file is verified against known ground truth, twice, including independently of this project.

Contributing

Improvements from people and AI agents are welcome. See CONTRIBUTING.md; AI coding agents should also read AGENTS.md. The bar for a feature is that it does one thing well, with a test against known ground truth.

License

MIT