Recut solves a real problem I have — how to automate the process of editing music using LLM capabilities, whether a specific requirement for the edit is present or is written vaguely (by somebody who is not a trained musician and does not understand the underlying musical semantics).
How it works
- Drop an audio file of the music you wish to edit
- Write out editing specs to an agent
- Let it edit audio based on the specs without audible artifacts and too much back-and-forth prompting. The result should sound musically correct.
Things progress quickly nowadays, so maybe there are better tools out there already, but as someone new to this still-emerging field I feel it’s important to try things out myself, learn new things along the way (custom framework design, new programming languages, DSP theory, to name a few), and simply have fun.
Quick architecture overview
audio file
→ analyze
→ music map (JSON)
→ agent + skills
→ compose()
→ output audio| Stage | What it does |
|---|---|
| analyze * | beats (madmom), chords (Chord-CNN-LSTM), structure (SongFormer), key (essentia) — merged into one structural map |
| agent + skills | reads the map, reasons about the edit, writes Clip / XFade code |
| compose() | validate() first — checks segment labels exist, durations fit, and there are no abrupt hard cuts — then renders |
* Apple is already developing its own music-understanding framework — see the WWDC 2026 session (it goes further, with on-device instrument-activity detection — something recut’s own map doesn’t have yet, see Limitations).
Models: madmom (beats), Chord-CNN-LSTM (chords), SongFormer (structure), Essentia (key). Full license breakdown, including two non-permissive ones, in the repo’s README.
Results
summer-party, the track used below, has a plain 4-section structure — this is what
analyze sees before anything gets cut:
| Segment | Start | End |
|---|---|---|
| intro | 0:01 | 0:16 |
| verse | 0:16 | 0:32 |
| chorus | 0:32 | 0:47 |
| inst | 0:47 | 1:03 |
These four labels are exactly what show up as Clip("intro"), Clip("chorus"), etc. in the code below — the map turns raw timestamps into names you can actually write against.
Original
Untouched track — the structure analyze mapped out in the table above.
Code below is simplified for readability (no imports, no validate() call — already
covered above) — the full runnable version of each is in
examples/summer-party/README.md.
chorus-first — reorder, fade-out only
Straight reorder — chorus pulled to the front, fade-out only. No effects.
compose(
music_map, audio,
Clip("chorus"),
Clip("intro"),
Clip("verse"),
Clip("inst", bars=7),
Clip("inst", offset_bars=7, fx=[Fade(vol_start=1.0, vol_end=0.0)]),
)creative-edit — hand-tuned, multiple effects
Hand-tuned multi-effect edit — filter sweeps, a crossfade, and a reverb sweep layered manually.
compose(
music_map, audio,
Clip("inst", offset_bars=4, bars=2, fx=[
FilterSweep(filter_type="high", freq_start=400, freq_end=400),
Fade(vol_start=0.5, vol_end=1.0),
]),
Clip("inst", offset_bars=6, fx=[
FilterSweep(filter_type="high", freq_start=400, freq_end=400),
]),
Clip("intro", offset_bars=4),
Clip("chorus", bars=4),
XFade(ms=15),
Clip("inst", offset_bars=4, fx=[
FilterSweep(filter_type="high", freq_start=200, freq_end=200),
FilterSweep(filter_type="low", freq_start=3000, freq_end=3000),
ReverbSweep(wetness_start=0.0, wetness_end=0.6, reverb_type="room"),
]),
)ambient-soundscapes — generative soundscape
Generative soundscape — the instrumental loop resampled with delay and reverb.
compose(
music_map, audio,
Clip("inst", offset_bars=4, loop=8, fx=[
FilterSweep(filter_type="high", freq_start=200, freq_end=200),
FilterSweep(filter_type="low", freq_start=1400, freq_end=1400),
Delay(delay_seconds=0.7, feedback=0.9, mix=0.15),
Reverb(wetness=0.65, reverb_type="hall", room_size=0.97, damping=0.12),
]),
)Reprocessed through the same effect chain a second time for extra depth — the full two-stage code is in examples/summer-party/README.md.
Limitations
Some documented in code, some here:
- composable API design missing — e.g. an effect can’t target part of a clip yet, so a fade-out on a clip’s tail means writing two clips instead of one
- analysis pipeline — currently only available via Modal inference (setup not tested locally)
- skills and knowledge — musical judgment (cadence quality, hooks) is inferred by the agent, not looked up from verified data
Read all limitations within the repo: LIMITATIONS.md.
The project is still in its early experimental phase, but ready to be tested on more complex music and improved from there. Feel free to clone the repo and wire it up to your harness of choice (currently tested with Claude Code) — I have included a folder with example data, so you do not have to run your own Modal instance.