"Cut the dead air out of the interview, swap the sofa background for a rooftop at dusk, and export it for Resolve." That's one message. Claude reads the project, makes the cuts on the timeline, sends the background swap to a video model, drops the new shot back in place and writes the EDL.
The clip above is that kind of change: the background behind the sofa is replaced while the people, the camera move and the lighting stay put. This guide covers what it actually takes to edit video with Claude, and where it still needs you.
Why "just ask Claude" didn't work before
Chat Claude can write you an FFmpeg command or a Premiere tutorial. It can't see your footage, it can't hold a timeline, and it can't call a video model. Every edit came back to you as homework.
Three things have to be true for Claude to do the edit:
- It can look. A tool that returns frames or a contact sheet, so it can check what's in shot 4 before cutting it.
- It can act on a real timeline. Place, trim, move, layer and re-time clips in a project file, not in a text reply.
- It can generate and fix shots. Hand a shot to Seedance, Kling, Runway or an image model with references, then put the result back on the timeline.
Claude Code already knows how to use tools well. What it was missing was a video project to use them on.
How it works in Shoty
Shoty is a desktop video editor (Mac and Windows) with an agent chat built in. In the model picker you choose Claude · my subscription. Then:
- Shoty runs the unmodified
claudeCLI on your machine, signed in with your own Claude account. Shoty never touches the token. - For each turn, Shoty opens a localhost MCP server and exposes its editing tools: look at an image or frame, read and update the timeline, generate a shot, update project memory, export.
- Claude calls those tools. Every call shows up in the chat ("Used 2 tools · look_at_image · Update project memory") and every change lands on the timeline you're looking at.
The timeline itself is a text format (.shoty), so Claude reads and edits a structure it understands instead of guessing at pixels. You can always undo, and you choose how much it does on its own: suggest, preview, or auto.
Five things to ask Claude to do
Copy these into the agent chat as a starting point.
Watch the first 90 seconds of the interview and cut every pause longer than 1.2 seconds. Keep the room tone under the cuts.
Rough cuts are where Claude saves the most time. It checks frames before cutting, so it won't chop a reaction shot.
Replace the background in shot 3 with a rooftop at dusk. Keep the two people, their motion and the camera move exactly as they are.
"Change one thing, keep the rest" is the instruction that matters for VFX. Naming what must not move is what stops the model from repainting the whole shot.
Make three versions of the opening shot with Seedance and Kling, put them on V2 and V3 above the original so I can compare.
The agent can send the same brief to several models and line the results up. You pick; it cleans up the losers.
Add burned-in English subtitles from the dialogue, white, bottom center, max two lines.
Export this cut as an EDL for DaVinci Resolve.
The hand-off is what makes this usable in real post. Grade and mix stay in Resolve or Premiere. Shoty does the cut and the AI shots.
Where you still step in
- Taste calls. Claude can find every pause; deciding which beat lands is still yours. Grab the timeline and trim by hand whenever you want.
- Approving generations. Generated shots cost credits, so set the run mode to review before anything renders if you're on a budget.
- Long jobs. Claude can queue a whole batch of shots and keep going while you're away, but read the results before you ship.
Claude Code, Codex or Gemini?
The tools are the same for all three, because they all reach Shoty over the same MCP bridge. Use the plan you already pay for. If one model gets stuck on an edit, switch the picker and send the same message again. The project memory carries over.
Download the free Shoty editor and connect Claude from the model picker. For prompt ideas for the generated shots, browse the Seedance 2.0 prompt gallery.