A sparse crowd in a shot grows into a full one, same framing, same lighting, same camera hold. That kind of change doesn't come from pasting a clip into a Gemini chat window. It's what Gemini video editing looks like once Gemini, through its CLI, is connected to an actual editor.
This guide covers what that connection takes, how it works in Shoty, and where you still decide.
Why chat Gemini can't do this
Ask Gemini in a browser tab to "add more people to this crowd shot" and you'll get a description of how you might do it, maybe a tool suggestion. You won't get the edit, because three things are missing:
- A way to look at the footage. A tool that returns a frame or a contact sheet, so the model checks what's actually in the shot before touching it.
- A real timeline to act on. Placing, trimming and layering clips in a project file, not describing the change in a text reply.
- A way to generate and place a shot. Sending a frame to a video model with references, then dropping the result back where it belongs.
Gemini's CLI is a capable agent on its own. What it's missing in a browser tab is a video project with tools attached to it.
How it works in Shoty
Shoty is a desktop video editor (Mac and Windows) with an agent chat built in. Pick Gemini in the model picker and:
- Shoty runs Gemini's own CLI on your machine, signed in with your own account. Shoty never touches the token.
- For each turn, Shoty opens a localhost MCP server and exposes its editing tools: look at an image or frame, read and update the timeline, generate a shot, update project memory, export.
- Gemini calls those tools. Every call shows up in the chat and every change lands on the timeline you're looking at.
The timeline is a text format (.shoty), so the agent reads and edits a structure instead of guessing at pixels. Undo is always available, and you choose how much runs on its own: suggest, preview, or auto.
A workflow to try
1. Start with one kind of task
Ask for the rough cut first, review it, then ask for the VFX pass. You can put "cut it, add subtitles, expand the crowd" in one message, but checking each step before the next keeps mistakes small.
2. Name what must not change
Expand the crowd in shot 4 so the street looks fully packed. Keep the people already in frame, the lighting and the camera move exactly as they are. Put the result on a track above the original so I can compare.
Naming what has to stay the same is what stops the model from repainting the whole frame.
3. Let it generate, then review
Try this shot with Seedance and with Kling, both on V2 and V3 above the original.
Gemini's CLI can send the same brief to more than one model and line up the results. You pick the one that works; it clears the rest.
4. Hand off the cut
Export this sequence as an EDL for DaVinci Resolve.
The handoff is what makes this usable in real post. Grade and mix stay in Resolve or Premiere; Shoty does the cut and the AI shots.
Where you still step in
- Taste calls. Gemini can expand a crowd or tighten a cut; whether it fits the story is still yours.
- Approving generations. Generated shots use credits, so set the run mode to review before anything renders if you're watching a budget.
- Long jobs. Gemini's CLI can queue several shots and keep going while you're away, but check the results before you ship.
Gemini, Claude or ChatGPT?
All three reach Shoty over the same MCP bridge, so the tool set is identical. Use the plan you already pay for, and your own account's limits and policies apply. If Gemini gets stuck on a shot, switch the picker to Claude or ChatGPT and send the same message again; project memory carries over.
Download the free Shoty editor and connect Gemini from the model picker. For the Claude side of this same setup, see Claude video editing, and for how the tool bridge itself works, see MCP video editing.