A rooftop at night, plain sky above it. A beat later, fireworks are bursting over the building, their light landing on the rooftop and the people standing on it. The camera move, the building and the crowd are the original plate, untouched. Only the sky changed. That's the clip above, and it's what AI compositing video actually does: add something that wasn't filmed there, in a way that reads like it was.
Why a dropped-in effect usually looks dropped in
Removing something from a shot only has to leave a convincing gap. Compositing a new element in has to invent something that belongs, and three things give it away when it doesn't:
- Light that doesn't match. The new element glows, flickers or casts color that has no source in the plate, or the plate's own light doesn't fall on anything new.
- No shadow, no reflection. Fire, smoke and fireworks all throw light onto what's near them. If nothing below the new element picks it up, the eye catches it instantly.
- Timing against the camera move. A static overlay on a shot with a moving camera slides across the frame instead of sitting in the scene.
A convincing composite has to get all three right on every frame, not just look plausible on one still.
A workflow that holds up
1. Mask and track the plate first
Decide where the new element lives in the frame, then track that region across the shot the same way you would for a removal or a roto job. The mask is what keeps the effect anchored to the scene instead of floating on top of it.
2. Match the light before you add anything
Look at where the existing light in the plate comes from, and build the new element to cast light the same direction. If it's a fire or fireworks burst, that means a corresponding glow on the surfaces and faces nearby. Skipping this step is the single biggest reason a composite looks pasted on.
3. Keep the composite on its own layer
Generate the new element so it sits above the original plate instead of baking directly into it. That way you can nudge the timing, adjust the blend, or pull it entirely without regenerating the shot underneath it.
4. Review at full size, on the frames that move fastest
Play the result at 100% through the frames where the camera or the new element itself moves quickest. Drift, flicker and timing mismatches show up there first, well before they're visible in a quick scrub.
Asking an agent to do it
Add a fireworks burst over the rooftop in shot 6. Match the glow to the building lights and the people below, keep the crowd and the camera move exactly as they are, and put the result on a layer above the original so I can compare.
With the run mode set to review, the agent masks the region, generates the composite and waits for your approval before anything replaces the original. You check the light match and the timing before it ships.
Composite drifting smoke over the chimney in shot 2, keep the building and the rest of the scene untouched, and show me the result at full size before replacing it.
Naming exactly what must not change, the building, the camera move, everyone already in frame, is what stops the generation from redoing more of the shot than you asked for.
Where it still needs a human
- Interactive light is the hardest case. A fireworks burst lighting a crowd's faces, or a fire casting flicker onto nearby skin, is the detail that sells the shot. Check it closely; it's also the first thing a generative pass tends to skip.
- Choreography is a judgment call. Where in the shot the effect peaks, how long it lingers, whether it should sync to a cut, those are creative decisions a tool won't make for you.
- Scale and distance. An element that's the right size up close can look wrong once you check it against the rest of the frame. Compare it against something in the plate you already trust the scale of.
Download the free Shoty editor to try compositing on your own footage. For the other five AI VFX jobs, see the AI VFX overview, or AI relighting video for matching light after the fact.