The clip above moves from a clear sky to a storm over the same shot: same framing, same subject, only the weather changes. That is what AI relighting video is actually for, changing the light, the time of day or the weather on footage you already shot, without reshooting it.
Most "ai video lighting changer" results people find are built for a single photo. Point one at a moving shot and the light flickers or drifts because it was never tracked through the frame in the first place.
Why relighting a video is harder than relighting a photo
A still image only has to be lit convincingly once. A video has to hold that light steady, in the same direction and color, on every frame while the camera and subject keep moving:
- The light direction has to stay fixed relative to the subject, even as they turn or the camera pans. If the new light source doesn't move with the shot, the result reads as a filter laid over the footage, not light actually hitting the scene.
- Shadows and reflections have to follow the new light, not the old one. A shadow still pointing toward where the sun used to be is the fastest way to spot a relight that didn't track.
- Color temperature has to shift everywhere at once. Daylight on a face but studio-white light on the wall behind them breaks the illusion even if each looks fine on its own.
None of that is solved by adjusting exposure or color grading after the fact. Grading changes the whole frame uniformly; relighting changes how light falls across the scene, which is a different problem.
The workflow that holds up
1. Mask what the new light has to hit
Draw the mask around the subject or the part of the scene the new light needs to land on, on the clearest frame available. This is the same tracked-mask step used for any AI VFX job, not something specific to relighting.
2. Track it through the whole shot
Let the mask follow the subject and camera move, then scrub the result at full size. Check the frames where the subject turns, where the camera moves fastest, and anywhere they partly leave frame.
3. Set the new light, then check where it lands
Decide the target: a different time of day, an added weather condition, a shifted light direction. Then look at where the light actually falls on the subject against where it should fall given the new source, including the shadow it casts.
4. Review the full shot before you commit
Relighting that looks right on one frame can drift a few seconds later. Play the whole shot at 100%, not a shrunk preview, and watch for the light direction or color sliding across cuts within the same shot.
Asking an agent to do it
In Shoty you can run the steps above by hand, or hand the instruction to the agent in the chat:
Change this shot from clear daylight to an overcast storm. Keep the subject, their motion and the camera move exactly as they are, and match the shadow to the new light. Show me the result before replacing the original.
With the run mode set to review, the agent masks the scene, tracks it, generates the relit version and waits for your approval before anything overwrites the original clip.
Relight shot 5 from midday to dusk, and keep everyone in frame lit consistently with each other.
Naming "keep everyone lit consistently" matters when more than one subject is in frame; it's what stops the model from relighting one person and leaving another on the old light.
Where it still needs you
- Judging whether the light match is convincing is yours to call. A tool can get the direction and color close; whether it's close enough to ship is a review decision.
- Big changes (day to full night, adding heavy weather) mean more of the frame's lighting is invented, so check those harder than a small shift in light direction.
- Consistency across a scene with several shots still needs you to compare them side by side. Relighting one shot in isolation can look right and still not match its neighbors.
Relighting is one of the six AI VFX jobs available on a tracked shot in Shoty, alongside removing, replacing, extending and compositing; see AI VFX: remove, replace, extend, relight or composite a shot for the full set. Download the free Shoty editor to try a relight on your own footage. If the change is to the background instead of the light, see AI background replacement video.