Deepfake Forensics Explained: How Video AI Detection Actually Works

Detecting deepfake video is substantially more complex than detecting AI-generated images or audio, because video adds a temporal dimension — not only must each frame be analyzed individually, but the relationships between frames must be evaluated for consistency. A single manipulated frame in an otherwise authentic video leaves very different traces than a fully synthetic clip. Understanding what forensic video analysis actually does illuminates both its capabilities and its limitations.

The Two Categories of Fake Video

Before diving into detection methods, it's worth distinguishing the two main categories of manipulated video. The first is face replacement or face swap — taking genuine video footage and replacing one person's face with another's. This was the original form of deepfake and remains common. The second is full synthesis — generating video content entirely from scratch using models like Sora, Kling, or Runway, with no original footage as a base. These two categories leave different forensic traces and require somewhat different detection approaches.

Frame-Level Analysis: GAN Artifacts

Generative Adversarial Networks (GANs), which underpin many deepfake face-swap techniques, produce characteristic artifacts at the frame level. The GAN's discriminator is trained to prevent the generator from producing unrealistic output, but the training process creates a kind of equilibrium that leaves specific fingerprints. These include: structured noise in regions that should be smooth (particularly in uniform-color areas like sky or skin); characteristic patterns in how high-frequency detail is rendered that differ from camera optics; and specific color distribution anomalies in the face region that result from the compositing process.

Temporal Coherence: The Video-Specific Tell

The most powerful forensic signal in video is temporal inconsistency — places where the content doesn't flow naturally from one frame to the next. Real video footage is shot at a consistent frame rate, with consistent motion blur, consistent sensor noise, and consistent lighting changes. When a face is replaced frame-by-frame by a neural network, the replacement introduces subtle inconsistencies that accumulate over time. The synthetic face may exhibit slightly different noise characteristics than the surrounding video, the lighting may not update quite correctly between frames, and the boundaries between the synthetic face and the real background may flicker in ways that are subtle but statistically anomalous.

Eye Movement and Blink Analysis

Human eye movement has specific, predictable patterns. Saccades (rapid eye movements) follow characteristic trajectories and durations. Microsaccades (tiny involuntary eye movements) are present in real footage at a specific rate. Blink duration and frequency are irregular but within a normal range. Many deepfake models struggle to synthesize convincing eye movement, either producing eyes that are too still, that follow unnatural trajectories during saccades, or that blink at suspiciously regular intervals. Frame-by-frame analysis of eye behavior is one of the more discriminating tests for face-swap deepfakes.

Lighting Consistency Across the Scene

Lighting in real video changes continuously and consistently across the entire frame. If a light source moves, brightens, or changes color temperature, every surface in the scene is affected according to its geometry and reflectance. Deepfake face replacements sometimes fail to update the synthetic face's lighting correctly relative to the scene, producing faces that seem to exist in subtly different lighting conditions than the surrounding environment. This is most visible in scenes with dynamic lighting — concerts, moving vehicles, flickering screens — where the lighting changes rapidly and the deepfake model can't keep pace.

Audio-Visual Synchronization

When deepfake video includes speech, the synchronization between lip movement and audio is an important forensic signal. Real speech involves precise coordination between breath, vocal fold vibration, and articulation. Deepfake synthesis must either generate both audio and video simultaneously or align separately generated components — both approaches can produce subtle desynchronization. Forensic analysis measures the millisecond-level timing relationships between audio events (particularly stop consonants like "p," "b," "t," and "k" that produce acoustic bursts) and corresponding visual articulation events.

Compression Artifact Analysis

Video compression (H.264, H.265, VP9) works by storing differences between frames rather than complete frame data. The compression algorithm makes predictions about how pixels will change from frame to frame and stores only the prediction errors. When a face has been synthetically replaced, the region around the replacement boundary behaves differently under compression than the surrounding original footage — the compression prediction errors are larger and more structured in the deepfaked region than in the original video. This compression-domain analysis can detect manipulations even when the visual quality is high enough to fool human observers.

Fully Synthetic Video

Detecting fully synthetic video from tools like Sora or Kling is in some ways easier and in other ways harder than detecting face swaps. Fully synthetic video doesn't have the inconsistencies introduced by compositing real and synthetic content, but it does exhibit the characteristic generation artifacts of diffusion-based video models: slightly unusual object physics (water, fire, and cloth move in ways that are approximately correct but not physically precise), background elements that don't maintain continuity between cuts, and subtle but pervasive texture characteristics that are specific to each generation system's training.

Practical Implications

For anyone evaluating video evidence — journalists, legal professionals, content moderators, or security researchers — the key practical point is that no single forensic signal is sufficient by itself. The most reliable detection combines multiple independent analysis methods: frame-level artifact analysis, temporal coherence scoring, eye movement analysis, audio-visual sync measurement, and compression artifact examination. SUS IT's video analysis pipeline applies this multi-dimensional approach, examining both individual frames and temporal patterns across the full duration of the clip. The result is a confidence-weighted verdict that reflects the convergence of multiple independent signals rather than any single test.