How Does Eye Tracking Evaluate AI Video Realism? New Study Proposes GazeEval Method

Why We Need New Methods for Evaluating AI Videos

Current mainstream approaches to evaluating the quality of AI-generated videos still rely on explicit human feedback—ratings, rankings, or text reviews. Such signals tell us what judgments viewers ultimately made, but they cannot reveal the perceptual processes behind those judgments. In other words, we only know that "viewers felt this video wasn't realistic enough"—we don't know how their attention distribution while watching influenced that conclusion.

A new paper proposes using eye tracking as a complementary evaluation signal to fill this gap. The researchers designed two sets of Visual Turing Tests (VTT), recruiting 90 participants to watch videos produced by the latest generation models. Each participant watched each video twice before providing feedback. The two experiments ultimately collected 4,286 valid eye-tracking trials across 438 videos.

Visual Turing Test (VTT) is a variant of the Turing Test in the domain of visual content: human observers judge whether a video (or image) was generated by AI or actually filmed, thereby measuring how well a generative model can "pass as real." Unlike traditional evaluation relying on fixed metrics (such as FID or IS), VTT directly uses human perception as the criterion, making it closer to the subjective dimension of "realism." However, in a standard VTT, participants typically output only a binary judgment—the experiment records the "final answer" rather than the dynamic process by which the judgment formed. This study overlays eye tracking on the VTT framework precisely to capture attention allocation information at the moment of judgment while retaining human judgment as the gold standard.

Consistency Between Gaze Trajectories and Subjective Judgments

The core finding of the study is that gaze patterns provide evaluation signals that align with explicit human feedback. This means that when viewers consider a video "lifelike" or "obviously fake," their eye movement data already contains capturable clues.

The value of this conclusion lies in advancing the evaluation dimension from "conclusion" to "process." Explicit ratings are products at the end of the perceptual chain, whereas eye movement data records how perception itself unfolds—where viewers linger, which regions are repeatedly fixated, and how gaze migrates across the frame. For generated videos, these patterns may reveal where the model gives itself away in the details more effectively than final ratings.

It is important to note that the paper emphasizes eye movement signals as "complementary" rather than "substitutive." Explicit feedback remains valid; eye movement data provides another layer of information, and only by combining the two can we more completely characterize human perception of video realism.

Eye tracking uses infrared cameras or video image analysis to record eye movements in real time. Common metrics include fixations (brief pauses of gaze on a region), saccade paths (rapid jumps between fixation points), and regressions (gaze returning to previously viewed areas). In visual perception research, fixation duration and hotspot distribution are widely regarded as indirect measures of attention and cognitive load. Applying this approach to AI-generated video evaluation carries the assumption that when viewers repeatedly linger on or scan back and forth over a region, it often indicates anomalies, distortions, or suspicious details in that area—such as unnatural hand movements, texture flickering, or physical inconsistencies. These process signals are difficult to express directly through ratings, yet may precisely correspond to typical failure modes of generative models.

GazeEval: Predicting Perceived Realism from Eye Movement Data

Based on these findings, the researchers proposed GazeEval—the first computational method for predicting the perceived visual realism of AI-generated videos from eye-tracking data. The method achieves 71.3% accuracy on the prediction task.

The figure of 71.3% is not stunning in itself, but its significance lies in demonstrating that eye movement data does carry information about how humans evaluate videos. As the first method of its kind, it opens a new path: if eye movement signals can be utilized by computational models, it may be possible in the future to build video quality evaluation pipelines that are closer to human perception without relying on large-scale manual ratings every time.

The paper also releases the research data to support further research on eye tracking for AI-generated video evaluation.

Perceived visual realism—unlike objective pixel-level or physical-level realism—refers to the degree to which viewers subjectively consider an image "lifelike," and is a construct at the perceptual level. It is influenced by multiple factors including content semantics, motion coherence, lighting consistency, and viewers' prior knowledge, making it difficult to characterize with a single objective metric. GazeEval aims to predict precisely this subjective quantity: the input is an eye movement sequence (such as fixation coordinates, durations, and saccade paths), and the output is the tendency for the corresponding video to be perceived as real. The 71.3% accuracy should be understood in the context of such subjective prediction tasks—due to individual differences and the inherent noise of the task, the accuracy ceiling for perceptual prediction is typically lower than for objective classification tasks. Therefore, the significance of this result lies more in demonstrating that eye movement signals have computable and utilizable predictive value rather than pursuing absolute precision.

Potential Impact on the AI Video Evaluation Ecosystem

This work touches on a practical problem in the iteration of generative models: the evaluation bottleneck. The capabilities of video generation models are rapidly improving, but evaluation methods remain primarily manual ratings—costly, difficult to scale, and providing conclusions without reasons.

Eye tracking offers an evaluation layer between physiological signals and subjective judgments. It is more fine-grained than simple ratings, yet closer to the task objective than pure physiological measurement. If methods like GazeEval continue to mature, potential application scenarios include: helping locate specific regions in generated videos that trigger a "sense of unreality," providing perceptual-level feedback signals for model training, and adding an automated quality screening layer beyond manual evaluation.

Of course, eye movement data collection itself requires specialized equipment, which limits its large-scale application. But from a methodological perspective, this study reminds us that when evaluating AI-generated content, "where viewers looked" deserves as much attention as "what ratings they gave."

One-line Summary

A new study uses eye tracking to complement AI video evaluation and proposes GazeEval to predict perceived realism.

Paragraph Summary

The article introduces a study that uses eye tracking to complement AI-generated video evaluation. Existing evaluation mainly relies on explicit feedback such as human ratings and rankings, yielding only judgment conclusions without revealing the perceptual process. The researchers designed two sets of Visual Turing Tests, recruiting 90 participants and collecting 4,286 valid eye-tracking trials across 438 videos. They found that gaze patterns align with explicit human feedback, and eye movement data carries capturable clues about video realism evaluation. Based on this, the researchers proposed GazeEval, the first computational method for predicting perceived visual realism from eye movement data, achieving 71.3% prediction accuracy, and released the research data. The article notes that eye tracking complements rather than replaces explicit feedback, and could in the future help locate regions triggering a sense of unreality and provide perceptual-level feedback for model training, though equipment costs limit large-scale application.