MAGIC: Using the Gini Coefficient to Detect Hallucinations in Medical Large Multimodal Models

MAGIC quantifies uneven multimodal attention with the Gini coefficient to detect hallucinations in medical LMMs.
Medical LMMs can produce "confident yet clinically unverifiable" hallucinations, a key risk in high-stakes care. MAGIC reframes detection as uneven cross-modal attention: it computes Gini coefficients of visual vs. textual attention across layers and heads, then trains a logistic regression classifier. Validated on two medical LMMs and three imaging modalities, it matches or beats baselines, uses only internal signals, and needs retraining per model/task.
Medical large multimodal models (LMMs) show promise in clinical decision support, but their tendency to generate "confident yet clinically unverifiable" hallucinated content remains a critical risk in high-stakes medical scenarios. Existing hallucination detection methods are mostly designed for single modalities and struggle to capture the dynamic interaction between vision and language in medical imaging. A study from rss proposes MAGIC (Modality-Aware Gini Inspection for Clinical Hallucination Detection in LMMs), which reframes hallucination detection as the problem of uneven cross-modal attention weight distribution when the model generates answer tokens, and uses the Gini coefficient to quantify attention concentration, training a logistic regression classifier for detection.

A New Perspective on Hallucination Detection: Uneven Attention Distribution
The research team redefines hallucination detection as: when generating answer tokens, the model exhibits an imbalanced distribution of attention weights across visual and textual modalities. Specifically, in hallucinated responses, the attention mass between visual and textual tokens undergoes significant redistribution. This perspective breaks away from the limitations of traditional single-modality detection and directly targets the internal mechanisms of multimodal models.
To quantify this distribution, the authors adopt the Gini coefficient—originally used to measure income inequality—to measure modality-specific attention concentration across different layers and attention heads. A higher Gini coefficient indicates that attention is more concentrated on a particular modality. By comparing hallucinated and non-hallucinated responses, the study finds statistically significant differences in attention distribution between the two.
Attention Weights and Multi-Head Attention Mechanism: In Transformer-based multimodal large models, each generation step computes attention weights of the current output token over all tokens in the input sequence through the Multi-Head Attention mechanism. These weights reflect which information the model "focuses on" when making decisions. In medical LMMs, the input typically contains visual tokens (e.g., patch embeddings of medical images) and textual tokens (e.g., clinical questions or reports), and the model must dynamically integrate information from both modalities. The distribution of attention weights—i.e., how much attention the model allocates to visual versus textual information—directly affects the reliability of the generated content. When the model over-relies on one modality and ignores the other, it may produce "confident hallucinations" that contradict the imaging evidence. Understanding this mechanism is a prerequisite for grasping the MAGIC method.
Method Core: Gini Coefficient and Logistic Regression
The core pipeline of MAGIC consists of two steps: first, compute the Gini coefficient of attention weights over visual tokens and textual tokens when the model generates each answer token, capturing modality-specific concentration features across layers and heads. Second, feed these features into a logistic regression classifier for training to distinguish hallucinated from non-hallucinated outputs.
The study evaluates the method on two medical LMMs and three medical imaging modalities, and compares it with contemporary hallucination detection baselines. Results show that MAGIC achieves competitive or superior performance. This validates the effectiveness of attention distribution features as a hallucination detection signal.
Definition of the Gini Coefficient and Measurement of Attention Concentration: The Gini coefficient is an indicator in economics that measures the degree of inequality in income or wealth distribution, ranging from 0 to 1, where 0 indicates perfect equality and 1 indicates perfect inequality. In MAGIC, researchers treat the attention weights of a given layer and attention head over visual tokens and textual tokens as the "distribution objects," and compute their Gini coefficient to quantify whether attention is concentrated toward a particular modality. Specifically, for each answer token generation, attention weights across all layers and heads are collected, and the Gini coefficients of the visual token group and the textual token group are computed separately, yielding modality-specific concentration features. A higher Gini coefficient indicates that attention is more concentrated on a few tokens or a particular modality. This measurement does not rely on external annotations and is extracted directly from the model's internal signals, offering interpretability and low overhead.
Experiments and Results: Cross-Model, Cross-Modality Validation
The experiments cover two medical LMMs and three medical imaging modalities, ensuring the generalizability of the conclusions. Compared with existing baseline methods, MAGIC performs comparably or better on multiple evaluation metrics, demonstrating that modality-aware attention features based on the Gini coefficient can effectively identify clinical hallucinations.
You may not have noticed that this method does not rely on external knowledge bases or additional annotations, using only the model's internal attention signals, and therefore has low deployment cost. However, the study also notes that hallucination detection itself remains an open problem, and MAGIC's classifier needs to be trained for different models and tasks.
Logistic Regression Classifier and Feature Engineering: Logistic regression is a linear classification model that maps a linear combination of input features to a probability via the sigmoid function, commonly used for binary classification tasks. MAGIC takes the computed Gini coefficient features (modality-specific attention concentration across layers and heads) as input and trains a logistic regression classifier to distinguish hallucinated from non-hallucinated outputs. Logistic regression was chosen over deep classifiers for two reasons: on one hand, the feature dimensionality is relatively low and has clear statistical meaning, so a linear model is less prone to overfitting; on the other hand, for interpretability—the classifier's weights can directly reflect which layers or heads' attention distributions contribute most to hallucination detection. However, this simple feature engineering also brings limitations: the classifier needs to be retrained for different models and tasks, and may fail to capture more complex nonlinear patterns.
Significance and Limitations
MAGIC provides an interpretable new approach to hallucination detection in medical multimodal models: through the Gini coefficient of attention distribution, it intuitively reflects whether the model is "overconfident" in one modality while ignoring the other. This has reference value for the safe deployment of clinical decision support systems.
However, the method currently relies on logistic regression with relatively simple feature engineering; future work could explore more complex classifiers or end-to-end detection schemes. In addition, the study validates only on two LMMs and three imaging modalities, and broader clinical scenarios still require further testing.
Overall, MAGIC introduces the Gini coefficient from economics into multimodal hallucination detection, revealing the statistical association between attention distribution and hallucinations, and providing a new tool for reliability assessment of medical AI.
One-line Summary
MAGIC uses the Gini coefficient to quantify uneven multimodal attention distribution for hallucination detection in medical LMMs.
Paragraph Summary
Medical large multimodal models (LMMs) hold great potential in clinical decision support, but their "confident yet clinically unverifiable" hallucinated content poses a critical risk in high-stakes scenarios. Existing hallucination detection methods are mostly designed for single modalities and struggle to capture the dynamic interaction between vision and language in medical imaging. To address this, the study proposes MAGIC, which reframes hallucination detection as the problem of uneven cross-modal attention weight distribution when the model generates answer tokens, uses the Gini coefficient to quantify modality-specific attention concentration across layers and attention heads, and trains a logistic regression classifier for detection. Experiments on two medical LMMs and three medical imaging modalities show that MAGIC achieves competitive or superior performance. The method uses only the model's internal attention signals without external knowledge bases, offering low deployment cost, but the classifier must be trained for different models and tasks.
Related articles

Programmers in the AI Era: How Deep Should You Learn Legacy Tech?
How should programmers transform under AI's impact? How deep should legacy tech be learned? Drawing on a Bilibili creator's share, this breaks down the levels of embracing AI, the shift from process-driven to project-driven learning, and how lower technical barriers affect junior and mid-level roles.

OpenAI Releases GPT-6.2 Dual Models Astra and Sol, API Price Cut by 20%
OpenAI releases GPT-6.2 Astra and Sol, with Astra's API price cut 20% to $8/$40 per million tokens and ranking second on the Artificial Analysis Intelligence Index. This article analyzes the dual-model strategy, price-cut impact, and developer selection advice.

Perplexity Computer Launches Multiagent: 16 AI Agents Cross-Validate Each Other
Perplexity launches Multiagent mode for Computer: a lead agent orchestrates up to 16 sub-agents on Claude, GPT, and Gemini, sharing findings and cross-validating work with effort above Ultra.