Prompt Sensitivity
The phenomenon in which a large language model produces different outputs in response to variations in prompt wording. This can arise from deficiencies in model robustness or from flaws in evaluation methodology design, and is a core challenge in the field of model evaluation.
Core Facts
Timeline (last 90 days)
Large language models are highly sensitive to the wording, format, number of examples and even punctuation of prompts, and different phrasings of the same task may produce significant score fluctuations.
When a model gives answers of vastly different quality for semantically equivalent but differently phrased questions, it indicates that the model has overfitted to surface-level wording rather than truly mastering the underlying task.
All Facts (2)
When a model gives answers of vastly different quality for semantically equivalent but differently phrased questions, it indicates that the model has overfitted to surface-level wording rather than truly mastering the underlying task.
65%UnverifiedLarge language models are highly sensitive to the wording, format, number of examples and even punctuation of prompts, and different phrasings of the same task may produce significant score fluctuations.
50%