WinoBias
A benchmark dataset for evaluating gender bias in NLP models. It constructs sentence pairs containing gendered pronoun ambiguities to measure a model's tendency toward occupational gender stereotypes.
Timeline (last 90 days)
Mainstream bias benchmarks currently include datasets such as WinoBias, StereoSet, and BBQ, which measure a model's stereotyping tendencies along dimensions such as gender and race by constructing contrastive sentence pairs or multiple-choice questions.
A common limitation shared by mainstream bias benchmarks is that they measure a model's explicit choices in specific controlled contexts, rather than implicit biases that emerge in open-ended generation, long-text reasoning, or complex role-playing scenarios.
All Facts (2)
Mainstream bias benchmarks currently include datasets such as WinoBias, StereoSet, and BBQ, which measure a model's stereotyping tendencies along dimensions such as gender and race by constructing contrastive sentence pairs or multiple-choice questions.
50%UnverifiedA common limitation shared by mainstream bias benchmarks is that they measure a model's explicit choices in specific controlled contexts, rather than implicit biases that emerge in open-ended generation, long-text reasoning, or complex role-playing scenarios.
50%