Unverified50% confidenceBenchmarkExact time
Layla is extremely sensitive to the number of options and their positions; on 77 classification tasks its accuracy drops to 0.425, and shuffling the option order causes 20% of answers to flip.
1
Sources
50%
Confidence
Long-term
Relevance
10/1/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedLaya has an issue of high confidence when making errors, with instances of giving a confidence score of 0.952 despite a score of 0.65% similarUnverified困惑度是测试集上每个token平均负对数似然的指数，值越低代表模型对语料的预测越准确65% similarUnverified在歧义点的四类行为中，'先搜再问'成功率最高，平均通过率达93.4%，远高于'直接猜'的56.5%，'搜多次再猜'为51.5%63% similarUnverified当相关文档被放置在上下文中间位置时，模型正确回答率相比放在首尾位置下降高达20-30个百分点63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/964404API
curl https://kongchang.com/api/v1/knowledge/claims/964404MCP
get_claim(id=964404)