Cross-Question Pooling
A statistical strategy that combines generation counts from different questions. By exploiting structural information across questions, it uses limited data more efficiently to estimate the model's answer distribution.
Timeline (last 90 days)
Mechanism two: in the intermediate regime, pooling the generation counts of different questions is more effective than estimating each answer distribution separately, and only logarithmic gains can be obtained from additional questions.
The statistical intuition of pooling: merging the generation counts of all questions to estimate shared weights is equivalent to using more data to estimate fewer parameters to reduce variance, but when the relative merits of models differ greatly across questions, forced sharing introduces bias.
All Facts (2)
Mechanism two: in the intermediate regime, pooling the generation counts of different questions is more effective than estimating each answer distribution separately, and only logarithmic gains can be obtained from additional questions.
50%UnverifiedThe statistical intuition of pooling: merging the generation counts of all questions to estimate shared weights is equivalent to using more data to estimate fewer parameters to reduce variance, but when the relative merits of models differ greatly across questions, forced sharing introduces bias.
50%