Llama-3.2-3B
A small-scale large language model released by Meta with 3 billion parameters, part of the Llama 3.2 series, used as the baseline model in data selection experiments.
Timeline (last 90 days)
The study used three models, Qwen2.5-1.5B, Qwen2.5-3B and Llama-3.2-3B, for experiments.
In LLM agent opinion simulations, agents' answers to complementary questions (rewriting a statement into negated form and replacing probability p with 1-p) fail to satisfy the basic requirement that they sum to 1, and this test fails broadly on the three small open-source models Qwen3-4B, Llama-3.2-3B and Gemma-3-4B.
A controlled benchmark containing 24 worlds (each with exactly two possible outcomes) reproduces the failure of the complementary consistency test on all three models (Qwen3-4B, Llama-3.2-3B, Gemma-3-4B).
In the final round, the averaged complementary answer method outperforms the better of the two original phrasings on only one of the three models.
On Llama-3.2-3B, fine-tuning a brand-new dense model with the 5% of data selected by CACTUS reached 49.44% accuracy on BIG-Bench Hard; selecting data directly with the dense model gave 49.54%; data selected by the ALPS method could only reach 47.22%.
Both the Qwen series models and Llama-3.2-3B can expose human-perceptible style axes through this method.
All Facts (6)
The study used three models, Qwen2.5-1.5B, Qwen2.5-3B and Llama-3.2-3B, for experiments.
50%UnverifiedIn LLM agent opinion simulations, agents' answers to complementary questions (rewriting a statement into negated form and replacing probability p with 1-p) fail to satisfy the basic requirement that they sum to 1, and this test fails broadly on the three small open-source models Qwen3-4B, Llama-3.2-3B and Gemma-3-4B.
50%UnverifiedA controlled benchmark containing 24 worlds (each with exactly two possible outcomes) reproduces the failure of the complementary consistency test on all three models (Qwen3-4B, Llama-3.2-3B, Gemma-3-4B).
50%UnverifiedIn the final round, the averaged complementary answer method outperforms the better of the two original phrasings on only one of the three models.
50%UnverifiedOn Llama-3.2-3B, fine-tuning a brand-new dense model with the 5% of data selected by CACTUS reached 49.44% accuracy on BIG-Bench Hard; selecting data directly with the dense model gave 49.54%; data selected by the ALPS method could only reach 47.22%.
50%UnverifiedBoth the Qwen series models and Llama-3.2-3B can expose human-perceptible style axes through this method.
50%