ARC
A AI alignment research organization founded by Paul Christiano, focused on ensuring that powerful AI systems remain aligned with human values. Its research emphasis includes scalable oversight and other foundational technical problems in AI safety.
Core Facts
Timeline (last 90 days)
ARC (Abstraction and Reasoning Corpus) is a general intelligence benchmark proposed by François Chollet in 2019, consisting of visual grid puzzles that require inducing transformation rules from examples and applying them to new inputs.
In the ARK AGI 3 report, OpenAI enabled two harness settings (preserving the model's reasoning process across steps, and summarizing history rather than truncating it), allowing the model to take more actions and pass more levels under the same number of output tokens.
For the same frontier model in the same puzzle game, the score rose from 13.3% to 38.3% solely due to differences in surrounding code and settings.
Standard downstream benchmarks include MMLU, HellaSwag, ARC and GSM8K, which respectively assess knowledge question answering, commonsense reasoning, scientific reasoning and mathematical ability.
CFS is known for its high-temperature superconducting magnet technology, and its SPARC and ARC roadmaps depend on breakthroughs in miniaturizing high-temperature superconducting magnets.
The Adaptive Replacement Cache (ARC) dynamically balances recently used and frequently used data through four linked lists (MRU, MFU and their corresponding ghost lists), unlike the traditional LRU.
ARC includes L2ARC (level-2 cache), which can extend the cache to high-speed devices such as SSDs.
ChromeOS runs Android apps through a containerized ARC environment and is gradually pushing apps to adapt to large screens.
ARC (Abstraction and Reasoning Corpus) was proposed by François Chollet to measure a system's generalization and reasoning performance when faced with entirely new tasks, rather than its memorization or pattern-matching capabilities.
Approaches to tackling the ARC benchmark broadly fall into three categories: program synthesis search, large model test-time reasoning sampling, and hybrid architectures combining symbolic reasoning with neural networks.
29 more timeline events
All Facts (20)
METR (Model Evaluation & Threat Research) is an independent third-party organization focused on safety evaluation of frontier AI systems.
90%UnverifiedZFS's ARC (Adaptive Replacement Cache) by default dynamically occupies a large proportion of available system memory and must be manually limited in memory-constrained environments.
60%UnverifiedARC (Abstraction and Reasoning Corpus) is a general intelligence benchmark proposed by François Chollet in 2019, consisting of visual grid puzzles that require inducing transformation rules from examples and applying them to new inputs.
50%UnverifiedFor the same frontier model in the same puzzle game, the score rose from 13.3% to 38.3% solely due to differences in surrounding code and settings.
50%UnverifiedIn the ARK AGI 3 report, OpenAI enabled two harness settings (preserving the model's reasoning process across steps, and summarizing history rather than truncating it), allowing the model to take more actions and pass more levels under the same number of output tokens.
50%UnverifiedStandard downstream benchmarks include MMLU, HellaSwag, ARC and GSM8K, which respectively assess knowledge question answering, commonsense reasoning, scientific reasoning and mathematical ability.
50%UnverifiedCFS is known for its high-temperature superconducting magnet technology, and its SPARC and ARC roadmaps depend on breakthroughs in miniaturizing high-temperature superconducting magnets.
50%UnverifiedARC includes L2ARC (level-2 cache), which can extend the cache to high-speed devices such as SSDs.
50%UnverifiedThe Adaptive Replacement Cache (ARC) dynamically balances recently used and frequently used data through four linked lists (MRU, MFU and their corresponding ghost lists), unlike the traditional LRU.
50%UnverifiedChromeOS runs Android apps through a containerized ARC environment and is gradually pushing apps to adapt to large screens.
50%UnverifiedApproaches to tackling the ARC benchmark broadly fall into three categories: program synthesis search, large model test-time reasoning sampling, and hybrid architectures combining symbolic reasoning with neural networks.
50%UnverifiedKaggle is the world's largest data science competition platform and has long hosted the ARC series of challenges.
50%UnverifiedARC (Abstraction and Reasoning Corpus) was proposed by François Chollet to measure a system's generalization and reasoning performance when faced with entirely new tasks, rather than its memorization or pattern-matching capabilities.
50%UnverifiedEach ARC problem presents several pairs of input-output grid examples (typically 3–5 pairs), requiring the model to infer a transformation rule from just those few samples and apply it to a new input.
50%UnverifiedHuman test subjects achieve an average accuracy of over 85% on ARC problems, while mainstream large language models have long hovered below 20%.
50%UnverifiedARC (AI2 Reasoning Challenge) is a scientific reasoning question-answering dataset released by the Allen Institute for AI in 2018, containing approximately 8,000 standardized science exam questions for US grades 3–9.
50%UnverifiedLRMs of comparable or even smaller scale have already saturated the ARC 1 benchmark, achieving accuracy close to 100%.
50%UnverifiedThe performance of base LLMs on ARC 1 has to date remained at approximately 10%–15%.
50%UnverifiedScaling the model size by approximately one hundred thousand times improved the base LLM's score on ARC from only 0% to approximately 10%.
50%UnverifiedTraditional knowledge-based benchmarks may mask the absence of fluid intelligence, while tasks such as ARC are better at exposing paradigm differences.
50%