Reinforcement Learning
Reinforcement Learning is a major branch of machine learning in which an agent learns an optimal decision-making policy through continuous interaction with an environment. At each state, the agent takes an action and adjusts its behavior based on reward or penalty signals from the environment, with the objective of maximizing long-term cumulative reward. It does not rely on labeled data, is applicable to sequential decision problems, and is widely used in game playing, robotics, autonomous driving, and resource scheduling.
Core Facts
Machine learning is typically divided into three major paradigms: supervised learning, unsupervised learning, and reinforcement learning.
DQN is an algorithm proposed by DeepMind in 2013 that for the first time combined deep neural networks with Q-learning reinforcement learning, enabling AI to learn to play 49 Atari games.
The pre-training process of large language models requires thousands of high-end GPUs running in parallel for months, with costs typically ranging from tens of millions to hundreds of millions of dollars.
Timeline (last 90 days)
Using reinforcement learning for quantum circuit generation is not an entirely new idea, but focusing on diversity rather than optimization is an entry point of practical value.
Adding diversity rewards to reinforcement learning (such as a penalty term based on circuit structure distance or entropy regularization) can encourage exploration of different structures.
Reinforcement learning (RL) is significantly higher in complexity and cost than supervised fine-tuning (SFT), but its ceiling is also higher.
The Noise Steering study proposes jointly controlling yaw angle and power setpoints via reinforcement learning, which can substantially reduce power loss while satisfying noise constraints.
With reinforcement learning using only a single example, the model's performance on math benchmarks can keep climbing and ultimately approach the effect of training with thousands of examples.
LoRA is essentially a form of regularization: the main weights are fixed and the update magnitude is small, which fits RL's need to limit policy drift while saving GPU and improving efficiency.
Ψ-0.6 acquires through reinforcement learning a scheduling policy that adaptively selects among different representations, and the model learns during training to judge which reasoning path suits the current task.
The idea of world models can be traced back to the field of reinforcement learning, where agents use world models to train through "imagination" to reduce the cost of interacting with the real environment.
During his time at Meta, Patel was mainly responsible for AI evaluation and reinforcement learning.
The OpenAI ES form is popular in reinforcement learning and black-box optimization because it is easy to parallelize, does not rely on backpropagation, and is somewhat robust to reward scale.
40 more timeline events
All Facts (20)
Machine learning is typically divided into three major paradigms: supervised learning, unsupervised learning, and reinforcement learning.
90%VerifiedDQN is an algorithm proposed by DeepMind in 2013 that for the first time combined deep neural networks with Q-learning reinforcement learning, enabling AI to learn to play 49 Atari games.
90%VerifiedThe pre-training process of large language models requires thousands of high-end GPUs running in parallel for months, with costs typically ranging from tens of millions to hundreds of millions of dollars.
90%Verified奖励函数设计不当会导致奖励欺骗（Reward Hacking）现象，即智能体找到在奖励函数上得分高但实际无意义甚至有害的行为路径
85%VerifiedThe concept of world models originates from cognitive science and reinforcement learning, strongly promoted by Yann LeCun and others, referring to an agent's internal representation of the external environment.
80%VerifiedReinforcement learning learns optimal policies through continuous interaction between an agent and its environment; its mathematical foundation is the Markov Decision Process (MDP), which consists of a state space, action space, transition function, reward function, and discount factor.
80%Verified大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律
80%VerifiedMachine learning is divided into three major categories: supervised learning, unsupervised learning, and semi-supervised learning.
80%VerifiedLearning-based methods have stronger performance when data is sufficient, but at the cost of robustness under domain shift.
75%VerifiedReward shaping refers to adding extra, denser guiding reward signals on top of the original sparse reward to help agents learn faster and more correctly.
75%VerifiedEach RL training step requires running inference (generating rollouts) and backpropagation (updating weights) simultaneously, making the compute requirements far higher than ordinary fine-tuning.
75%VerifiedThe full-stack GPU paradigm keeps environment state, dynamics simulation, policy network forward passes, and gradient computation entirely within video memory, eliminating the data transfer bottleneck between CPU and GPU.
75%VerifiedIn sparse-reward scenarios, agents often need techniques such as curriculum learning, reward shaping, or intrinsic motivation-based exploration to converge effectively.
75%VerifiedIn reinforcement learning training, the model's goal is to maximize the score given by the reward function.
75%VerifiedRewards in real-world tasks are often extremely sparse and delayed, causing long-horizon credit assignment problems that make it difficult for agents to determine which action led to the final outcome.
75%VerifiedWhen language models are repeatedly trained on data generated by themselves or similar models, systematic degradation occurs, with the diversity of model outputs decreasing with each generation and the tail information of the distribution gradually being lost.
75%VerifiedAI misalignment behavior is fundamentally a direct product of current mainstream training paradigms, particularly reinforcement learning.
75%VerifiedReinforcement learning training is the key stage in which a model transitions from general language capability to precise task execution, further optimizing reasoning, planning, and instruction-following through reinforcement learning after large-scale pre-training.
75%VerifiedDeepMind's AlphaGo learned Go strategies by playing millions of games against itself.
75%VerifiedA Markov Decision Process (MDP) is mathematically defined by a five-tuple (S, A, P, R, γ).
75%