Test-Time Training
Test-Time Training (TTT) is a machine learning paradigm that permits lightweight parameter updates or adaptations during the model inference phase, dissolving the conventional boundary between training and inference. This allows a model to adaptively adjust based on the current input at test time, making it effective at addressing distribution shift problems.
Timeline (last 90 days)
In the absence of token-level measurement, certified scores demonstrate the effectiveness of single-problem TTT, but one cannot directly infer that its mechanism is homologous to RL's 1-3% token phenomenon.
The key difference between test-time training (TTT) and conventional training lies in when the update happens: the model temporarily adjusts its parameters for the current input during the inference stage, rather than being pre-trained on a fixed dataset.
Discussants say they have not yet seen work that performs token-level measurement of single-problem TTT runs, which is a concrete research gap: mapping small updates in parameter space back to the distribution of changes in token space.
The trend revealed by Latent Compaction is that static components in the inference process can dynamically adapt to the input through test-time training.
Latent Compaction treats the KV cache itself as a trainable neural network and optimizes it through test-time training.
Training refers to repeatedly iterating model parameters with large-scale data, an offline, long-cycle task; inference is the forward computation process by which the model responds to each user request after going live, a real-time, highly concurrent online task.
在搜索与训练耦合的循环中，仅保存最新权重无法完整恢复实验状态，因为模型检查点与搜索档案共同描述同一个演化中的实验，恢复时必须成对还原。
在 TTT 与搜索耦合的场景中，PUCT 搜索不是一次性完成的，而会在每个训练 step 后继续扩展，因此搜索树本身（哪些节点被访问、访问次数、价值估计等）构成必须被持久化的状态。
All Facts (9)
The key difference between test-time training (TTT) and conventional training lies in when the update happens: the model temporarily adjusts its parameters for the current input during the inference stage, rather than being pre-trained on a fixed dataset.
50%UnverifiedIn the absence of token-level measurement, certified scores demonstrate the effectiveness of single-problem TTT, but one cannot directly infer that its mechanism is homologous to RL's 1-3% token phenomenon.
50%UnverifiedDiscussants say they have not yet seen work that performs token-level measurement of single-problem TTT runs, which is a concrete research gap: mapping small updates in parameter space back to the distribution of changes in token space.
50%UnverifiedLatent Compaction treats the KV cache itself as a trainable neural network and optimizes it through test-time training.
50%UnverifiedThe trend revealed by Latent Compaction is that static components in the inference process can dynamically adapt to the input through test-time training.
50%UnverifiedTraining refers to repeatedly iterating model parameters with large-scale data, an offline, long-cycle task; inference is the forward computation process by which the model responds to each user request after going live, a real-time, highly concurrent online task.
50%Unverified在搜索与训练耦合的循环中，仅保存最新权重无法完整恢复实验状态，因为模型检查点与搜索档案共同描述同一个演化中的实验，恢复时必须成对还原。
50%Unverified在 TTT 与搜索耦合的场景中，PUCT 搜索不是一次性完成的，而会在每个训练 step 后继续扩展，因此搜索树本身（哪些节点被访问、访问次数、价值估计等）构成必须被持久化的状态。
50%Unverified测试时训练（TTT）在推理阶段利用当前输入的自监督信号对部分参数进行短暂局部梯度更新再执行预测
50%