Foundation Model

Training a frontier large model requires hundreds of millions of dollars in compute, data cleaning, and human labor.

Fine-tuning involves taking an existing large model and further training it with a specially prepared proprietary dataset, thereby changing the model's weight values

A Foundation Model refers to a large-scale pre-trained model that can serve as a general-purpose base to be fine-tuned for downstream tasks or directly invoked.

A pre-trained model refers to a model that has been trained in advance on large-scale datasets, which users can directly fine-tune or call to complete specific tasks.

Simply stacking parameters without sufficient training data results in a model that is large but not capable; a leap in parameters must be accompanied by a proportionally expanded training corpus and computing power to realize capability gains.

Agent 的核心架构通常由「感知-推理-行动」循环构成

Base models refer to general-purpose large language models such as GPT, Doubao, and DeepSeek, while post-trained models refer to models fine-tuned for specific industries based on base models.

Work that requires deep creativity and fundamental breakthroughs is harder to automate with AI, while standardized, highly repetitive application-layer engineering is more likely to be replaced by AI tools

Foundation model capabilities are spreading rapidly through APIs, and the AI features of office tools are easy to replicate.

The paper defines a work agent as a combination of "foundation model + runtime harness" that executes tasks in a stateful workspace and delivers verifiable results.

The post mentions that the lab does not have enough data to build foundation models, and the data scale is very small.

Upgrades to foundation models may render old customizations redundant, and promptly removing parts that are no longer needed keeps the agent simple and efficient.

The paper's controlled experiments cover 5 foundation models, 5 base classifiers and 3 scales, and the results show that the gains hold across different encoders, with larger improvements observed at smaller scales.

The method uses one Mamba model at each scale to encode features from two sources—a frozen foundation model and the lightweight FastVim—in linear time, after which a jointly trained classification head fuses the encoded representations into a single prediction.

Foundation models refer to models pretrained on large-scale general-purpose corpora that can be adapted to a variety of downstream tasks through fine-tuning or prompting, and are usually categorized by modality into text, image, audio and other types.

Most developers call foundation models through token-billed APIs rather than training models themselves.

Some optimists believe that the generalization capability of foundation models is enough to bypass some hardware bottlenecks, accelerating robot evolution through sim-to-real transfer.

Foundation models are a necessary condition for robot intelligence, but far from a sufficient one.

When foundational model capabilities become stable, the highlight of the launch event is no longer "what features were added" but "what the model can now do that it could not do before."

Pre-training a foundation model of the right size on proprietary data and then adapting it to high-value business problems can avoid dependence on external general-purpose large models and provides greater controllability in terms of cost and compliance.