Fine-tuning
Fine-tuning is a transfer learning technique in machine learning that involves further training and adjusting the parameters of a pre-trained large-scale model using labeled data for a specific task or domain, adapting it to the target scenario. Fine-tuning typically requires relatively little data and computational resources to significantly improve performance on specific tasks such as text classification, question answering, and image recognition, and is the mainstream model adaptation method in natural language processing and computer vision.
Core Facts
Modern large language models typically go through three training stages: pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF).
Fine-tuning refers to performing secondary training on a foundation model that has completed general-text pre-training, using domain-specific labeled data to adjust the model's parameters
当代前沿模型的训练方法包括RLHF（基于人类反馈的强化学习）及其后继者RLAIF和过程奖励模型（PRM）
Timeline (last 90 days)
To make a model reliably output text without the letter "e," prompt constraints at inference time alone are often not reliable enough; fine-tuning is a more direct means.
Skill.md differs from fine-tuning (which changes model weights) and prompt engineering (which relies on the context of a single conversation): it is a reusable asset independent of the model that can be loaded by an agent at runtime and followed step by step.
Fine-tuning to update knowledge is costly and time-consuming, and once knowledge is written into the parameters, updating it requires retraining.
Teacher Li Jin pointed out that many companies do not lack the conditions for fine-tuning but are unwilling to do it, the core reason being that the difference in effect before and after fine-tuning is hard to verify; sometimes the effect actually gets worse after fine-tuning, and hiring one person specifically to do fine-tuning can cost several hundred thousand a year.
In terms of technical difficulty, the technical threshold of fine-tuning is lower than that of distillation, and distillation is lower than that of pretraining.
In practice, a combined approach is common, using fine-tuning to teach the model the process and format for rule-based judgments, and retrieval to ensure the timeliness and traceability of specific provisions.
Fine-tuning suits high-frequency, stable rules that need to become a conditioned reflex; RAG suits regulatory provisions that change frequently, require citations, and are easy to audit manually.
Fine-tuning is better suited to stable, enumerable rules; for information that changes frequently or requires real-time verification, it often needs to be combined with retrieval-augmented generation (RAG) and other methods, otherwise the model tends to memorize outdated rules or produce answers that appear compliant but are actually wrong.
Compared with training from scratch, fine-tuning costs far less and can preserve the base model's general language capabilities.
The technology selection judgment process is: first prepare test data to verify feasibility; when basic capabilities are insufficient, prioritize RAG; when external systems need to be connected, use Function Calling; and only consider fine-tuning in scenarios such as improving stability, reducing inference costs, and private deployment.
40 more timeline events
All Facts (20)
Modern large language models typically go through three training stages: pre-training, supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF).
90%VerifiedFine-tuning refers to performing secondary training on a foundation model that has completed general-text pre-training, using domain-specific labeled data to adjust the model's parameters
80%Verified当代前沿模型的训练方法包括RLHF（基于人类反馈的强化学习）及其后继者RLAIF和过程奖励模型（PRM）
80%Verified大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律
80%VerifiedFine-tuning involves taking an existing large model and further training it with a specially prepared proprietary dataset, thereby changing the model's weight values
80%VerifiedA pre-trained model refers to a model that has been trained in advance on large-scale datasets, which users can directly fine-tune or call to complete specific tasks.
75%VerifiedAlignment training optimizes the model's statistical behavior in known scenarios rather than establishing formally verifiable security boundaries, and may fail when faced with rare instruction combinations or role-playing frameworks outside the training distribution.
75%VerifiedA well-aligned model will guide meta-prompts toward compliant creative expression, while an insufficiently aligned model may exhibit unexpected behavior.
75%VerifiedFine-tuning continues to update parameters on top of pre-trained weights through the backpropagation algorithm, adapting the model to a specific domain or task style.
75%VerifiedFine-tuning requires hundreds to thousands of high-quality training samples, hours of GPU compute time, and ongoing maintenance costs, and the entire process must be repeated whenever knowledge is updated.
70%VerifiedModels can still experience stitching-type hallucinations when integrating multiple retrieved results, so RAG is not the ultimate solution for eliminating hallucinations.
65%VerifiedCompared to model fine-tuning, the advantage of RAG is that the cost of updating knowledge is extremely low—it only requires replacing the document repository without retraining the model.
65%VerifiedFine-tuning burns knowledge into model weights, making knowledge updates costly as they require retraining; RAG stores knowledge in an external database, so updates only require modifying the database without changing the model.
65%VerifiedIn AI coding, overly specific instructions 'lock down' the model's solution space, whereas moderately abstract high-level instructions can instead activate broader reasoning pathways in the model.
65%UnverifiedThe observation noted that models with strong base capabilities (such as the official original) are not necessarily more impressive at literary creation than specially fine-tuned versions, while models focused on anthropomorphic chat instead have more of an advantage in emotional expression.
70%UnverifiedA study published on arXiv (arXiv:2609.21096) proposes distinguishing hallucinated responses from trustworthy responses by analyzing the topological structure of information flow patterns in attention graphs.
70%UnverifiedFine-tuning to update knowledge is costly and time-consuming, and once knowledge is written into the parameters, updating it requires retraining.
60%UnverifiedThe cost of a single model fine-tuning ranges from hundreds to tens of thousands of dollars.
60%UnverifiedResearchers have found that fine-tuning only a small number of model parameters is sufficient to remove safety alignment (de-alignment attacks), and that certain jailbreak prompts can transfer across different models.
60%UnverifiedTo make a model reliably output text without the letter "e," prompt constraints at inference time alone are often not reliable enough; fine-tuning is a more direct means.
50%