Synthetic Data: Cure or Poison? Breaking Through the AI Training Data Drought

Internet Data Has Peaked: The 'Food Crisis' for Large Models

Have you ever wondered where the training data for today's large models comes from? Simply put, it's all the publicly available text that can be scraped from the entire internet—Wikipedia, books, news articles, academic papers, forum posts, code repositories, blog comments—all fed to models so they can learn human language.

Someone calculated that GPT-4-level models require training data on the order of tens of terabytes, which is roughly equivalent to consuming every piece of publicly available text you can imagine. But here's the problem: the internet is not infinite. All the text ever produced in human history has an upper limit, and that limit is not far away.

Ilya Sutskever, OpenAI's former Chief Scientist, said something at the NeurIPS conference in late 2024 that shook the entire industry:

We've reached peak data, and there won't be more data in the future. We have to work with what we have, because there's only one internet.

The subtext is clear: the path of improving models by crawling more data is reaching a dead end.

In the past, large model improvements largely relied on Scaling Laws: more model parameters + more training data = stronger model capabilities. This law held almost like an iron rule from 2018 to 2024.

Scaling Laws were first systematically proposed and quantified by OpenAI researchers in the 2020 paper Scaling Laws for Neural Language Models. The research found that language model performance follows a power-law relationship with model parameters, training data volume, and compute—when all three scale up simultaneously, model capabilities improve at a predictable rate. This discovery fundamentally changed the direction of AI research: rather than meticulously refining architectures, just stack more compute and data. DeepMind's Chinchilla paper in 2022 further refined this law, pointing out that previous large models were generally "data-underfed," and optimal training should maintain approximately a 20:1 ratio of data to parameters. However, Scaling Laws are fundamentally empirical curves—they describe patterns "under the premise of sufficient data." Once data supply becomes a bottleneck, the curve hits its ceiling early.

Around 2025, the problems began to surface:

• Insufficient quantity: Reddit posts have been scraped dry, public GitHub code repositories have been exhausted, and the growth rate of academic papers and Wikipedia can't keep pace with model training appetites
• Uneven quality: Low-quality content is proliferating while high-quality corpus growth remains limited
• Increasingly difficult legal access: Paywalled New York Times articles, medical privacy data—copyright and privacy regulations are tightening

Data exhaustion is a trinity of challenges, and this is precisely the backdrop for synthetic data's entrance.

What Is Synthetic Data: Training AI with AI-Generated Data

The basic idea behind synthetic data is straightforward: if real data isn't enough, can I create my own?

For example, suppose I need to train a model to respond more naturally to user follow-up questions in multi-turn conversations. I can have one model play the teacher and another play the student, letting them converse with each other—the teacher sets up a scenario, the student responds, the teacher scores the response and provides improvement suggestions. After hundreds of thousands of rounds, I have hundreds of thousands of high-quality conversational training samples.

Or consider training a model to identify rare lesions in medical imaging, where real images of rare diseases are scarce and involve patient privacy. In this case, generative models can synthesize new images that are statistically highly similar to real distributions, which can then be used to train models.

In fact, synthetic data is not exclusive to large language model training—it has mature applications across multiple AI subfields. In autonomous driving, Waymo and Tesla extensively use simulation engines to generate synthetic driving data for extreme weather and rare accident scenarios, since such data is extremely difficult to collect in the real world and poses safety risks. In medical AI, GANs (Generative Adversarial Networks) and diffusion models are used to synthesize medical images, and the U.S. FDA issued specific guidelines in 2023 recognizing the regulatory validity of synthetic medical data under certain conditions. The common lesson across these fields is that synthetic data delivers the most value in two dimensions: "supplementing long-tail scenarios" and "substituting in privacy-sensitive scenarios"—not as a primary data source.

Synthetic data isn't about randomly generating digital garbage—it's about accurately reproducing the statistical structure, distribution proportions, boundary conditions, and long-tail scenarios of real data.

Microsoft Research Asia developed a framework called SynthLM, specifically studying how to use pre-training corpora to generate synthetic data at scale. Their conclusion: synthetic data has predictability—you can use scaling laws to rationally select model size and synthetic data volume to maximize performance gains. In other words, synthetic data isn't used blindly—it can be mathematically modeled, predicted, and optimized.

The Biggest Pitfall: Model Collapse from Synthetic Data

But there's a very significant trap here. Between 2024 and 2026, extensive research uncovered a concerning phenomenon—feedback loops from synthetic data can cause model collapse.

What is model collapse? Suppose you have a base model M0. You use M0 to generate a batch of synthetic data to train the next-generation model M1. M1 then generates synthetic data to train M2, M2 generates data to train M3... By generation M3 or M4, you'll notice the model's generated content is degrading:

• Vocabulary becomes increasingly monotonous
• Sentence structures converge toward a few high-frequency patterns
• Long-tail content (those rare but important expressions, styles, and viewpoints) disappears
• The model begins outputting statistically "homogenized" content

Like a slowly fading photograph, getting blurrier and more uniform over time.

Model collapse has a clear explanation at the information theory level. Each time a model generates synthetic data, it's essentially performing a "lossy compression" of the original data distribution—the model can only learn statistically significant patterns from its training data, while the low-frequency but genuinely existing distribution tails get truncated. When this truncated distribution becomes the next generation's training data, the tails narrow further, forming a positive feedback degradation loop. From the perspective of KL divergence (Kullback-Leibler Divergence), the divergence between each generation of synthetic data and the true distribution increases monotonically, and this error accumulates rather than averages out across iterations.

A landmark study published in Nature (Shumailov et al., 2024) confirmed this: even tiny proportions of synthetic data contamination (as low as one-thousandth of training samples) can trigger measurable degradation in vocabulary, syntactic, and semantic diversity. That is, even if 99.9% of your training data is real and only 0.1% is synthetic, after multiple iterations