This is a free excerpt from one of Shortform’s Articles. We give you all the important information you need to know about current events and more.
Don't miss out on the whole story. Sign up for a free trial here .
What is AI model collapse? What are the consequences of model collapse? Is it already happening?
Model collapse is the complete erosion of AI models that occurs when they’re trained on flawed content generated by other AI models. Model collapse can lead to nonsensical, inaccurate, and biased AI output. Experts say that diverse, human-curated data is key to stopping AI model collapse.
Keep reading to learn about AI model collapse, and for examples of it happening today.
AI Could Destroy Itself
AI “model collapse”—where AI models rapidly deteriorate due to being trained on flawed content generated by other AI models—poses serious risks. But can this runaway train be stopped?
Background
Artificial Intelligence (AI) models are computer programs that perform tasks such as language translation or image recognition by analyzing vast amounts of information, known as datasets. They identify patterns within this data to make predictions and enhance their performance and accuracy over time.
However, AI systems can experience “model collapse,” a problem where they’re trained on data produced by other AI models without human-created data. This leads to a rapid decline in output quality due to a feedback loop.
A recent study from the University of Cambridge and the University of Edinburgh reveals how each iteration of synthetic training data (artificially generated data) significantly amplifies errors: Mistakes replicate like a digital “genetic mutation,” escalating inaccuracies exponentially. The problem is exacerbated by the tendency of AI models to quickly forget their original training data. Consequently, AI systems can completely collapse after just seven to 10 generations, producing nothing but gibberish.
Model Collapse Implications
This significant breakdown in AI function isn’t just theoretical; it’s evident in real-world applications such as AI-generated news articles. NewsGuard, a company that assesses news platform credibility, has identified more than 500 news outlets that primarily rely on AI to generate news articles with minimal human oversight. This lack of meaningful human involvement in the process and recursive reliance on AI can lead to a cascade of compounded errors, resulting in news stories being consistently marred with inaccuracies.
A case in point is Microsoft’s MSN news portal, which relies heavily on AI to generate stories without human editorial supervision. In September, it published an article describing deceased NBA player Brandon Hunter as “useless at 42.”
While the presence of inappropriate or incorrect content in AI-generated news articles is concerning, experts say the potential amplification of harmful biases due to model collapse presents an even more serious risk.
Looking Ahead
Experts say human creativity is key to preventing collapse. Currently, engineers scrutinize data to prevent AI from self-training on its synthetic output.
Given the potential threats related to model collapse—including an emerging cybersecurity risk—experts advise AI companies and researchers to treat training data as a threat. They recommend that those working with the technology take proactive measures to prevent it from self-destruction, including:
- Preserve and periodically retrain AI models on “clean,” pre-AI data sets.
- Introduce and retrain models on new human-generated content.
- Explore protective measures, such as watermarking content.
These are key first steps for companies to maintain the integrity and accuracy of their AI systems.
Want to fast-track your learning? With Shortform, you’ll gain insights you won't find anywhere else .
Here's what you’ll get when you sign up for Shortform :
- Complicated ideas explained in simple and concise ways
- Smart analysis that connects what you’re reading to other key concepts
- Writing with zero fluff because we know how important your time is
