This paper discusses the impact of large language models like GPT-2 and GPT-3.5 on online text and images, as well as their potential for causing irreversible defects in resulting models due to Model Collapse. The authors highlight the importance of genuine human interactions with systems to counteract the effects of model-generated content. The research sheds light on the future implications of LLMs contributing much of the language found online, emphasizing the need to take the issue of Model Collapse seriously. This paper provides theoretical insight into the phenomenon and its prevalence among various generative models, urging caution in using data scraped from the web.
https://arxiv.org/abs/2305.17493