Researchers suggest OpenAI trained AI models on paywalled O’Reilly books

Concerns Raised Over OpenAI’s Use of Copyrighted Material

OpenAI is currently facing allegations from various sources regarding the training of its AI models on copyrighted material without obtaining proper permissions. A recent report from an AI oversight organization has intensified these claims, suggesting that OpenAI may have increasingly utilized non-public literary works that were not licensed to develop its advanced AI systems.

Understanding AI Models

AI models function as intricate prediction systems. They analyze a vast array of data, including literature, films, and television programming, to discern patterns and generate responses based on simple prompts. For instance, when an AI model composes an essay about a Greek play or creates visuals reminiscent of Studio Ghibli, it does so by referencing its extensive knowledge. However, it’s crucial to note that this process does not produce original ideas; rather, it approximates existing information.

As AI development progresses, several labs, including OpenAI, have started incorporating AI-generated data into their training regimens to complement their dwindling pool of real-world data, primarily drawn from public online sources. Nonetheless, few organizations have completely shifted to synthetic data alone due to concerns that such an approach could diminish the performance of their models.

Findings from the AI Disclosures Project

The report from the AI Disclosures Project, a nonprofit founded in 2024 by notable figures such as Tim O’Reilly and economist Ilan Strauss, points to OpenAI’s GPT-4o model as likely trained using paywalled content from O’Reilly Media, where O’Reilly serves as chief executive. Notably, there is no licensing arrangement between OpenAI and O’Reilly for this content.

According to the authors, GPT-4o displays a higher recognition of O’Reilly’s paywalled material when compared to its predecessor, GPT-3.5 Turbo, which demonstrates greater recognition of publicly available excerpts. This analysis raises significant questions about the data used in model training.

Methodology Behind the Findings

The methodology utilized, known as DE-COP, was first unveiled in a 2024 academic research paper. It aims to identify copyrighted content within the training datasets of language models by testing the models’ ability to differentiate between human-crafted texts and AI-generated paraphrases. A model’s success in this task may suggest its prior exposure to the text in question during training.

Researchers examined the knowledge of GPT-4o and GPT-3.5 Turbo concerning O’Reilly Media’s publications, using nearly 14,000 excerpts from 34 different titles to estimate the likelihood that each excerpt was included in the training data. The report concluded that GPT-4o demonstrated significantly more recognition of paywalled O’Reilly content than the older model.

The Implications of the Findings

While the findings suggest that GPT-4o has prior knowledge of many non-public O’Reilly books, the researchers stressed that this does not serve as definitive proof. They acknowledged the experimental method’s limitations and noted the possibility that some excerpts may have been sourced from user-generated content pasted into ChatGPT.

The study did not assess OpenAI’s latest models, such as GPT-4.5 and other reasoning variants, leaving open the possibility that these more recent models could have differing content training histories. Nonetheless, OpenAI has been clear about its pursuit of high-quality training materials, having even hired journalists to assist in refining model outputs. This is reflective of a broader trend in the AI industry to enlist specialists from various fields to enrich AI systems with expert knowledge.

While OpenAI does maintain some licensing agreements for specific training data, the nonprofit asserts that the company is currently facing numerous legal challenges regarding its approach to copyright and data practices. The findings from the O’Reilly-focused report provide a controversial perspective on OpenAI’s training methodologies.