Students aren’t the only ones in academia using AI for writing. According to recent research, there’s evidence that many new research papers are being written by large language models and that these AI models are not doing the best job when it comes writing to quality.
For the study, which was lead by Thomas Walther at the Utrecht University School of Economics, researchers analyzed 41,489 articles from 34 finance journals. The team found that “readability declined and the use of LLM-associated terms increased following the release of ChatGPT at the end of 2022.” They also saw that this usage was “more pronounced among authors from non-English-speaking countries and in lower-ranked journals.”
The paper, Certainly! Generative AI and its Impact on Academic Writing (in Finance), is not the only recent research to suggest that AI might be lending many academics too much of a helping hand. Another recent paper that looked at the impact of AI on scientific research more broadly also found that it had become a common tool to assist researchers with their writing.
I recently spoke with Walther about his research and its implications for teachers and their students.
What Inspired This Research Into AI Use By Academics
Walther explains that the inspiration for his team’s research came from encountering students who were submitting AI-generated work.
“We have an AI task force trying to figure out how we should deal with students using it,” he says. Soon after ChatGPT was released, Walther and others noticed that on the surface, many master’s thesis statements suddenly seemed to have gotten better, but upon further examination, that wasn’t the case.
“So it reads well. But it also is very shallow,” Walther says. “Especially a year ago, or two years ago, it always had the same words, kind of telltale signs of LLM use, like ‘intriguing’ and ‘intertwined,’ and there’s ‘certainly,’ and so on.”
He adds that many turn to AI writing tools because they think it improves their writing, “but if you look a little bit deeper, it does not at all.” He wanted to see if that would also hold true for researchers using AI to write.
“We thought, ‘let’s, let’s look at the readability, how readable papers are, actually.’ And whether the introduction of large language models affected that,” he says. “In theory, they should make writing better, but after the first couple of tests we ran, we found the complete contrary to that.”
What Researchers Learned
Walther and his colleagues saw that since ChatGPT was introduced, academic papers got worse from a writing standpoint.
“The readability of papers dropped massively, and that is mostly due to the use of more complex words, and we argue that this is unnecessary,” he says. “It doesn’t increase papers being more precise.”
The research also found that the impact of AI increased in less prestigious outlets. “This decrease of readability is mostly in lower- and mid-ranked journals, not in the top-ranked journals. There is no change at all in the top-ranked journals,” Walther says.
Additionally, the use of AI seemed to widen existing disparities in academic publishing. “This drop in readability is mostly with authors coming from non-English speaking countries, they actually suffer the most,” Walther says, adding that instead of narrowing the readability gap between native and non-native speakers as one might hope, AI actually widens it.
But the research also indicated that researchers were using AI regardless of their first language.
“We have evidence it’s also native [English] speakers using it,” Walther says. “But they don’t suffer the same consequences, the drop in readability. So our interpretation of this result is that both native and non-native speakers, writers, and authors are using it, but they’re using the output differently.”
He adds, “One is saying, ‘Okay, this sounds like good English,’ maybe not knowing better, and the other says, ‘Well, no, this is not correct, maybe I can improve this.'”
Does This Mean Academic and Scientific Journals Are Just AI Slop Now?
I posed this question to Walther, and the answer, is thankfully, no.
While his study did not look at the accuracy of papers, he notes that those examined were published papers and still appeared to be based upon original, not AI-generated, research. These also went through a peer review process.
“So we still have this peer review filter,” he says. He adds that given where the less readable papers were published, “the filter works probably better at the higher ranked the journals are.”
Various disciplines have preprint servers on which users can post working versions of their papers before publication. Walther suspects you might see more evidence of AI slop at this preprint stage. However, the study did try to assess what the impact was for academics when they turn to AI.
“What we find is that authors who use ChatGPT and the like, tend to publish in lower-ranked journals,” Walther says. This was true, even when the AI-using authors had previously published in higher-ranked journals before AI writing tools became more widely available.
In other words, as we educators are constantly telling our students, using AI might make things easier right now, but in the long run, there’s a good chance using it will be counterproductive.