A machine learning tool can easily spot when chemistry articles are written using the chatbot ChatGPT, according to a study published Nov. 6 in Cell Reports Physical Science. From a report: The specialized classifier, which outperformed two existing artificial intelligence (AI) detectors, could help academic publishers identify articles created by AI text generators. “Most of the field of text analytics wants a really general detector that will work on everything,” says co-author Heather Desaire, a chemist at the University of Kansas in Lawrence. But in creating a tool focused on a particular type of paper, “we were really looking for precision.”
Desaire and his colleagues first described their ChatGPT detector in June, when they applied it to Perspective articles in the journal Science. Using machine learning, the detector looks at 20 characteristics of writing style, including variation in sentence length and the frequency of certain words and punctuation marks, to determine whether an academic scientist or ChatGPT wrote a piece of text. The results show that “you could use a small set of features to get a high level of accuracy,” Desaire says. The results suggest that efforts to develop AI detectors could be boosted by tailoring software to specific types of writing, Desaire says. “If you can create something quickly and easily, then it’s not that difficult to create something for different fields.”
