ChatGPT offers a further productivity boost to weaker writers

ChatGPT can help people do writing tasks faster and better, a new study¹ He says. The artificial intelligence (AI) tool is particularly useful for those with relatively weak writing skills, boosting their performance to levels closer to that of more skilled individuals.

The findings raise questions about whether ChatGPT and similar tools will replace human labor. But the findings also suggest possible social benefits, say the authors.

This could potentially mean that, in the long run, we could see reductions in inequality as performance is equalized across different skill groups, says Whitney Zhang, an economics doctoral candidate at the Massachusetts Institute of Technology (MIT) in Cambridge, and a co – author of the study.

Written assignment

When ChatGPT launched in November 2022, there was widespread speculation about how the tool that generates flowing text when prompted would affect the job market, says Zhang. So she and her colleague Shakked Noy, also an economics doctoral student at MIT, designed an experiment to understand the effect of chatbots on worker productivity. The results were published today in Science.

The researchers recruited 453 college-educated professionals, including marketers, grant writers and managers, and asked them to write press releases, brief reports, analysis plans and other texts.

How Nature readers use ChatGPT

After completing an initial task, about half of the participants were asked to register for ChatGPT and use it to generate a second piece of text if they found it useful. The group exposed to the chatbot finished the second activity significantly faster: an average of 16 minutes, compared to an average of 27 minutes for the group that didn’t have access.

The authors had experienced professionals rate the quality of the texts on a scale of 1 to 7. The increase in score from the first task to the second task was on average 18% higher for the group with chatbot access compared to the group without access.

The increase in quality was greatest for participants who received a low score on their first task. Their score increased by 12 points with access to the chatbot, while those who initially received high scores kept it after the introduction of ChatGPT. Time to complete the second activity decreased regardless of whether participants scored high or low on the first activity.

It almost has a democratizing effect, says economist Robert Seamans of New York University’s Stern School of Business, in New York City, who was not involved in the research. Less experienced workers are the ones who would benefit the most.

Bot-based workforce

That’s good news considering growing inequality in the job market, says Anton Korinek, an economist at the think tank The Brookings Institution, which is based in Charlottesville, Virginia. But there’s also a hint of bad news, he adds: ChatGPT could obviate the need for certain skills. All the white-collar workers who have great writing skills and also other analytical skills that language models can play their skills are suddenly devaluing.

Some of the study results seem to confirm this point. Participants using ChatGPT didn’t heavily edit the text it produced, and the small changes they introduced didn’t contribute to any improvement in their score.

Does this mean that AI tools could completely replace some workers? Most jobs involve a wide variety of tasks, and writing is a part of that. That part can be facilitated and augmented with language models, but all the other parts are still there and will still require humans, at least for now, Korinek says.

It’s hard to predict how this increased productivity will affect wages, Seamans says. But it’s really necessary for all workers to learn as much as possible about the technology, he says. AI will be ubiquitous, just like the internet is now ubiquitous. And the people who understand how to work with technology are going to be the ones to be well rewarded.