The Future Is Here
We may earn a commission from links on this page

Office Overachievers Won't Be Happy About ChatGPT, Study Says

Customer support agents given access to a generative AI chatbot were 14% more productive, but those gains were much higher for lower-performing workers.

Image for article titled Office Overachievers Won't Be Happy About ChatGPT, Study Says
Photo: William Thomas Cain (Getty Images)

Tech doomsayers and self-interested AI boosters have warned the rise of ChatGPT and other generative AI systems could wipe entry-level jobs or lower performers, but new research says experienced workers may actually have more to worry about. Customer support agents using a generative AI conversation assistant in a new study saw a 14% uptick in productivity compared to others who didn’t use the tool. Though the introduction of the AI assistant led to some improvements across the board, the research suggested those gains “accrue disproportionately to less-experienced and lower-skill workers.”

The new study comes from researchers out of Stanford University and Massachusetts Institute of Technology who found those results after surveying more than 5,000 customer support agents working at an unnamed Fortune 500 company. A portion of those workers were given access to a “recent version” of one of OpenAI’s GPT large language models, though it’s unclear which exact model they used. The workers’ productivity in this case was determined by how quickly they were able to solve customer issues and the overall amount of customer cases they resolved per hour.

Advertisement

The company’s least experienced and least skilled workers, according to the research, saw the biggest gains from the AI tools and were able to complete work 35% faster with the tool’s assistance. Customer service agents with just two months of experience who used the AI were able to perform just as efficiently as other more experienced agents with six months of tenure who didn’t use the AI assistant. Highly skilled workers, in this example at least, may initially gain less from using the AI tools because its recommendations are essentially tips and tricks they’ve already learned working on the job. Lower-skilled or newer workers, on the other hand, can use the tool to leap from some of those earlier steps and quickly catch up to their more experienced peers.

Advertisement

Previous studies have shown AI models’ ability to (barely) pass major law and medical licensing exams, but this research marks the first of its kind looking at generative AI’s impact on a living workforce. The researchers say they opted to survey customer service agents because that industry has one of the highest rates of AI adoption so far.

Advertisement

“Our overall findings demonstrate that generative AI working alongside humans can have a significant positive impact on the productivity and retention of individual workers,” the researchers wrote.

ChatGPT levels the playing field 

The 5,179 customer support agents involved in the survey were mostly located in the Philippines, with some others in the US. They all work for an enterprise software company where they regularly have to rely on a combination of product expertise, problem-solving skills, and an ability to deal with pissed-off customers to complete their jobs. The top-performing workers at the company, according to the study, typically reached solutions twice as quickly as average workers before AI was introduced.

Advertisement

Those disparities start shrinking once the AI assistant gets involved, in part because the AI model itself is trained on a dataset of successful customer service interactions. In other words, the AI can synthesize successful conversations from the more productive employees and then use that to improve the performance of others. The top performing workers might not have seen much meaningful benefit after using the AI, but their expertise funneled through an AI may help other workers catch up to their level. That, in turn, can improve the company’s overall productivity.

“AI recommendations can be thought of as expanding the marginal productivity of high-skill workers by encoding their conversational patterns and disseminating them to other workers,” the study says. “In our setting, high-skill workers are not compensated for these contributions.”

Advertisement

The researchers argue that disparity in who benefits from AI may actually leave higher-performing employees worse off than they were prior to the introduction of the AI. Future organizations could someday pay those workers even higher wages since their successful performance could be used by an AI to increase the productivity of entire swaths of the company, the researchers wrote.

“Our findings raise questions about whether and how workers should be compensated for the data that they provide to AI systems,” the study notes. “High-skill workers, in particular, play an important role in model development but see smaller direct benefits in terms of improving their own productivity.”

Advertisement

Though many worried analysts and experts have predicted AI could decimate wide swaths of underperforming employees and even lead to a lowering of wages, the new research presents a slightly different reality. Here, the real losers of AI assistants in the corporate world aren’t new or underperforming employees, but rather their more senior managers. Those uncomfortable findings may not sit well with some in the managerial class who’ve been quick to advocate for ever more AI tools in the workplace in the name of efficiency.

Want to know more about AI, chatbots, and the future of machine learning? Check out our full coverage of artificial intelligence, or browse our guides to The Best Free AI Art Generators and Everything We Know About OpenAI’s ChatGPT.