While generative AI has the potential to enhance the work of a chief executive officer, it can’t yet replace her entirely, says Oded Netzer, a professor at Columbia Business School.
Generative AI, a term used to describe products like chatbots and certain image-generating platforms, can be defined as a tool that uses patterns and rules of training data to generate something new. This could be a new image, text or video that’s created using the data entered into the system.
“Can AI replace executives? Not yet,” said Netzer. “What we do really well as humans is connecting the dots. It’s being able to synthesize information, bringing in information from different sources, and bring it into the context of what we are currently seeing.”
As an example, Netzer gives raw data of a company’s performance over the past two years to his MBA students and asks them to summarize and provide a synthesis of the data. Then he asks ChatGPT to do the same.
In the experiment, he finds that students and ChatGPT are similarly able to summarize the information in the chart. But when it comes to synthesis — which requires bringing in outside knowledge from different sources to provide a robust answer of what’s going on with the company, and figure out next steps — students are able to provide a much stronger response, whereas ChatGPT falls short of bringing in information from various sources and figuring out the big questions.
“When I provide it to students, they truly go into synthesis: So what? What does it mean for us? Now what are we going to do about it?” he said. “That’s kind of the CEO question, and that’s definitely in our domain.”
This, he says, is a noticeable limitation of the technology right now. He gives ChatGPT’s response a grade of 6 out of 10 — meaning that it could currently pass his class at a low grade, but it “definitely wouldn’t be a good student.” The demonstration is done using GPT 3.5, versus the current version, which is GPT-4.
The latest hype around generative AI kicked off in November with the public release of ChatGPT, which reached 100 million users in just two months. Since then, the product has been used by the public for tasks as varied as writing a work email, creating travel itineraries and planning a date.
But with its promise, there have also come a great number of challenges around privacy, security and the legality of it all. And new research shows the chatbot might actually be getting worse at basic math.
Still, despite its shortcomings, business executives have promised it will unlock great potential for their companies, and mention of AI in earnings calls has gone up to record highs. The surge in companies talking about AI reflects Wall Street’s optimism for the potential of generative-AI tools to boost efficiency across industries.
Some tech executives, like Sam Altman, who’s behind ChatGPT and its creator OpenAI, has warned about the tool’s potential harms, and has pushed for regulation. And even before the public launch of ChatGPT, Elon Musk warned about how AI poses a potential threat to humanity.
“We are [in the] very, very, very early stages” of this new revolution of generative AI, Netzer said. “Despite huge technological innovations over the last two, three decades, productivity did not increase. Productivity was actually pretty much plateaued. I think that tool is likely to break this plateau on productivity.”