It’s been called the most important chart in tech. But what does it mean?
The two charts simply show the performance of OpenAI’s o1 model on the American Invitational Mathematics Examination (AIME) benchmark. As you move to the right, the amount of computation used goes up, but so does the performance. The left shows how performance increases as the model is trained more in advance. The right hand shows how performance increases as the model is given more time to come up with an answer.
They both go up and to the right. More compute, more performance.
This is the famous ‘scaling law’ in action. It was very early versions of charts like these that caused OpenAI to invest $100m in building ChatGPT. It’s charts like these that have caused Open AI CEO Sam Altman to post a new blog, cryptically titled: three observations.
The first of his three observations is what that ‘most important chart in tech’ is telling us – the scaling laws are still working. Although each step in performance improvement needs a 10x increase in compute (either before taking the test, or during it… or both!)
His second observation is that for a given level of AI, the cost falls by 10x per year. We have seen this with Open AI’s GPT4 models, which have consistently dropped in price by about 90% per year.
Sam’s point is that models are getting better, and cheaper. But can that continue? Can we afford to build enough chips and data centers for the next 10x, 100x or even 1000x in compute required to keep climbing up and to the right on ‘the most important chart in tech’?
That’s where his third observation comes in. Sam claims that, while the amount of compute needs to rise exponentially (10x, 100x… 1000x) the benefit that abundant intelligence brings is ‘super exponential’.
The costs rise fast, but the benefits rise even faster, according to Sam.
And I think he’s right. Look around you. Pretty much everything we humans built we built using our intelligence.
There’s yet to be a problem we’ve failed to solve with our intelligence. Given enough resolve.
Sam talks about how OpenAI are rolling out AI agents, that will be increasingly like junior coworkers. He asks us to imagine having 1000 of them. Or 1 million 🤯
I see it. I see that future.
But there’s a problem. How are you going to manage those 1000 co-workers. Let alone 1 million.
Whether it’s your contact center, or your software development team, or your marketing team, AI agents promise to allow you – and your customers – access to a virtually unlimited team of helpers.
But to access that incredible resource, you need to figure out how you, your team, and your customers can leverage that virtual team.
Increasingly, I think the bottleneck to AI adoption isn’t the power of the AI itself.
It’s the power of the user interface, to leverage that power that matters.
The future of AI, is UI.
Because now we have much more powerful capabilities, we need much more powerful user interfaces.
But the future of UI, is AI, too.
Because AI lets us generate UI. Imagine creating the perfect user interface for every customer, in every moment, for the specific task they’re trying to accomplish. Or the specific issue they’re trying to address.
I’m sure we’ll figure it out. But it’s going to take a while. It was decades before we settled on the right user interface approach for variable sized screens, especially touch-screens.
In the meantime, you need team members – and contact center agents – who are adept at using the AI we have today, with all its UI and other limitations.
That’s why your AI first contact center needs to combine AI first Business transformation, with Personal AI empowerment.
Kerry
PS: If you want a more regular dose of insights, follow or connect with me on LinkedIn for regular posts on conversational AI, mindset, and egg juggling, among other things!
PPS: You are building with GenAI right now, aren’t you? If not, what’s stopping you? Check out our blog on Gen-AI blockers, or sign up for a complimentary Strategy Workshop to help you get started.
If someone forwarded this to you, please subscribe yourself for weekly insights that’ll make you think differently about your IVR, voice, and chatbots.
Helping you get maximum ROI from conversational AI — whatever the platform.