Google DeepMind CEO Demis Hassabis on AI’s Next Breakthroughs, What Counts As AGI, And Google’s AI Glasses Bet

AI is evolving fast, but AI researchers still have substantive work ahead of them. Figuring out how to get AI to learn continuously, for instance, is a problem that “has not been cracked yet,” Google DeepMind CEO Demis Hassabis told me last week. Tackling that problem, along with building better memory and finding more efficient use of the context window, should keep Hassabis and his team busy for a while.

In a live Big Technology Podcast recording at Davos, Hassabis spoke with me about the frontier of AI research, when it’s time to declare AGI, Google’s product plans — ranging from smart glasses to AI coding tools — and plenty more. I always find Hassabis’s perspective to be a good indicator of where the AI field is headed, and today I’m publishing our conversation in full.

You can read the full Q&A below, edited lightly for length and clarity, or listen to our discussion on Apple Podcasts, Spotify, YouTube, or your podcast app of choice.

Alex Kantrowitz: A year ago, there were questions about whether AI progress was starting to tail off. Those questions seem to have been settled for now. What specifically has helped the AI industry get past the concerns?

Demis Hassabis: For us internally, we were never questioning that. Just to be clear, I think we’ve always been seeing great improvements. So we were a bit puzzled by why there was this question in the air.

Some of it was people worried about data running out. And there is some truth in that — Has all the data had been used? Can we create synthetic data that’s going to be useful to learn from? But actually, it turns out you can wring more juice out of the existing architectures and data. So there’s plenty of room. And we’re still seeing that in both the pre-training, the post-training, and the thinking paradigms, and also the way that they all fit together. So I think there’s still plenty of headroom there, just with the techniques we already know about and tweaking and innovating on top of that.

A skeptic would say there have been a lot of tricks put on top of LLMs. There’s ‘scaffolding’ and ‘orchestration.’ An AI can use a tool to search the web, but it won’t remember what it learns. Is that just a limitation of the large language model paradigm?

I’m definitely a subscriber to the idea that maybe we need one or two more big breakthroughs before we’ll get to AGI. And I think they’re along the lines of things like continual learning, better memory, longer context windows—or perhaps more efficient context windows would be the right way to say it—so, don’t store everything, just store the important things. That would be a lot more efficient. That’s what the brain does. And better long-term reasoning and planning.

Now it remains to be seen whether just scaling up existing ideas and technologies will be enough to do that, or we need one or two more really big, insightful innovations. And probably, if you were to push me, I would be in the latter camp. But I think no matter what camp you’re in, we’re going to need large foundation models as the key component of the final AGI systems. Of that, I’m sure. So I’m not a subscriber to someone like Yann LeCun, who thinks they’re just some kind of dead end. I think the only debate in my mind is, are they a key component or the only component? So I think it’s between those two options.

This is one advantage we have of having such a deep and rich research bench. We can go after both of those things with maximum force—both scaling up the current paradigms and ideas. And when I say scaling up, that also involves innovation, by the way. Pre-training especially I think we’re very strong on. And then really new blue sky ideas for new architectures and things—the kinds of things we’ve invented over the last 10 years as Google and DeepMind, including transformers.

Can an AI model with a lot of hard-coded stuff ever be considered AGI?

No—well, it depends what you mean by a lot. I’m very interested in hybrid systems, is what I would call them. Or neuro-symbolic, sometimes people call them. AlphaFold, AlphaGo are examples of that. So some of our most important work combines neural networks and deep learning with things like Monte Carlo Tree Search. So I think that could be possible.

And there’s some very interesting work we’re doing, building the LLMs with things like evolutionary methods, AlphaEvolve, to actually go and discover new knowledge. You may need something beyond what the existing methods do.

But I think learning is a critical part of AGI. It’s actually almost a defining feature. When we say general, we mean general learning. Can it learn new knowledge, and can it learn across any domain? That’s the general part. So for me, learning is synonymous with intelligence, and always has been.

If learning is synonymous with intelligence, these models still don’t have the ability to continually learn. They have goldfish brain. They can search the internet, figure things out, but the underlying model doesn’t change. How can the continual learning problem be solved?

I can give you some clues. We are working very hard on it. We’ve done some work—I think the best work on this in the past—with things like AlphaZero. The learn-from-scratch versions of AlphaGo. AlphaGo Zero also learned on top of the knowledge it already had. So we’ve done it in much narrower domains. Games are obviously a lot easier than the messy real world, so it remains to be seen if those kinds of techniques will really scale and generalize to the real world and actual real-world problems. But at least the methods we know can do some pretty impressive things.

And so now the question is, can we blend that, at least in my mind, with these big foundation models? And so of course, the foundation models are learning during training, but we would love them to learn out in the wild, including things like personalization. I think that’s going to happen, and I feel like that’s a critical part of building a great assistant—that it understands you and it works for you as technology that works for you.

And we’ve released our first versions of that just last week. Personal Intelligence is the first baby steps towards that. But I think to have it, you want to do it more than just having your data in the context window. You want to have something a bit deeper than that, which is, as you say, actually changes the model over time. That’s what ideally you would have. And that technique has not been cracked yet.

Share

Sam Altman, toward the end of last year, told me that AGI is under-defined. And what he wishes everybody could agree to was that we’ve sort of whooshed by AGI and we move towards superintelligence. Do you agree?

I’m sure he does wish that, but absolutely not. I don’t think AGI should be turned into a marketing term for commercial gain. I think there has always been a scientific definition of that.

My definition is a system that can exhibit all the cognitive capabilities humans can, and I mean all. So that means the highest levels of human creativity that we always celebrate, the scientists and the artists that we admire. So it means not just solving a math equation or a conjecture, but coming up with a breakthrough conjecture—that’s much harder. Not solving something in physics or some bit of chemistry, some problem, even like AlphaFold’s protein folding. But actually coming up with a new theory of physics, something like Einstein did with general relativity. Can a system come up with that? Because of course, we can do that. The smartest humans with our human brain architectures have been able to do that in history.

And the same on the art side—not just create a pastiche of what’s known, but actually be Picasso or Mozart and create a completely new genre of art that we’d never seen before. And today’s systems, in my opinion, are nowhere near that. Doesn’t matter how many Erdős problems you solve, which—I mean, it’s good that we’re doing those things, but I think it’s far, far from what a true invention, or someone like Ramanujan would have been able to do.

And you need to have a system that can potentially do that across all these domains. And then on top of that, I’d add in physical intelligence. Because of course, we can play sports and control our bodies to amazing levels—the elite sports people that are walking around here today in Davos. And we’re still way off of that on robotics as another example.

So I think an AGI system would have to be able to do all of those things to really fulfill the original goal of the AI field. And I think we’re five to ten years away from that.

I think the argument would be that if something can do all