When we talk about the chronology of artificial intelligence research in our lifetimes, there’s a specific kind of timeline that’s interesting – it involves a deliberate pause on research from some of the top leaders in the field, based on a consensus that it wasn’t the right time to move forward.
Combine that with subsequent waves of technology, like those happening right now – maybe more than one fundamental AI revolution – and you have a good set of insights on what’s happening, in the business world and beyond.
Yann LeCun used to head research at Meta. He’s a pre-eminent voice in tech, and took the stage at Davos this January to talk about some of the dynamics in play here.
In interviewing LeCun for segment on AI today, I thought about this timeline and how it informs the work that MIT people and others are doing now.
Yann LeCun: A Journey Toward Tech
From a young age, LeCun said, he always found this type of science fascinating. His parents took him to see 2001: A Space Odyssey, which was actually informed by the advice of Marvin Minsky, who I’ll talk about more a little bit later.
“One of the things that I found always fascinating since I was a kid, was the emergence of intelligence in animals and humans,” he said, detailing how that leads into work on neural nets.
Later, he explained, he signed onto his role at Meta only after clarifying that the work done there would be open source, and that he could stay in New York and teach at NYU.
The open source part is important, too – and LeCun was emphatic about the nature of the work that should occur to usher in the next AI age in a democratic way.
The AI Winter and How it Happened
If you watch the video of the interview, you’ll see us talking about Marvin Minsky a great deal. This individual really informed the world of AI in a fundamental way, starting with his work on the perceptron in the 1950s, and collaboration with others like Seymour Papert (LeCun also mentions Piaget and Chomsky talking about linguistics). Essentially, LeCun suggested, the limitations of 1980s data and learning models convinced Minsky and others that it was time for an AI winter, or a pause on work.
In going over this eventual slump in AI research, which lasted for many years, LeCun pointed out that when he asked Minsky about this more recently, the now-deceased scientist stood by his earlier decision.
“I actually discussed this with Minsky once,” LeCun said, “but I was a very young student at the time, and he said, ‘No, it was good to kill it, because it got us to kind of invent other things that would otherwise not have emerged.’”
The Future of Superintelligence
I asked LeCun about the prospect of artificial general intelligence, and he said he didn’t like the term. He prefers ‘AMI’ (advanced machine intelligence) and had a pretty bold answer for the clang of voices suggesting that we’re going to get to a place where “AI is smarter than us.”
“The idea that somehow intelligence is kind of a linear scale is nonsense,” he said. “Your cat is smarter than you are on certain things, and you’re smarter than it on certain things, A $30 gadget that you can bu, that can beat you at chess, is smarter than you at chess. So…the idea that somehow it’s a linear scale, that at some point it’s going to be an event when we reach AGI, is complete nonsense. It’s going to be progressive.”
In the end, he suggested, AI will be “smart” in human ways, but there’s no single tipping point. What he described was much more like the notion of an eventual singularity.
“There is no question that at some point in the future, we will have systems that are as intelligent as humans, in pretty much all of the domains where humans are intelligent,” he said.
LeCun also weighed in on generative AI, and how it will be replaced with a different model called Joint-Embedding Predictive Architecture (JEPA). “GenAI has a shelf life of three years,” he said.
Two AI Revolutions
Going back to this idea of open source, in looking at how quickly these technologies are evolving, LeCun suggested that committing to open source is how you build highly diverse fundamental models, and turn the flywheel of innovation, in the same ways that societies maintain a diverse press, and democracy. He mentioned PyTorch as the vehicle for ChatGPT and other technologies, as well as the value of “free exchange and acceleration” in LLM and neural net research. As for challenges, he contended that the benefits of AI outweigh the risks.
“Everyone will be smarter,” he said. “AI is not going to kill us all.”
LeCun predicted a different blueprint for the next generation of tools, safer more controller systems, and objective-driven AI with “common sense.”
And then there’s SSL:
“Self supervised learning, I think, is probably the most revolutionary concept that has really completely changed the way we practice machine learning over the last ten years or so,” he said. “And then there are all these things, like systems augmented with associated memories (etc.)”
He mentioned open source for wearables:
“In the future, every single one of our interactions with the digital world will be mediated by AI assistants, … smart devices like … glasses,” he said. “All of our information diet will come from Ai assistants. We cannot afford to have those assistants come from a handful of companies on the West Coast of the U.S., or China. It has to be highly diverse. And the only way for it to be diverse, which relates to the previous question, is open source foundation models that are then fine-tuned for vertical applications, or for learning every language in the world, every culture, every value system, and that’s how you get a diverse population of AI assistants.”
Concluding, LeCun had some words, too, on the place of this current time in history.
The last time this (kind of innovation) happened in a big scale was following the invention of printing press, which, you know, allowed the dissemination of knowledge, and philosophy, and basically brought down the feudal system in Europe, brought the enlightenment, and then caused the American Revolution, the French Revolution and emergence of democracy,” he said. “This had (an) enormous impact, right? Just the nature of knowledge. So AI may have kind of the same effect, but the next step (is going to) be a new renaissance.”
All of that is fascinating in looking at where we are going. I wanted to showcase these remarks for the next generation of people working on genAI and AGI systems.
