Perhaps the new year is a good time to look back on the old year, and see where we’ve come within the annual cycle.
There will never be another year like 2024 again for artificial intelligence.
Throughout the year, obscure product demos became household names. People started to really zero in on using non-human sentient agents for problems like climate change. We also saw radical changes in the infrastructure behind these models.
I was looking at some of the round ups that are out there as we launch into the new year. This one is fairly detailed, and has several dozen points, many of which I’ve covered on this blog. But here are some of the big ones that stand out to me as I look back through the last 365 days.
AGI is Closer
One of the overarching ideas that comes back, time and time again, is that we’re closer to AGI than we thought we were at the beginning of last year.
Here’s a survey that I did with a variety of people close to the industry in January. You can see those different time frame predictions, balanced against each other.
Now, though, much of the cognoscenti is thinking that we’re on the cusp of AGI itself right now. So a good number of those forecasts are going to be revised a lot.
AI Can Solve Language
Toward the end of the year, we also found out that we actually have the power right now to build real-time translation into our consumer products.
That mainly came about through the demos of Meta’s AI Ray-Ban glasses just weeks ago. When Mark Zuckerberg interviews people with the AI engine that transforms his question to other languages in real time, we see this technology at work.
Language is important, too.
I was looking at this interview with Lex Fridman from last February, where he was talking about the importance of applying AI to different world languages. We can’t take for granted, he explained, that people speak English.
“Anything where there’s interaction going on with a product, all of that should be captured, all that should be converted into data,” he said at the time. “And that’s going to be the advantage – the algorithms don’t matter … you have to be able to fine-tune it to each individual person, and do that, not across a single day or single interaction, but across a lifetime, where you share memories, the low, the highs, and the lows, with your large language model.”
I’ve consistently brought the analogy of the tower of Bible story to the process of figuring out how to use AI to communicate. It’s a ‘reverse tower of Babel’ in which various language speakers come together to celebrate their new ability to understand one another without the use of a human translator.
So that’s something else that was brand new in 2024.
The Transformer is the Engine, but It’s Also Replaceable
As 2024 wore on, I covered the use of transformers in new language model systems.
Experts talk about the transformer as an ‘attention mechanism’ that allows the program to focus on things that matter more, to it, and to the human user.
But 2024 also brought glimmers of brand-new concepts to replace the transformer, ideas that move toward the realm of quantum computing and super powerful processing of information that’s not gated by a traditional logic structure.
Which brings me to my next point, which is extra-important.
Revolutionizing Neural Network Capacity
Another thing we saw grow in prominence is liquid neural networks.
Now is the time to add the usual disclaimer – that I have consulted on liquid neural network projects tackled by the MIT CSAIL lab group under director Daniela Rus. So I have some personal affiliation with this trend.
Liquid neural networks change the essential structure of the digital organism, in order to allow for much more powerful AI cognition on fewer resources.
That’s, to a large extent, the type of thing that’s been useful in allowing people to put powerful LLMs on edge devices like smartphones. It’s probably the deciding factor in the ability of Google to roll out Gemini on personal devices late this year. So now we’re able to ‘talk to our pockets’ quite literally, and that’s a big difference. Part of the acceptance of AI itself is going to be in its ubiquity – where we encounter it, and how it impacts our lives.
AI is Winning at Multimedia
Here’s one more big overarching premise of the work that people have done with AI in 2024. It has to do with media.
I looked back, and it turns out I covered an early notice on OpenAI’s Sora in February. And sure enough, late last year we saw an early version roll out. I used it personally to create some interesting and whimsical little film clips, all without any casting or shooting or production at all. It was pretty amazing.
That’s not to mention the groundbreaking text-to-podcast model where you can actually plug in a PDF or some resource info sheet, and have two non-human ‘people’ gabbing about your chosen topic, sounding exactly like a couple of traditional disc jockeys. (Also: check out the brand-new blizzard of stories about Scarlett Johansen protesting the use of a Scarlett-esque voice for the now-pulled Sky assistant.)
This is another example of personal use of AI to bring home the point that we’re in a new era now. As you listen to these people talk, or even interact with them in conversation, you have to ask yourself: are these people real? And how do I know that? They’re responding to me personally in real time – how do they do that if they don’t exist?
You could call this a ‘deep Turing test’, and it’s clear that the systems are passing with flying colors.
Anyway, that’s our roundup for 2024. There’s a lot more, of course, from genomics to publishing, and everything in between, but now that we’re past the auld lang syne, people are asking themselves what’s to come in 2025? We’ll see, pretty soon.
