Deep learning and artificial intelligence continue to push our ideas of what’s possible, and expand the capabilities of digital systems housed on electronic hardware.
One such development is notable for its integration of different kinds of deep learning tasks, all on one chip: over at MIT news earlier this month, Adam Zewe described this hardware benchmark as something built on a decade of research, and explained how this works.
Essentially, the photonic chip replaces traditional circuitry with light, where optical data flows through the system to accomplish computing goals on a lower energy footprint.
The major development here is the ability of a single chip to process both matrix multiplication, and non-linear operations.
Matrix multiplication involves taking two matrices and multiplying them together, with takes a specific kind of logic. As for nonlinear operations, these don’t follow the classical linear equations that are critical to solving conventional problems.
“Nonlinearity in optics is quite challenging because photons don’t interact with each other very easily. That makes it very power consuming to trigger optical nonlinearities, so it becomes challenging to build a system that can do it in a scalable way,” explains one of the primary scientists in Zewe’s piece.
By designing something called non-linear optical function units or NOFUs, the team designed a process where the data can “stay in the optical domain” the entire way through the life cycle.
That in turn combines low latency with low energy use and high accuracy for relevant testing results.
The Van Neumann Architecture: Is It Obsolete?
In a paper documenting some of these new approaches, the authors refer to traditional methods that have demonstrated their limitations in supporting neural networks:
“Machine learning technologies have been extensively applied in high-performance information-processing fields,” they write. “However, the computation rate of existing hardware is severely circumscribed by conventional Von Neumann architecture. Photonic approaches have demonstrated extraordinary potential for executing deep learning processes that involve complex calculations…”
If it’s not familiar to you, the Von Neumann architecture is named after the mathematician and physicist who pioneered the emergence of mainframes like the ENIAC in 1945. It combines these components – a stored memory concept, a memory space, sequential execution, and a central processing unit, along with some input/output interface.
There’s also the corresponding Van Neumann bottleneck, describing limitations in transfer between the CPU and the memory.
“The system bus is used to transfer all data between the components that make up the von Neumann architecture, creating what has become an increasing bottleneck as workloads have changed and data sets have grown larger,” explains Robert Sheldon at TechTarget. “Over the years, computer components have evolved to try to meet the needs of these changing workloads. For example, processor speeds are significantly faster, and memory supports greater densities, making it possible to store more data in less space. In contrast to these improvements, transfer rates between the CPU and memory have made only modest gains. As a result, the processor is spending more of its time sitting idle, waiting for data to be fetched from memory. No matter how fast a given processor can work, it is limited by the rate of transfer allowed by the system bus. A faster processor usually means that it will spend more time sitting idle.”
As computer science experts will tell you, we’ve used this kind of architecture for decades, but now it seems to be receding as anything relevant in today’s technology world. We’re developing brand new systems that really challenge the conventional thinking when it comes to hardware support for these very muscular models and AI agents that are coming online now.
That’s a key point when you’re talking about the ability of AGI to emerge in our societies. It has to run on something, and unless the hardware keeps evolving, you’ll see those kinds of bottlenecks (which you could call Von Neumann bottlenecks if you want) hamper the forward progress of how computers think and learn.
New Quantum Systems
I was wondering where people are at with quantum computing, since it’s another paradigm that’s challenging conventional binary operations.
It turns out that Google has made an enormous announcement of progress just this week, where the parent company Alphabet is experiencing a big stock increase over a new chip called Willow.
It sounds like the main capability of this system is to use greater numbers of quantum bits or ‘qubits’ to do sophisticated kinds of error correction. Reuters coverage provides this prediction by Thomas Hayes, chairman and managing member at Great Hill Capital:
“While (there are) no current uses, (Willow) will have major implications in science, medicine and finance. Willow reduces errors exponentially and could lead to major breakthroughs and discoveries across industries.”
What’s Ahead
So combining advances like the photonic system-on-chip model and new quantum capabilities, we’re starting to see the rough outlines of how the next generation of hardware will power supercomputers, whose digital brains are going to be pretty enigmatic and awe-inspiring to us simple mortals.
And for what it’s worth, understanding the hardware is going to be a pretty important component of this. It’s one thing to know a little about LLMs and how they work: the hardware expertise is going to be a valuable skill set for the next generation. Stay tuned for more on what’s happening in this fascinating field.
