While the concept of an AI PC is several years in the making, the messaging around AI PCs hit a feverish pitch in late 2023 with the launch of a new generation of processors with higher AI processing capabilities and the push by Microsoft integrate Copilot into every PC. However, making a PC intelligent is more than just running neural network models on the PC, it’s also about making it contextually aware of its environment and user (or users). This requires the use of sensors, network information, user information, and low-power processing that can be always on and constantly adapting. In this respect, this is one of thousands of opportunities to leverage the capabilities of Field Programmable Gate Arrays or FPGAs.
Contextual Awareness
Context awareness, or the ability to understand and adapt to a given environment at a given time, is something that is inherent in humans, but it is rather new and of growing importance to electronic devices.
PC OEMs and technology providers have already been developing AI technology that uses image sensors information to increase the security of PCs, such as human detection, facial recognition, attention sensing, and threat monitoring to allow for both sign-in and locking of PCs when not in use or when a security threat, such as an unauthorized user or a third party looking over your shoulder is detected. But it should not end there. A mobile PC should be able to determine if you are at home, in the office, or at a restaurant to determine and grant you access to the networks and peripherals available and adjust security levels accordingly. An intelligent PC should also be able to use AI agents to adjust the environment (temperature, lighting, music, etc.) to your likes or needs. In the end, building an intelligent PC is more than just running AI models to improve productivity applications.
The Wide World Of Sensors
This is where sensors and sensor processing come into the equation. PCs, especially mobile PCs, already have some level of sensing for system temperature, image, and audio. There is an amazing amount of information that can be gained just from the external image and audio sensors, including identification of people and other objects, the identification and authentication of the user and other people in proximity, movement by the user, the health of the user (yes, even the heart rate and temperature), the spatial dimensions of the environment, objects within the environment, and identification of the environment itself. This speaks to the continued advancements of sensor technology, which will continue to improve and like most electronic platforms, there is likely to be integration of additional sensors to collect even more environmental information. However, all this raw sensor data requires processing in real-time or close to real-time and on-device to ensure the security of the data and minimize latency, even when the rest of the system is in sleep state. This processing also must adapt to changing AI models and use cases/experiences over the life of the PC. This is where low-power FPGAs come in.
Lattice Semiconductor has been working with leading PC OEMs including Dell, Lenovo, LG, and others using its industry leading low-power CrossLink-NX family for vision processing and even its Avant general purpose FPGAs for higher levels of sensor processing in the future. The Lattice CrossLink-NX products combine processing, a Digital Signal Processor (DSP) for matrix multiplication, on-chip memory, and a flexible FPGA fabric for flexible I/O configurations which is important given the often disparate nature of sensor interfaces. With a small package size, the Lattice devices can be located on or close to the sensors. Additionally, by leveraging parallel processing, the Lattice devices can operate at just 150MHz, which allows them to be always powered on while drawing just milliwatts. This saves a significant amount of power by not only offloading the sensor processing from the host processor but also by allowing the multi-GHz host processor and most of the system to enter a sleep state more often, even between key clicks or frame updates.
Additional power savings and security come from using the sensor information to automatically dim the display when the user is not looking at the screen or when someone else is walking behind the user. And for a better user experience, the sensor information is used to automatically increase the brightness of the display if the user appears to be falling asleep or having difficulties seeing the screen by squinting. These are just a few of many ways the image sensor with a Lattice CrossLink-NX is being used in conjunction with AI models today and new sensor applications are under development.
With multiple programmable functions, the Lattice products also provide adaptability to different product configurations, new AI models, new data standards, and even different interfaces without a change in hardware.
The best part of this is that the PC is benefiting from the innovation driven by other platforms and vice versa where Lattice FPGAs are used. User attention sensing is similar to the driver monitoring technology that is being used in vehicles. The security controls are similar to those in government and healthcare applications. The display brightness control is similar to the brightness control in commercial lighting. And security monitoring comes directly from commercial security systems. The applications for sensors are almost endless as innovative companies determine new ways to use sensors and sensor data. One of the most unique uses is for a user interface, such as gesture control, a common feature for VR headsets that is likely to become more common on other consumer devices such as TVs and PCs in the future.
The Focus On Intelligence
While AI is the buzz word of the day, intelligence is the ultimate goal of every consumer and commercial electronic platform. And while a multi-GHz processor can probably provide all the processing required to accomplish the task, it is not an efficient use of the performance nor is it likely to provide a positive user experience given its prodigious drain on the battery in these types of use cases. Creating intelligent solutions will require a hybrid approach to improve overall system performance, efficiency, and security. One of the key technologies to enabling the latest developments in AI and intelligence is FPGAs. While in the past, FPGAs were often viewed as a niche technology used in the development process of more advanced technologies or for use in very low-volume specialized applications, this is changing with the focus on small/low-power for factors with increased performance and functionality. FPGAs are flexible and adaptable to changing requirements, such as different sensors, and can be a cost and power efficient solution for more intelligent PCs and other electronic platforms.