Partner Content: What is an NPU and why is it key for a new age of PCs?
What is an NPU and why is it key for AI PCs?
The demand for AI use cases is surging across nearly every industry, highlighting the necessity for a computing architecture specifically tailored for on-device AI processing. AI workloads primarily consist of calculating neural network layers comprised of scalar, vector, and tensor maths followed by a non-linear activation function[1]. An NPU, or neural processing unit, is a dedicated processor on a larger system on chip (SoC) designed specifically for accelerating neural network operations and AI tasks. AI workloads consume large amounts of power when executed on the CPU or GPU, but NPUs are specifically designed to efficiently handle AI inferencing, leaving CPU and GPU free for other tasks and meaning the system performance or battery life are not compromised.
Eventually, some PCs will have 40+ TOPS , achieved by utilizing the CPU, GPU and NPU in tandem, resulting in poor PC performance. So driving the efficiency between the NPU, GPU and CPU is crucial.
Organizations planning device refreshes are increasingly considering AI capabilities and the necessary processing power for future advancements. Before embarking on AI implementation, organisations must carefully consider their compute power, networking, data handling, data processing frameworks, and security to determine what they need from new environments. This evaluation will help determine the requirements for new environments, including whether an updated computing architecture is needed to support on-device AI. Assessing the performance of these specialized processors requires a comprehensive understanding of the metrics that underpin their capabilities.
What does TOPS mean?
TOPS, or trillions of operations per second, is the cornerstone performance metric for NPUs. It measures the number of operations (e.g. additions and multiples) that can be executed within one second. Exploring parameters of the TOPS equation like frequency and precision can offer a deeper understanding of an NPU’s capabilities.
How do NPUs help with everyday tasks?
NPUs have been around for several years but have received renewed attention with the rise of AI. As of now, most AI workloads mainly reside in the cloud, but AI PCs with built-in NPUs mean that AI applications can be run directly on the device.
Experiences like image generation, document creation, text-to-speech, and speech-to-text are all enabled thanks to the NPU. The ability for a PC to efficiently run large, complex AI models entirely on-device is becoming crucial, providing immediacy, reliability, and reduced latency for users. By shifting workloads to the edge, AI applications can function independently of cloud connectivity, offering flexibility and accessibility.
NPUs facilitate the localization of this process, enhancing the security of sensitive data by eliminating the need to send it to external servers. This also ensures smooth operation in areas with poor or no internet connectivity, improving both the user experience and the reliability of the device. Consequently, users benefit from quicker, more responsive devices that better comprehend and predict their requirements[2].
What devices use NPUs?
Microsoft requires Copilot+ PCs to have at least 40 TOPS of NPU processing capacity. To ensure the most power and efficiency, the Snapdragon® X Series processors go even further—setting a new performance standard at 45 TOPS. As AI applications grow in sophistication, so will the processing power required. Therefore, NPUs will become increasingly important to handling vast quantities of data without compromising on user experience.
Click here to learn more from Qualcomm and TD SYNNEX on AI PCs and the future of personal computing.
[2]What Are NPUs? | Microsoft Surface
Snapdragon and Qualcomm branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
This Article is sponsored by TD SYNNEX