Pick up your phone. It is probably doing almost nothing right now.
Not because it lacks the power. The AI chip pressed into the glass sandwich in your hand is, by the measure of raw operations per second, extraordinary hardware.
It is just sitting there.
The chip nobody told you was an AI supercomputer
Every flagship phone sold in the last two years ships with a dedicated Neural Processing Unit, or NPU.
Apple’s A18 Neural Engine delivers 35 trillion operations per second. On the PC side, AMD Ryzen AI 300 offers up to 50 TOPS, and the Qualcomm Snapdragon X Elite reaches 45 TOPS.
These are not modest numbers.
NPUs process AI workloads with higher speed and lower power consumption, enabling features like facial recognition, voice processing, and real-time language translation to run directly on your device without cloud connectivity.
But running those features takes only a fraction of what the chip can handle.
What the phone is doing the rest of the day
The answer is largely nothing.
Smartphones and other personal devices accumulate enormous quantities of idle computing time every year, a number large enough to matter to the biggest AI labs on the planet.
That idle capacity is not a rounding error. It is a resource sitting unclaimed at global scale.
Most NPU hardware is sitting idle right now. The software ecosystem has not caught up.
NPUs in Copilot+ PCs are observed at very low utilization on typical workloads. The exceptions are a small fraction of what the hardware can handle.
The same gap exists on phones. The AI silicon is already paid for, already in billions of pockets, already drawing near-zero additional power.
Why the gap between hardware and software opened so wide
On Windows PCs, running standard applications typically does not touch the NPU at all.
Microsoft’s Windows ML API and DirectML are the intended routing layer, but application developers need to explicitly target these.
Mobile is similar. App developers default to the CPU or the cloud because that is the path of least resistance.
Most consumer applications still route AI workloads to the CPU or cloud, leaving the dedicated NPU silicon idle in devices that could be running inference locally, instantly, and privately.
Meanwhile, the demand for AI compute keeps growing.
The market for AI features in mobile apps is developing fast: from $27.7 billion in 2025, it is projected by industry analysts to grow to $322 billion by 2034.
The irony is almost perfect: companies are racing to build more data centers while a distributed supercomputer worth hundreds of billions of dollars sits dark in people’s jeans pockets.
The Microsoft Research paper that put a number on what those idle chips can actually do
Researchers presented a paper at EuroSys 2026, held April 27, 30 in Edinburgh, Scotland, that addressed this gap directly.
The paper highlights that mobile NPUs have underutilized computational resources, particularly their matrix multiplication units, during typical LLM inference, and proposes applying parallel test-time scaling techniques on mobile NPUs to leverage this idle compute capacity.
In plain terms: the idle portion of your phone’s AI chip is large enough to run a smaller language model significantly better, right now, without any new hardware.
As the researchers demonstrate, applying test-time scaling to the idle matrix multiplication capacity during decoding allows smaller models to match or exceed the accuracy of larger ones, improving generation quality by making use of compute that would otherwise go to waste.
The Microsoft Research paper demonstrated this on a real end-to-end inference system built for Qualcomm Snapdragon platforms, not a simulation.
The implications connect directly to a debate already reshaping the chip industry. Researchers working on NVIDIA’s Vera Rubin architecture are targeting post-training efficiency at the data-center end of the same problem: doing more AI work with the same silicon.
The Microsoft Research finding suggests the same lever exists at the opposite end of the compute stack, in a device already in a billion hands.
What comes next for all that sleeping power
Developers and product teams who build applications that route to CoreML, DirectML, and ONNX Runtime on NPU will deliver user experiences that cloud-dependent competitors cannot match on latency, privacy, or offline capability.
That is a genuine competitive opening, and it requires no new factories, no new chips, and no new mining of rare earth minerals.
The NPU market was valued at USD 2.5 billion in 2024 and is forecast by industry analysts to reach USD 15.7 billion by 2033. But the hardware those numbers represent is already manufactured and distributed.
The trend is already visible in how the industry thinks about inference costs. Enterprise partners citing a 20x inference cost advantage from on-device open models are pointing at the same underlying reality: the cheapest compute is the compute you already own.
Your phone’s AI chip will spend most of tonight doing absolutely nothing.
The question researchers are now seriously asking is whether that changes in the next two years, not with a new device, but with a software update that finally talks to the hardware that was always there.
