Every time you send a message to an AI assistant, a data center somewhere runs a sprint.
It moves through two very distinct phases. Most data centers today handle both on one kind of chip. That is the part a San Francisco startup is now betting is wrong.
And $300 million just landed to prove it.
The bet is not about building a faster chip. It is about smarter routing between the chips that already exist.
So what does that routing actually fix, and why has it taken this long to matter?
Why a single GPU architecture leaves half the job running slowly
An AI inference request is not a uniform workload. During the prefill phase, the model processes the input prompt and constructs the state needed to respond, making raw computational throughput the central constraint. During the decode phase, it generates tokens one by one while repeatedly accessing weights and cached context, which makes memory bandwidth the binding limit instead.
Those are two fundamentally different bottlenecks happening back to back inside the same job. A heterogeneous system can place prefill on hardware suited to dense computation and decode on processors optimized for memory movement. That is the core insight, and it is simpler than it sounds.
Running both phases on a single GPU type means the chip is either overbuilt for decode or under-resourced for prefill, burning power and money either way. The company says its software solves that by turning hardware heterogeneity into an orchestration problem rather than a procurement headache. Neither phase has to wait for silicon that was never designed for it.
What the platform actually does inside a data center rack
The company operates what it calls the industry’s first multi-silicon inference cloud, disaggregating AI model workloads and routing each inference phase to the most appropriate hardware. Partners span GPUs, purpose-built accelerators, and CPUs from a range of major vendors, covering the dominant chip architectures in production today.
The software allows an AI workload to run simultaneously across diverse hardware types, splitting an application’s work across traditional CPUs, AI-tuned GPUs, and high-memory systems. The result is a kind of live traffic management for compute: each token, each phase, each memory call lands on whichever physical chip can handle it fastest right now. That dynamic dispatch is the thing no single-vendor stack can replicate by design.
The company is also developing an inference-optimized server with no motherboard, designed to function inside a data center or as a standalone system beyond one. That detail points toward a longer ambition: if the software can route across chips in the cloud, custom motherboard-free hardware is the logical next step for pushing the platform into deployments that no longer depend entirely on conventional data center infrastructure.
The funding round, the valuation and where the numbers come from
Gimlet Labs announced a $300 million Series B, bringing its valuation to $3 billion and total funding to $392 million. The round was led by Andreessen Horowitz with participation from Sapphire Ventures, M12, Arm, Menlo Ventures, and Factory.
The presence of Arm as an investor is worth noting on its own. Arm designs the chip cores inside a growing share of AI accelerators, giving it a direct financial stake in how inference workloads are routed across mixed silicon fleets. Backing the routing layer is, for Arm, a hedge on its own hardware’s utilization.
Since emerging from stealth in October 2025 with eight-figure revenues, Gimlet Labs has tripled its customer base and added one of the top three frontier labs as well as one of the top three hyperscalers as customers. The company says heterogeneous disaggregation can deliver roughly 3 to 10 times faster performance on frontier workloads, though those figures are company-reported rather than independently audited.
That last distinction matters. Independent benchmarks for multi-silicon orchestration are still rare, and the wide range in those performance claims reflects real variance across workload types rather than a single verified outcome.
Where the model gets complicated and what it still has to prove
Enterprises adopting the platform must accept real tradeoffs: software complexity, hardware qualification work, and another operational control layer sitting above the infrastructure they already manage. A company running a stable GPU cluster has to weigh whether orchestration gains outweigh the cost of managing mixed silicon fleets and a new software dependency with its own failure modes.
The explosion of agentic workloads has exposed a genuine limitation in homogeneous hardware. Inference volumes are reaching quadrillions of tokens per month, and single-vendor stacks were not designed for that kind of sustained, varied pressure. The question is whether multi-silicon orchestration is the structural answer, or whether chip makers simply close the efficiency gap themselves through successive generations.
That tension is sharpest at the very top of the market. The company’s customer list already includes one of the world’s largest cloud providers, organizations that could, if they chose, build their own routing layer in-house. The shift in enterprise AI spending toward a handful of dominant players makes staying indispensable to well-resourced, technically capable customers the harder long-term challenge.
Commoditization risk is real. If the routing logic becomes well understood, larger players have both the incentive and the engineering depth to absorb it.
What a $3 billion bet on routing software says about where AI hardware is heading
The $3 billion valuation reflects two things: expected growth in inference demand and the possibility that heterogeneous orchestration becomes a differentiated control layer sitting above the underlying chips. If that software layer proves portable across vendors and workloads, the company can monetize both cloud capacity and the scheduling intelligence that makes mixed hardware economically useful at scale.
The funding will help grow infrastructure capacity for the platform’s serverless edition, with the company scaling to hundreds of megawatts in managed heterogeneous infrastructure. Running the cloud itself gives Gimlet control over utilization data, which feeds back into making the routing smarter over time. That feedback loop is the compounding asset a pure software licensing model would never accumulate.
The industry is tracking toward an estimated $765 billion in AI capital expenditure in 2026 alone. Inside that number, the question of which chip handles which task is worth enormous sums. The hardware advances moving through the pipeline will only add new silicon types to route, making the orchestration problem harder and the right software answer more valuable with every new chip that ships.
