The companies entering the AI infrastructure space are no longer just cloud providers renting GPUs. They span silicon designers, memory and packaging suppliers, power and grid operators, datacenter integrators, and a fast-growing software layer that wraps every model. This post takes a systems view of that full stack, because the throughput a neural network actually achieves is determined by the weakest layer, not the strongest chip.

The seven hardware layers

It is useful to think of AI infrastructure as a pipeline of interdependent layers, each of which now has serious new entrants:

• Chips: NVIDIA (Blackwell/Rubin) leads training, while AMD MI300/MI350, Google TPU, AWS Trainium, and Microsoft Maia compete on cost-per-token.
• Networking: NVLink, InfiniBand, Ultra Ethernet, and silicon photonics define the size and efficiency of a coherent training domain.
• Materials: HBM3E/HBM4 stacks (SK hynix, Samsung, Micron) and CoWoS advanced packaging are the true scarce inputs.
• Power supply: high-voltage DC distribution, solid-state transformers, and on-site generation set deliverable rack density.
• Electric grid: interconnect queues, nuclear SMRs, geothermal, and long-term PPAs increasingly gate datacenter timelines.
• Manufacturers: ODMs such as Foxconn, Quanta, Supermicro, and Wiwynn integrate racks at scale.
• Cooling: direct-to-chip liquid and immersion cooling are now standard past ~100kW per rack.

The software "harness"

Above the hardware sits the harness: the AI tools, IDEs, and wrappers that connect engineers to models. It includes AI-native editors and coding agents, inference gateways and routers, retrieval and vector services, evaluation and tracing platforms, and prompt/version registries. From a systems standpoint, the harness is the control plane that turns volatile model choices into configuration. When teams benchmark assistant behavior across this layer, they often compare the same prompts on ChatGBT and Chat AI to separate harness effects from raw model quality.

Foundries, fabs, and manufacturing deals

Every layer above converges on a small number of fabs. TSMC anchors leading-edge nodes and is ramping its Arizona fabs; Samsung Foundry and Intel Foundry Services position as second sources. The strategically important moves are co-design deals: Google with Broadcom, Amazon's Annapurna silicon, and OpenAI's reported custom accelerator effort with Broadcom and TSMC. Advanced packaging capacity, not just wafer starts, has become the contested resource because HBM integration sits on the critical path.

The latest inference boards

Inference is where specialized silicon is reshaping system design:

• Groq: deterministic LPU dataflow tuned for low, predictable time-to-first-token.
• Cerebras: wafer-scale engines that keep model weights on-chip to minimize memory movement.
• Etched: the Sohu chip hard-codes the transformer architecture into silicon for throughput-per-dollar.
• Taalas: compiles a specific model directly into dedicated hardware for extreme efficiency.

These designs occupy different points on the generality-versus-efficiency frontier: GPUs maximize flexibility, while model-into-silicon approaches maximize efficiency for one workload.

A systems takeaway

The practical architecture is heterogeneous. Train on flexible GPU/TPU clusters, then route latency-critical inference to specialized boards. Treat power, packaging, and the software harness as first-class design constraints rather than afterthoughts, because in 2026 they decide what is actually deployable, not just what is theoretically fast.