IBM AIU Chip Family
What IBM's Artificial Intelligence Unit is, how the AIU research line produced Telum II and Spyre, and why IBM builds its own inference silicon instead of buying it.
The AIU, or Artificial Intelligence Unit, is IBM's research line for purpose-built AI inference silicon. It is not a product you order. It is the design lineage that produced the on-chip accelerator in Telum II and the Spyre Accelerator card, and understanding it explains a strategic choice that puzzles people: why IBM designs its own inference hardware in a market where accelerators can simply be bought.
Why IBM designs its own inference silicon
The general-purpose AI accelerator market is built around training: very large models, very large batches, throughput measured over hours. IBM's enterprise customers have a different problem. They want to score a transaction while the transaction is still open, inside a system of record that has been running for decades, without shipping the data anywhere. That is a latency and integration problem far more than a raw-throughput one.
Latency inside a transaction
A fraud check that returns after the payment has cleared is worthless. Designing for scoring inside a live transaction produces very different silicon from designing for a training run, and that is the requirement the AIU line targets.
Native instructions, not an I/O detour
On Telum II, matrix multiplication and AI primitives run as native processor instructions rather than as memory-mapped I/O to a separate device. Removing that round trip is the point of putting the accelerator on the die.
Low precision on purpose
Telum II uses int4 and int8 formats for faster and lower-energy inference. Enterprise scoring rarely needs training-grade numeric precision, and giving it up buys speed and power efficiency where they actually matter.
Data stays where it already is
The whole architecture assumes the data is already on the machine and should not leave it. For regulated industries running IBM i and z, that is frequently the deciding constraint rather than a preference.
What the AIU line indicates about future Power hardware
IBM has confirmed very little about the generation after Power 11. What it has confirmed is that the accelerator strategy continues: Bill Starke, the Power processor chief architect, has said Spyre "will be a piece of the Power 11 portfolio and beyond." Since Spyre descends from the AIU line, the durability of that research investment is a reasonable read on IBM's direction, even though no specification exists.
This is a direction, not a specification. Nothing about the AIU lineage tells you the core count, clock speed, cache or process node of any future Power processor, and no such figure has been published.
Related
The full AI silicon picture is on the AI trajectory: Telum II, Spyre and AIU, with the directory entry at IBM AIU chip family. For the card you can actually order today see the Spyre Accelerator, and for what any of it means for the next processor, the Power 12 CPU page.