Where FPGAs win

You know an FPGA is a fabric of LUTs, hard blocks and programmable routing configured by a bitstream (see Lookup tables and the FPGA fabric) and compiled by a synthesis, place and route flow (From hardware code to bitstream). This chapter is about the build-or-buy decision and the markets it creates. Three quantities set it:

• The efficiency gap. Per unit of function, an FPGA spends 35× the area, 3.4–4.6× the delay and 14× the dynamic power on plain logic at 90 nm; hard multipliers and memories cut area to 18× and power to 7.1× but leave delay near 3×.¹ A later study of neural-network accelerators, as reported in a 2021 survey, found the area gap smaller but still about 9× for full-featured designs that use RAM and DSP blocks heavily.²
• NRE versus unit cost. Mask sets alone rose from $65K at 250 nm to $5.7M at 16 nm in Moonwalk’s model, and labor, CAD tools and licensed IP add more.³ That fixed cost sets a break-even volume below which the FPGA is cheaper.
• The value of change. An ASIC is specified once for its whole service life. Azure’s networking team had to spec its SmartNIC logic for 7 years ahead (1–2 years to silicon plus a 5-year server life) and judged that impossible.⁶ A field upgrade turns that risk into a software release.

The rest follows from those: prototyping (time-to-silicon is the cost), latency-critical streaming (where a spatial pipeline beats a processor whatever the gap), datacenter offload (scale amortizes NRE but services change too fast for silicon), and space, defense and long-lived products (low volume, obsolescence, and radiation behavior). The last section closes the guide by placing FPGAs beside CPUs, GPUs and AI accelerators.

Kuon and Rose implemented identical RTL on a 90 nm Stratix II and in a 90 nm standard-cell flow, measuring FPGA area as every used LAB, memory and DSP tile including its share of routing.¹ Logic-only averages: 35× area, 3.4× critical-path delay against the fastest speed grade and 4.6× against the slowest, 14× dynamic power. Static power came out at 87× (typical silicon, 25 °C) or 5.4× (worst case), which the authors declined to interpret: leakage is too process- and screening-dependent to compare across two foundries. It did correlate with area (r ≈ 0.8).¹

Hard blocks behaved asymmetrically. Area fell to 25× with DSP blocks, 33× with memories, 18× with both; dynamic power to 12×, 14× and 7.1×. Delay barely moved (3.5×, 3.5×, 3.0×), and some circuits got slower with the hard multipliers: 5×5 multiplies gained nothing from 9×9 hardware, 32-bit multiplies wasted 36×36 blocks, and routing to the fixed columns ate any remaining advantage.¹ Boutros and Betz report the same pattern in later FIR-filter studies: hard DSPs gave large area gains and about 2× frequency, still well short of 35×, because each DSP tile carries its own programmable routing interface.²

Two caveats on the headline numbers. First, they are core-logic comparisons on one process generation; I/O, hard transceivers and memory controllers are near-parity, which is why whole-system gaps (2–3× in Azure’s SmartNIC estimate) can be far smaller than logic gaps.⁶ Second, the area and speed gaps compound if you try to buy back speed with parallelism: to match ASIC throughput by replication you need about 35 × 3.4 ≈ 119× the area, Kuon and Rose’s “at least 119”.¹ Under the hood works through what that implies for hard-block-heavy devices.

Model each option as a fixed cost $F$ plus a variable cost $v$ per unit, where $v$ includes the unit price and lifetime energy:

with $W$ in watts, $L$ years in service and $e$ dollars per kWh. Then $N^{*} = (F_{\mathrm{A}} - F_{\mathrm{F}}) / (v_{\mathrm{F}} - v_{\mathrm{A}})$. The model is linear in everything, so sensitivities are easy: $N^{*}$ scales with the NRE difference and inversely with the per-unit difference. Moonwalk’s NRE model for ASIC Clouds breaks $F_{\mathrm{A}}$ into mask cost (a step function of node), backend labor proportional to gate count, frontend labor and CAD (roughly node-independent), IP licensing (rising quickly for DDR and PCIe at newer nodes), package and system design.³ Its headline conclusion is that node selection is the main NRE lever: for many datacenter workloads an advanced node like 16 nm gave worse total cost of ownership, and older nodes like 65 nm made more accelerators viable, because the NRE saved outweighed the energy and silicon given up.³

Two things the linear model hides. First, NRE is paid up front and units over time, so a discount rate raises the effective NRE. Second, schedule: an FPGA product ships a year or more earlier, which for a market with a short window can be worth more than any unit-cost difference. Microsoft’s Catapult team set the FPGA’s budget as a TCO cap instead of a break-even: at most 30% added to the cost of ownership of a server, including at most 10% added power.⁷ Total cost of ownership for accelerators is worked through in Comparing chips.

The Expert view adds electricity price and the FPGA’s own NRE, and prints the break-even arithmetic. The model is $C_i(N) = F_i + N v_i$ for each option. The structured ASIC is illustrative: 80% of the ASIC’s design cost, a quarter of its mask set, twice its unit cost and power. A mid-life change costs the FPGA 20% of its NRE; a fixed chip pays its custom masks plus 20% of its design cost and replaces the half of its units already shipped. Try:

• Find the volume band where the structured ASIC wins at the defaults (about 37,000 to 124,000 units). How does it shift with the change switch on?
• Set electricity to $0.40/kWh and 15 years. How low does the FPGA price have to go to keep break-even above 100,000?
• With the change on, push the ASIC design cost to $3M. Is the ASIC’s advantage now the masks, the unit cost or the power?

The economic case is time to the bug. A platform’s useful speed is cycles per second of wall clock once compile time is included, and the cost of a bug rises steeply after tapeout, when a fix needs a respin. Bachrach et al. set out the trade-off: software simulators (kHz on large designs) compile fast with full visibility; processor-based emulators (up to 4 MHz, several million dollars) keep visibility at high cost; FPGA boards (10–50 MHz on one device, sub-MHz across many) are cheap and fast but compile slowly and need re-synthesis to observe new signals.¹¹ The general platform trade-offs and the testbench problem (non-synthesizable stimulus must stay on a host) are in Verification.

Two structural points matter for the FPGA business. First, capacity: large prototypes are built from the biggest devices available, which is part of why vendors adopted multi-die interposer parts early, and only a fraction of routing tracks cross die boundaries (23% of vertical tracks on Virtex-7 interposer parts), which shapes how designs must be partitioned.² Second, elasticity: on cloud FPGAs, FireSim simulated 1,024 quad-core nodes at 3.4 MHz (under 1,000× slower than real time) and runs single nodes at tens to hundreds of megahertz, turning a capital purchase into an hourly cost of about $100 for the full cluster.¹²

Latency-critical streaming is where the FPGA–ASIC gap matters least and the processor–FPGA gap most. A cut-through pipeline starts work on the first bytes of a frame, holds state in on-chip SRAM, and has no instruction fetch, cache miss or scheduler in its path, so its latency is a fixed cycle count.

• Trading. Book handling in 253 ns with cuckoo hashing into external QDR SRAM, against tens of microseconds, nondeterministic, in software.⁹ STAC-T0 measures only network I/O (last inbound bit needed for the decision to first outbound bit) with essentially no trading logic; the June 2024 minimum was 13.9 ns on an AMD Alveo UL3524 with Exegy’s framework.¹⁰
• SmartNICs. AccelNet offloads Azure’s software-defined-networking flow tables to an FPGA placed as a bump in the wire between the host NIC and the top-of-rack switch. The paper rejects ASIC NICs (1–2 years spec-to-silicon against a 5-year server life, and firmware-limited embedded cores) and multicore SoC NICs (often 10 µs or more and high variability at 40 GbE and above, and single flows pinned to one core).⁶ Reported results: $<15\,\mu\mathrm{s}$ VM-to-VM TCP latency and 32 Gb/s, on over a million hosts, built by a team averaging fewer than five FPGA developers.⁶

The design method mattered as much as the hardware: Azure deployed iteratively, measured real traffic, and changed its hashing and caching strategies several times after first release, which an ASIC schedule would not have allowed.⁶

Catapult placed one Stratix V on a PCIe daughtercard per server, under 25 W so the PCIe slot could power it, and wired 48 FPGAs per half-rack into a 6 × 8 torus. Bing’s ranking was split across pipelines of eight FPGAs. Results: +95% ranking throughput per server at a fixed latency distribution, or 29% lower tail latency at equal throughput, for +10% power and within a 30% TCO increase.⁷ Catapult kept the FPGAs off the datacenter network on purpose, which did not suit networking offload. AccelNet’s converged design instead puts the FPGA between the NIC and the top-of-rack switch, with two PCIe links to the CPUs, so one board serves as both SmartNIC and accelerator and large models can span FPGAs over the network.⁶

Brainwave is the clearest case for FPGA inference. Its NPU is a single-threaded SIMD instruction set over a distributed microarchitecture that dispatches over 7 million operations from one instruction, with up to 96,000 MACs on a Stratix 10 280, and reached 10 to over 35 TFLOPS on large RNNs with no batching.⁸ Two FPGA-specific levers made that possible: synthesis specialization (native vector dimension, lane count and matrix tiles chosen per model at compile time) and narrow block floating point (a shared 5-bit exponent per 128 values, 2–5-bit mantissas after brief fine-tuning), which removes per-MAC shifters and packs 2–3-bit multiplies into DSP blocks.⁸

Vendors moved the same lever into silicon. Intel’s Stratix 10 NX replaced DSP blocks with AI tensor blocks (30 int8 or 60 int4 MACs per block at about the same size); Agilex DSPs added int9, fp16 and bfloat16; Versal’s DSP58 supports int8; Achronix’s Speedster7t added machine-learning processing blocks from int16 down to int3 plus fp24, fp16 and bfloat16.² Each step trades some generality in the hard block for density on the dominant workload. The ranges CSET reports (FPGA inference efficiency ~10–100× a CPU, ASIC ~100–1,000×) are broad survey estimates, not measurements.⁵ Measured benchmarks and TCO methods for AI chips are in Comparing chips.

In an SRAM-based FPGA, the configuration memory is a large target whose upsets are persistent functional faults, because its bits define the logic and the routing. NASA Goddard’s error model for FPGAs adds three terms: configuration upsets, functional-logic upsets (flip-flops, block RAM), and single-event functional interrupts in control logic.¹⁶ Two architectural answers exist:

• Harden the configuration. Flash- or antifuse-configured parts remove configuration upsets. RTG4 combines hardened flash configuration cells with self-correcting TMR flip-flops, SET filters at flip-flop inputs, and hardened SRAM, multipliers and PLLs; the tested RT4G150 has 158,214 4-input LUTs.¹⁶
• Mitigate around SRAM configuration. Commercial and radiation-tolerant SRAM parts rely on scrubbing plus design-level redundancy. In a cyclotron test, distributed TMR (triplicating everything but global clocks, resets and enables) beat flip-flop-only TMR.¹⁵ The payoff is reconfiguration in orbit; AMD’s space-grade XQR Versal parts advertise unlimited on-orbit reprogramming with a configuration-memory scrubber built in.¹⁷

For obsolescence, the FPGA’s advantage is portability of the design, not of the part: the part goes out of production like any other, but RTL can be retargeted to a successor device with a new compile and requalification, where an ASIC would need a redesign on a new process. The cost is that the bitstream and the FPGA supply chain become assets to protect; the National Academies criticize published contract notices that named the exact FPGAs bought for the F-35.¹³ In regulated markets, a field upgrade is a regulatory event, so the option value of reprogrammability is discounted by the cost of re-approval.¹⁴

The hybrids are different answers to one question: where should the reconfigurable boundary sit?

• Adaptive SoC. Hard CPUs for control, fabric for bit-level and streaming work, a tiled array of SIMD VLIW vector processors (Versal’s AI Engines) for dense arithmetic, hard memory controllers, and a hard NoC.¹⁸ A hard NoC uses far less area and power than one built in soft logic, runs at a fixed frequency decoupled from the fabric, and makes good use of the few wires that cross interposer boundaries; Versal and Speedster7t both include one.² Altera’s Agilex 5 HPS combines Cortex-A76 and Cortex-A55 cores.¹⁹
• Co-packaged CPU + FPGA. Intel’s Xeon+FPGA v2 placed an Arria 10 die in the Xeon package with one coherent QPI/UPI and two PCIe links. Choi et al. found QPI-attached shared memory gave the lowest latency, but only 64 KB of coherent cache on the FPGA (under 5% of its on-chip memory), and that link quality matters only for kernels with a low computation-to-communication ratio.²¹
• eFPGA. Fabric at FPGA density, but only for the volatile function, with an FPGA tool flow. SLAC’s FABulous-based eFPGAs at 130 nm and 28 nm ran a machine-learning classifier for particle-detector data reduction.²² FABulous reports more than a dozen tapeouts across five processes, including SkyWater 130 nm.²³
• Structured ASIC. Metal-programmable bases amortize the expensive lower-layer masks across designs. Ahmed et al. found cost minimized with the fewest custom layers, while a few more layers improve delay and power, and attributed slow adoption partly to immature CAD.²⁴ Reconfigurability is lost after fabrication; only the NRE changes.

Read the gap numbers as a 2006-era, 90 nm, core-logic measurement. Architecture has moved since (6-input fracturable LUTs, larger logic blocks, hard NoCs, AI tensor blocks), and, as Boutros and Betz report from a later CNN-inference study, full-featured designs that lean on RAMs and DSPs measured about 9× in area rather than 35×.² The direction is stable: hard blocks close area and power, programmable routing keeps delay at a few times an ASIC’s. For the economics, Moonwalk’s mask table is the anchor; NRE components other than masks vary more by project than by node.³

• Efficiency versus option value. The FPGA premium buys the right to change the function later. Its value scales with the probability of change times the cost of a respin plus field replacement; it vanishes in regulated markets where every change needs re-approval.¹⁴
• Hard-block mix. The gap you pay depends on how much of your design maps to hard IP: Azure’s 2–3× for a NIC dominated by SRAM, transceivers and MACs, against 35× for pure logic.⁶¹ Unused hard blocks are pure overhead, which is why vendors keep specializing families (communications DSPs, AI tensor blocks).²
• Clock rate and timing closure. Programmable routing keeps fabric clocks a few times below an ASIC’s, so FPGAs scale by width and pipelining, not frequency. Brainwave ran at 250 MHz and made up for it with 96,000 MACs; the five platforms in Choi et al.’s study clocked their FPGA logic at 200–400 MHz.⁸²¹
• Lock-in and supply. Vendor-specific primitives (memory widths, DSP modes) tie RTL to a family; Azure found porting manageable when planned for.⁶ The part itself is a commercial product with its own end of life and supply chain risk.¹³
• The middle options have their own costs. eFPGAs add an FPGA tool flow and fabric area to an ASIC project; structured ASICs give up reconfigurability and have had limited adoption.²⁴

1. Break-even with change and risk

For options $i \in \{\mathrm{F}, \mathrm{A}\}$ with fixed cost $F_i$ and per-unit cost $v_i$ (unit price plus lifetime energy), add a design change that happens with probability $q$ partway through production. A change costs a fixed $R_i$ (a respin for the ASIC, engineering for the FPGA) and replacement of a fraction $r_i$ of shipped units at unit price $p_i$:

The sim sets $q = 1$ when the change switch is on, $r_{\mathrm{A}} = 0.5$ (half the units are already in the field), $r_{\mathrm{F}} = 0$, $R_{\mathrm{A}} = \text{masks} + 0.2D$ and $R_{\mathrm{F}} = 0.2 F_{\mathrm{F}}$. At its defaults ($10M design, 28 nm masks at $2.25M, $1M FPGA NRE, $200 versus $20 units, 10 W versus 1.5 W, 5 years at $0.15/kWh) the break-even moves from about 47,700 units to about 67,700. Mask costs are Moonwalk’s³; the rest is illustrative. In practice $q$ is the number to argue about: Azure’s SmartNIC case is $q \approx 1$ over a 7-year horizon.⁶

HOURS = 8760
def per_unit(price, watts, years, usd_kwh):
    return price + watts * HOURS * years / 1000 * usd_kwh

F_fpga, F_asic = 1.0e6, 10e6 + 2.25e6      # NRE: FPGA board/eng; ASIC design + 28 nm masks
v_fpga = per_unit(200, 10.0, 5, 0.15)       # 200 + 65.70  = 265.70
v_asic = per_unit(20,  1.5, 5, 0.15)        #  20 +  9.855 =  29.855

n_star = (F_asic - F_fpga) / (v_fpga - v_asic)   # 11.25e6 / 235.845 ≈ 47,700 units

# mid-life change, q = 1: ASIC respins and replaces half the fleet
F_asic_c = F_asic + 2.25e6 + 0.2 * 10e6     # + masks + 20% of design
v_asic_c = v_asic + 0.5 * 20                # replacements spread per unit
F_fpga_c = F_fpga + 0.2 * F_fpga            # new bitstream engineering
n_star_c = (F_asic_c - F_fpga_c) / (v_fpga - v_asic_c)   # ≈ 67,700 units

2. Why the gap compounds: area × delay

If an FPGA implementation is $g_a$ times larger and $g_d$ times slower, matching an ASIC’s throughput by replicating the datapath needs $g_d$ copies, so the silicon ratio at equal throughput is

which is Kuon and Rose’s “area gap of at least 119”, assuming ideal parallelization.¹ This is the right metric for throughput workloads (inference, packet processing), and it explains why FPGA accelerators compete only where their specialization (custom precision, no instruction overhead, batch-1 pipelines) recovers an order of magnitude or more against a programmable processor.

3. How much must map to hard blocks?

A simple mixing model (our arithmetic, not the paper’s): let a fraction $\phi$ of the ASIC’s area be functions that map onto hard blocks with gap $g_h$, and the rest onto soft logic with gap $g_s$. Then the area gap is

With $g_s = 35$ and $g_h \approx 1.5$ (Kuon and Rose assumed 1 for large memories and 2 for other hard blocks), reaching their fully-utilized estimate of $G \approx 4.7$ needs $\phi \approx 0.9$: about nine-tenths of the work must land in hard blocks.¹ That is the economic logic of adaptive SoCs, AI tensor blocks and hard NoCs: each moves a large, common share of designs into hard silicon so the fabric is reserved for the part that needs it.² It is also why a SmartNIC (mostly memory and I/O) sees 2–3× and a logic-heavy design sees 20× or more.⁶

Across the guide, each architecture removes a different overhead and gives up a matching freedom. CPUs keep full generality and spend area on single-thread latency (speculation, caches, coherence). GPUs amortize instruction overhead across warps and matrix instructions. AI accelerators hardwire dataflow and memory hierarchy to tensor workloads. FPGAs remove instructions entirely and pay the programmable-routing tax instead. CSET’s survey ranges for inference relative to a CPU at the same node put GPUs at about 1–10× in efficiency, FPGAs at 10–100× and ASICs at 100–1,000×, with generality falling in the same order.⁵ Chapters: 1–4 (CPUs), 5 (GPUs), 6–13 (AI accelerators), 14–16 (FPGAs).

The Systems guide takes the chip as given and covers what surrounds it: Packaging and chiplets, Board and server (PCIe, CXL and the cards SmartNICs sit on), Scale-up fabrics, Scale-out networking, Optics, parallelism and Power and cooling.