Introducing Build-A-Chip-Bench: From Workload Specification to GDS

AI introduction

ChipAgents, together with Nvidia, has developed the Build-A-Chip-Bench benchmark, which uses open-source tools and an open-source PDK to evaluate the ability of AI agents to complete the entire chip design flow from workload specification to GDS layout.

Article · English

Evaluate AI agents from workload specification through GDS layout, using open-source tools and an open-source PDK to assess their ability to complete the chip-design workflow.

popularity

Existing chip design benchmarks test isolated steps, such as writing RTL from a specification, and are increasingly saturated. ChipAgents is developing Build-A-Chip-Bench with contributions from Nvidia to the benchmark suite and scoring rubric. The benchmark evaluates AI agents from workload specification through GDS layout, using open-source tools and an open-source PDK to assess their ability to complete the chip-design workflow. Each design must pass a hard gate, covering functional correctness against a hidden verification IP as well as routability, hold timing, and DRC, and designs that pass are scored on PPA against the task’s constraints. The best agent passes the hard gate on 65% of tasks and reaches an overall score of 16.6%. Frontier agents struggle to revisit earlier decisions, protect working behavior while optimizing, and plan around slow tool runs.