Test Struggles To Keep Pace With AI

Key Takeaways:

• AI workloads are changing faster than chips can be designed, manufactured, and tested, forcing chipmakers to rethink test strategies.
• More complex packaging and additional test insertions are generating vast amounts of data, much of it siloed across companies and facilities.
• Chipmakers need better ways to monitor aging and reliability as customers push to extend the life of increasingly expensive AI hardware.

The rapid pace of change in AI models and approaches is creating confusion over what to test, when to test it, and what to monitor once chips are in the field.

Until recently, test targets and the timing for test insertions generally were determined by fab processes and the laws of physics, and the results of those tests were deterministic and quantifiable. But starting with the rollout of generative AI and the ensuing explosion of data centers, the gap has widened between the rollout of new algorithms and use models and the time it takes to deploy a chip in the field.

An increasing number of high-performance processors used in data centers, and many at the edge, are multi-die assemblies. The chips are more complex and take longer to design, manufacture, package, and assemble, and the workloads for which they are designed often change by the time those chips leave final test. Along the way, there are more test insertions along the way, each of which generates more data.

The tests are more complex and varied, as well. In addition to electrical tests, these chips need to be analyzed for mechanical integrity. AI workloads can stress one part of a multi-die assembly more than others, and they can change quickly as algorithms and use cases evolve. That, in turn, requires increasingly complex multiphysics simulations and more real-time monitoring of processor and memory utilization, thermal gradients, and datapath throughput, along with accelerated aging that can affect signal integrity.

“It’s a huge challenge,” said Scott Segura, vice president of sales at Advantest. “People are starting to put more active components into the substrate, and moving things closer to each other, as well, so it’s turning into more test insertions. What shifts left? What shifts right? There are substrate testers today that are just trying to make sure you have contact, but now we’re seeing functional testing moving into substrate testing. And at the die level, we’re probing performance. We’ve got to control thermal. For known good dies, they have to be able to handle 3 kilowatts at the die level, so now we’re testing singulated dies. There’s going to be a shift in how you slice up where things go, and packaging is the key. As you’re putting more and more stuff in the package, into the substrates, whatever you’re stacking this stuff on, you’re going to have to start testing those different steps. Silicon photonics has five insertions right now.”

It only gets more challenging from here with the transition from wafers to panels, which will require entirely new equipment and processes. At the same time, packaging and assembly, which used to be done in one location, is now being split between fabs and OSATs. That creates ongoing challenges around who sees what data, when they see it, and how much of it can be shared with other companies.

“Test insertions typically are done at different locations,” said Jeorge Hutarte, senior director of product marketing in the Semiconductor Test group at Teradyne. “For example, from wafer sort to singulated die probe could be done at a fab. They have their data collection workflows. They have their own test programs. And they gather failure mechanisms as a result of the testing done at wafer sort and singulated die probing. When it comes to the next step, which is where we put together a compute die plus a stack of HBMs, plus perhaps other chiplets into an interposer — and it’s going to panels today from round wafers to optimize the utilization of equipment and putting as many dies as possible on a 310 × 310mm panel — this wafer/panel interposer substrate test will likely be done by the OSAT. That’s a different location with a different test program and a different data collection environment or workflow. They don’t necessarily talk to the previous test insertion data. And unfortunately in the fabless world, people don’t share data unless the customer owns everything.”

Fig. 1: Evolving test strategies and emerging test insertions. Source: Teradyne

Testing gets more complicated
It took slightly more than two years from the introduction of ChatGPT in November 2022 to widespread commercial adoption of agentic AI systems. That’s roughly the time it takes to design, verify, and validate a chip, get that design onto a shuttle in a fab, package it, and integrate that chip into a functional system. But during that time, the industry shifted from arrays of GPUs for massively parallel multiply-accumulate operations to a combination of GPUs, CPUs, and customized and programmable accelerators for orchestrating how agents work and interact. Those involve very different optimization and prioritization schemes — hierarchies that are constantly being re-evaluated as workloads shift.

Hyperscalers and large chipmakers have been pushing EDA vendors to reduce the time it takes to design these mini-systems. But even if the chip design cycle could be shrunk from today’s 18 to 24 months to as little as 6 months — a stated goal by some large chipmakers — there would still be a significant gap between the time it takes to get to working silicon and the time it takes to roll out new training and inference models. And that could change again if recurrent self-improvement lives up to its promise.

“We have customers now that will tape out chips, run their models on them or look at what the end application is, and they’ll end up not going to market with them,” said Advantest’s Segura. “They’ll do a derivative of it, or they’ll do a different version of it. That could change how much memory they’re stacking on it, how many cores it has, how much cache it has, and the overall architecture. It totally changes depending on whether you’re training a frontier model or whether it’s a CPU hosting something. Even our customers sometimes guess wrong. They go through a chip development cycle, get the silicon out, start to run models on it, and they find out the use model has shifted, or they didn’t put enough memory in, or it’s too expensive. They’ve got to be pretty nimble about what they develop, as well. You see a lot of chiplet designs because it lets them scale a different number of cores, for example. If they originally planned for 256 cores, they may only need 192.”

Breaking down data silos
The reasons underlying the gap between hardware and software schedules are complex. Algorithms can be updated in hours or days. Chip architectures can take years to optimize. The goal is to make every process step bulletproof. Achieving that requires more testing, more data, and a clear plan for making sense of that data.

“Not only are there more tools, but they’re all generating more data — and pretty rich modes of data with lots of images,” said Jim Schiely, chief of staff for Calibre Semiconductor R&D at Siemens EDA. “Over time, customers had been building their own local systems for managing revisions, trying to develop context, because you can’t just have a bunch of models floating around without knowing what the provenance of all that data is.”

In some cases, this involves multiple companies. In others, it involves different groups within the same company.

One big challenge here is breaking through silos that have been controlling this data. “There are two aspects to that,” said Schiely. “One is on the technical side, and the other has to do with the relationships between the entities that control that data. On the technical side, the good news is that it’s sort of AI to the rescue. Waiting for everybody to define the format is going to take a long time. But you can very easily take these digital threads of information and turn them into a model or representation with the help of AI tools, even if there’s a slightly different data model being used by another department. That stuff can be handled more or less automatically. So that’s an area where some of the old problems might start to dissolve.”

Battles over who controls critical data intensified in the 1990s, when manufacturing began shifting from vertically integrated companies to foundries. To get designs to work, the foundries instituted restrictive design rules. Those rules have proven very effective for improving yield, but they also limit design freedom. The so-called design rule deck also has become thicker at each new process node, and the amount of data that needs to be processed and analyzed by different groups has ballooned.

“This needs to go from historical analysis done by a group to analytics that are site-wide or maybe group-wide to now being effectively networked analytics either across your own supply chain, or collaboration with your partners,” said Kimon Michaels, co-founder and executive vice president of products and solutions at PDF Solutions. “We’re driving toward how you can have a standard platform that enables the efficient sharing of data and pulling it together and reaching into different facilities, but the obfuscation of the protection of the IP. We’re not there yet as an industry for sure.”

The orchestration required by agentic AI has further blurred the lines of what belongs to whom. “The complexity in our industry is beyond coming up with the latest, greatest model,” said Michaels. “It’s about dealing with the geographical dispersion across the customer-vendor relationship to manage and collect and understand what model, what data has to be at what place at the right time.”

This matters because chips are increasingly becoming assemblies of heterogeneous components.

“You have active components sitting on a substrate that are very difficult to access, so now they’re building BiST into it,” Segura said. “They’re looking at how different dies are talking to each other. And not just dies. You could have inductors, capacitors, or some type of voltage regulator. How do we self-test that, because it’s really hard to get access to that on the substrate level? All of that’s changing how you’re going to test when you don’t have access to it. Some of these dies have CPUs in them, so they already have built-in self-test for burn-in. But in cases where you can’t get access, either you don’t test it, or you use something that’s connected to it. It’s getting more and more complicated, and people are making this up as they go.”

That was considered business as usual when developing a chip was relatively inexpensive, and field failures were manageable. But in a complex collection of chiplets with HBM memory running mission-critical or safety-critical systems, those costs can quickly spiral out of control. Avoiding them requires more simulations, more testing, and more analysis of live systems.

“In some cases, because of silent data corruption and other potential defects, you may want to do system-level tests, as well,” said Teradyne’s Hutarte. “Some companies are asking to insert between wafer sort and singulated die a wafer-level burn-in stage, where you can accelerate infant mortality before you actually commit it to the interposer as a stack-up. What’s changed in the AI accelerator world, as compared to the mobility era, is that every wrong screening translates to thousands of dollars of loss because of failures at the data center. So the trend is toward more and more shift-left screening and added insertions. The test intensity has at least doubled over the AI era, because once they hit the rack assembly, they could be worth $30,000 to $50,000. We cannot throw away $30,000 because of a defective chiplet.”

Longer lifespans, changing workloads
As a result, chipmakers are increasingly looking at rework rather than scrapping dies or full chips, and their customers are looking at extending the life of these chips.

Some of this can be improved with more data, including synthetic data that is being used where real data is unavailable. But more data also requires more planning as to how to use that data effectively.

“One of the new trends involves workload scheduling,” said Rahima Mohammed, distinguished engineer and semiconductor technical advisor at Vinci. “How do you move the workloads in a way that you don’t heat it up as much and keep it safe? You also need to really think through all the applications very carefully.”

Efficiencies can be gained at multiple points in a system-level architecture, whether that’s access to memory, understanding which data needs to be processed in real-time and how to parse the processing across a system to ensure that, and how and where to reroute data if data pathways become slower due to electromigration. The list goes on, but all of these decisions also need to take into account structural changes inside customized packaging, the degradation of dielectrics, and the effectiveness of the cooling architecture, any of which can impact how a system behaves over time.

“When you think about chiplets and the smallest processes, how do the transistors come together? Where is the self-heating coming from? You have to think about this from the nanometer level all the way up to the package level,” Mohammed said. “With chiplets, how do you bring those into the package? In a previous time, PCB packaging was just about adding layers. You didn’t have to think about all the copper lines or vias. We just kind of winged it. You don’t have that option anymore. You have to get power through all the possible layers that you can think about, whether that’s horizontal, vertical, lateral, and you have to do all these ‘what ifs.’ If I change the location of the metal lines here, what happens?”

Conclusion
Test is no longer a fixed step in chipmaking. Historically, test represented a small percentage of manufacturing costs, and test schedules were more predictable. But as leading-edge chips become increasingly heterogeneous collections of chiplets and other components in a customized advanced package with high utilization rates and very specific applications, test is no longer cookie-cutter. It is a unique process that is tailored to a specific design, and one that is increasingly costly and time-consuming. On top of that, the use case for these chips is changing so quickly that by the time a chip is fully tested and ready for work, it may already be outdated.

AI may be used to design better AI chips and help integrate data from various processes such as test, but keeping up with the changes within AI is a much tougher challenge. And until the technology settles into a pattern of use that isn’t uprooted every month or so, the chip industry is likely to face some hard choices about how much customization is needed to remain competitive, and how long it’s expected to last.