Why Heat Is Becoming A Bigger Chip Design Issue

Key Takeaways:

• Complex designs now require voltage, timing, resistance, power, and temperature to be analyzed together, and in the context of each other.
• Vertical scaling will require new materials and cooling options.
• Agentic AI could help engineers cross traditional discipline silos, which will be necessary because thermal is a system-level problem.

Experts at the Table: Semiconductor Engineering sat down to discuss the growing thermal challenges and trade-offs as dies are stacked, vertically integrated, and harder to cool, with Lang Lin, director of product management at Synopsys; Jack Berg, vice president of business development at Silvaco; Satish Radhakrishnan, head of semiconductor and electronics at Vinci; Chris Mueth, director of new markets management at Keysight EDA; and John Ferguson, senior director of product management at Siemens EDA. What follows are excerpts of that conversation, which was held behind closed doors at the recent Design Automation Conference. To view part one of this discussion, click here. Part two is here.

L-R: Silvaco’s Berg; Synopsys’ Lin; Vinci’s Radhakrishnan; Keysight’s Mueth; Siemens’ Ferguson.

SE: One of the big trade-offs in advanced designs is voltage versus current. How do you balance that?

Lin: If you reduce the voltage, it will change your timing and reduce the performance. But it also reduces heat. If you keep running the chip fast, you will run into a lot of thermal problems. These are conflicting physics. You need to find the sweet spot for your chip. You need to try the real combination of, say, this voltage versus this timing budget. And you need to do a loop of timing, thermal, and supply voltage or power integrity co-optimization. You have to close this loop and run a couple different workloads to make sure there is not a failure or any other problems. We don’t have a tool yet to do this big loop, so the engineer may be asking how to solve their place-and-route problem while also running timing, thermal, and voltage. And then, they still have to make sure they can close the design.

Radhakrishnan: Along with thermal, you also have the resistance change because of the temperature. You need to understand the dynamics of a temperature-based transient. As you pass the current, the temperature rises and the resistance changes. Are you going to keep the same performance? Are you going to change the total current. How much power do you budget? That’s where the coupled nature of the entire simulation is going, and that’s where a lot of focus in the industry will be. As power rises, temperature rises, so you’re going to allow more cooling, and you’re going to push up the power. All of this is going to be coupled, probably starting next year.

SE: So given all these issues, is a full 3D-IC possible?

Mueth: It’s just a matter of time. We’ll solve a lot of the problems. That will require a lot of materials science, which needs more investment. For the aging problem, we will use specialized materials science types of solvers, which need to be extended to achieve that. There are exotic things you can do, like stacked intercoolers with fluid circulation. Of course, that means it’s going to be incredibly expensive to make those, and you’re only going to use them for high-performance applications. You’re not going to use them for everything else.

Radhakrishnan: People are already talking about 3D DRAM and 3D-IC. From a thermal view, there are different techniques like a cooled interposer, but the industry is going more toward a 3D-IC where everything is stacked together and you have optics coming in on the side. It will be more expensive, though, so maybe you need to start with a narrow application.

Mueth: One thing that hasn’t been mentioned yet is heterogeneous technology, and how you can bring that in to use the right modal technologies for the right application and integrate that stuff together. That can help if you have a super high-performance application and you need a small size. You can use that for a specialty application to integrate the different technologies together.

SE: This goes into orchestration, where it’s not just whatever you’re doing as a process, but also the materials that you use, as well.

Berg: Absolutely. The materials are extraordinarily important, and the time constant associated with thermal is different from the speed of electrons through a transistor. That disconnect needs to be consistently taken into account. But for 3D-ICs, it’s not a matter of if we’re going to do them. It’s a matter of when. It will happen.

SE: For full 3D-IC, or 2.5D or 3.5D?

Berg: Well, you can call a stack of 16 HBMs in a single package a 3D-IC. So, depending on your definition, we already have it. But that z-level scaling is going to be very important, and it will take a group of very intelligent materials scientists, and people with thermal and mechanical skills to make that happen.

Radhakrishnan: Hybrid bonding is enabling that. More innovation, like having nanometer-based TSVs, will allow that.

Ferguson: The whole discussion is interesting. It reminds me of the early days of Moore’s Law, where every time we went to a new process, somebody would say, ‘That’s it. We can’t do it. We’re never going to get to the next node.’ But we get there. Maybe we don’t get there exactly when we want, but we do get there. And then we say, ‘Well, that was great, but now I need it to be faster. Now I need better signal intent.’ Now, as we bring in the MCPs (Model Context Protocols), it’s not just bringing in the intelligent people. Intelligent MCPs can do this so much faster than we can, and we need the people to find and build the right MCPs. One of the things we need to think about in this industry is that we’re looking at each thing almost in isolation. Here we’re talking about thermal. What are the thermal impacts? But in the meantime, somebody else is looking at changing the chemistry of some of the materials. And if you didn’t know that, they’re going to come together at the wrong time. We’ve got to think about bringing the whole ecosystem together so we’re all communicating and we know exactly when things are going to fit correctly.

SE: We lost that with the end of the ITRS, and now we’re looking to AI to pick up the mantle.

Mueth: Yes, and I’ve been surprised at how much AI has advanced in the last year. It’s astounding. We had a hackathon a couple months ago. We were doing an amazing number of things with the tools we never thought were possible. If the customer knows the problems they have to solve, it’s our job as vendors to supply the tools to help them out. But in the end, because they know what they need to do, they’re going to want to drive the orchestration, and we need to enable them to do that.

Lin: This becomes a very interesting engineering regrouping, or upgrading of the organization time. In the old days, we would have thermal engineers solve the thermal problem, and the semiconductor team wouldn’t look at it. ‘Just send the data to the thermal team, and they will resolve any issues.’ But today, even the place-and-route engineer has the luxury to deal with the thermal problem because we have chip agents. So they can use natural language to tell an agent, ‘I want to see if this design passes thermal. Show me the T_max. The agent will help you run the simulation and give you back the result.

Mueth: Traditionally, everyone trained on one discipline, so you had single-domain engineers. But they really need to do multi-domain problems now. How do you accomplish that? One way is through AI. Another is to dumb down the offerings of multi-physics, such that a person in a single domain can run those simulations. But AI would be a big help.

Radhakrishnan: AI is good, but it has to be accurate. That’s the most important thing, because while a designer can run it, if he’s going to make a decision using the AI and it’s wrong, it will lead to a wrong design. But I agree that it needs to go beyond just a specialized simulation team. It needs to involve other people, and you have to make it easy for them to get the geometry and simulate it very quickly so they can get the design right. And, of course, it needs to talk to other teams where mechanical, electrical, and everything else are combined. This has to be done correctly, accurately, and together.

SE: As we move toward agentic AI, you need more than just GPUs. You need CPUs, and probably different types of accelerators. Does that add to the thermal issues, or does it make it easier to manage them because they’re just one more variable rather than trying to make this a single thing?

Mueth: It’s hard to manage two different technologies, obviously. But what you’re gaining is that you’re pushing multiple technologies to gain efficiencies. If you gain efficiencies, you gain thermal performance, and hopefully electrical performance. So it’s a trade-off of design complexity against performance of an electrical circuit — and the thermal performance, as well.

Berg: I generally agree. Having multiple solutions at this point for both training and inference is important, and it’s an important consideration to make things a bit more effective and cost-effective.

Lin: For traditional EDA tools, we’re typically running CPU-based acceleration. But when we go into the multi-physics domain, we see GPUs playing an important role because there is a lot of 3D solving. You can parallel the matrix solver with a GPU kind of load. That’s another important use of GPUs — to accelerate the tool simulation. This is runtime improvement and parallel processing.

SE: So massively parallel systems?

Lin: Yes, with thousands of GPUs running all together to solve partition-based design of your 3D-IC. The CPU is going to be orchestrating and doing some kind of reasoning, and then driving the process.

Berg: In order to create synthetic data with multiple physics solvers that are cross-coupled together, you need a very large bank of GPUs. Once you have that synthetic data, you’re capable of coming up with surrogate models that can be run on CPUs on somebody else’s PC to take a look at ‘what ifs.’ The key is ensuring that your multi-physics is calibrated to data, and that it is also capable of ensuring that you have boundaries where interpolation is occurring, versus extrapolation.

Read parts one and two of the discussion:
Scaling Thermal Analysis From Transistors To Data Centers
Rising power density and chiplet complexity drive thermal considerations earlier in the design process.

Thermal Complexity Grows With AI Chips And Photonics
Increasing power density makes heat, stress, and aging harder to predict, and more critical to address earlier in the design flow.