RISC-V 用于 AI：是突破还是坏主意？

Key Takeaways:

• RISC-V’s role in AI is contested. Its customization offers flexibility, but splitting inference workloads between CPUs and accelerators can introduce costly data movement and compiler complexity.
• Control may be RISC-V’s best option. Custom instructions can help orchestrate accelerators, reduce load/store latency, and handle limited mathematical tasks without taking on heavyweight inference.
• Correct instructions don’t guarantee a working chip. Custom AI systems also need observability and system-level verification to expose performance bottlenecks, cache-coherency problems, and interconnect deadlocks.

Experts At The Table: Demand for parallelism and power efficiency is shifting chip design from general computing to AI, increasing the need for flexible, modular, and open hardware. This trend is bringing RISC-V into AI acceleration. Customizable hardware can implement AI computations directly instead of relying on fixed CPU designs. Semiconductor Engineering sat down to discuss these issues with Steve Roddy, chief marketing officer at Quadric; Francisca Tan, director of product management and product engineering for Tessent at Siemens EDA; Ewald Liess, senior vice president of sales and business development at SignatureIP; and Dave Kelf, CEO of Breker. What follows are excerpts of that conversation.

L-R: Quadric’s Roddy; Siemens EDA’s Tan; SignatureIP’s Liess; Breker’s Kelf.

SE: With AI models evolving rapidly, architects use RISC-V to design highly specialized extensions. From an IP provider standpoint, how would you balance delivering a standardized, reusable IP core with the extreme customization that customers are demanding for AI workloads?

Leiss: We don’t think that we need to configure because we provide NoC and PCI Express and CXL CHI AMBA interfaces. Based on this, we don’t need to customize. Instead, it’s the NoC that will be configured. Because we have a pre-verified NoC, we can do that with our tools. We can configure customized RISC-Vs. Of course, the customers may modify the RISC-V core with new instructions, but they will stick to the interfaces like the AMBA bus, or CXL, PCI Express interfaces, because this enables them to access the external components that they need, like memory or going off-chip. And this is where we come in. So the RISC-V guys should feel free to do whatever they want.

Tan: Custom verification and validation also become important, as does observability. For test and embedded analytics, engineers doing debug or silicon bring-up can monitor, introspect, and inspect internal signals, or calculate whether performance is being met. In my view, it’s also about hardware observability, as the industry moves into various challenges and complexity in the chip.

Roddy: We’ll take the contrarian view in this particular area. We think that starting with RISC-V to do an AI accelerator is a flat-out bad idea, because you instantly create a problem. You now have a partition between some things running on a programmable processor with good ALU capacity, and some external accelerator where you’ve now had to partition the system. The minute you partition, the minute you create a problem, you need fancy compilers to go fix it and figure out how to move data from the accelerator or accelerators across a nice NoC over to a programmable core, and back and forth. That’s a problem that scales badly. The bigger the accelerator, the worse it gets. And so now you’re compounding the problem. If you have something that’s, say, 100 TOPS and you have to have, say, 16 RISC-V processors slave to a whole bunch of accelerator blocks, sure, you need great interconnect, and you need great verification, and all those things that go with it because you’re trying to solve the underlying fundamental problem, which is starting with a CPU. And CPUs are great for chasing pointers. That’s what CPUs do. CPUs have wonderful logic to handle the fact that you’ve got 40 threads running, and on the very next clock tick, you have no idea what you’ll be running. That’s not what you’re doing when you’re doing AI inference. You know for millions of cycles in advance what’s happening. You don’t need most of the CPU hardware. DSPs are a little closer to what you need, but DSPs are vector machines — vector-by-scalar or vector-by-vector. And most of what is challenging in AI is matrix-by-matrix. So we built a matrix-native machine. The only other one I know of that’s close to that is in the data center, and that’s the big giant TPUs from Google. Literally, everybody else has a partitioned problem, and then they struggle and spend billions of dollars trying to handle that partitioning problem and figure out how to make up for the fact that they started from, we think, a bad place.

Kelf: I agree with Steve’s point, because what we’re seeing is large AI accelerators — which might be different architectures, SIMD, SIMT — are pretty common. These kinds of things basically are controlled, or have been controlled, by a complex state machine of some sort, and then there’ll be some kind of host processor driving several of these different things and telling the state machine what to do, and then driving the accelerator. What we’ve seen in four instances now, both big and small companies, is that the state machines are hopeless. So they’re taking a RISC-V processor, and they’re using that as the control function on top of these accelerators for a number of reasons. One is that they can, in fact, build in a bunch of mathematical engines — exactly what Steve was mentioning — vector type, using either the RISC-V vector instructions or those. And then they add in their own, as well, or some of the new matrix multiplication instructions. They’re also putting in their own instructions for very efficient load/store to reduce the latency that you see between partitions. So they’re trying to figure out how they can adjust these RISC-V processes to reduce that latency and to reduce that partitioning problem. We have a number of examples where some are successful, some are less so. They’re still working on this problem, but RISC-V is nice because they can add whatever custom instructions they want, still use a regular compiler, and create a mechanism to drive a very efficient load/store that they can’t with other processors. Then, of course, there are people like Arm and others who have built instructions into some of their processors, but they’re not variable. RISC-V is ideal for this, and pretty unique in its ability to create these powerful custom instructions.

SE: Regarding those custom vector instructions, are EDA tools accounting for this and automating the verification?

Roddy: I spent 17 years at Tensilica. We automated all that. The downside, of course, is that you pay a royalty. RISC-V is an opportunity to not pay a royalty, a tribute, a tax to the big guy. If you want customization, the folks at Tensilica automated that many years ago. The idea is to change instructions and instantly get compilers and simulators that match, with verification handled because it’s in a structured environment. RISC-V gives a bit more flexibility to go outside those bounds. My take on that is if you really want to use RISC-V, you better be going all out and customizing the hell out of it to go through the effort to manually create all that stuff, because if you want just a little bit of customization, my old friends at Tensilica have an automated system that makes that happen. Yes, it comes with a commercial cost. RISC-V is a very interesting technology, and it absolutely serves a purpose. The market leader doesn’t allow that level of flexibility for control and CPU functions. You can get a lot of value out of tweaking and tuning a processor for the control function at hand. We just don’t think it makes sense to do inference on it. But there are thousands of things that CPUs are used for, and RISC-V CPUs can do those things perhaps more flexibly and more efficiently, and certainly at a lower price point than fixed-function CPUs from other vendors. We just don’t think doing heavy lifting and inferencing is the right way to deploy that kind of technology.

Kelf: I tend to agree. Some companies are trying to build in the vectors and build their own mathematical sort of DSP-type instructions. I was talking with some customers who were having some issues. They had a SIMT array — single-instruction, multiple threads — and they were doing a ton of parallel mathematics and hoping one of the parallel lanes wasn’t going to diverge from the others. And then one does. The nice thing about SIMT is you can make it diverge. They were using the RISC-V control processor to do some of the mathematical processing from the divergence, leaving the rest of the accelerator to do the rest of the mathematical processing that’s going on. They’re not doing heavyweight processing inside the RISC-V. They’re just doing enough to keep the whole machine working and reduce that latency, but doing this odd thing inside the RISC-V. It took them a lot of work to get this working properly, but now they are seeing this as quite successful. I agree that there’s the business side of RISC-V, which is you don’t have to pay a royalty, and I agree on the Tensilica technology as well, which was fantastic for this kind of stuff, especially DSP communications. But they can do similar things inside RISC-V. They have to build it themselves. They have to know how to do that. But once they do, there’s a lot of flexibility, a lot of standardization around RISC-V. They can get a compiler. They can potentially shift to a different company and build some of their proprietary stuff into that RISC-V. That does allow for more flexibility, but more standardization, as well, which you may not get out of a fixed processor and a proprietary processor.

Tan: In terms of what testing and better analytics provide for custom instructions, especially from the RISC-V arena, we support debuggability for custom-instruction tracing. We support RISC-V users specifying which custom instructions they want to trace while the processor is running, to support that debuggability environment. We are also one of the members of the RISC-V organization, whereby we donated a few years back the Efficient Trace (E-Trace) specification. That is a highly compressed instruction trace catering to complex systems where there’s going to be a lot of data, and therefore we need to think about compressing all this data as well. Within our infrastructure, it’s not just about the processor. It’s about the entire system. So our infrastructure can capture instruction traces from the RISC-V core, as well as from various parts of the RISC-V SoC, such as the interconnect, bus monitoring, or specific signals we want to monitor.

Leiss: We’ve talked a lot about verifying the custom instruction set, passing the ISA-level verification, and most of the time, maybe generated IP comes with verification collateral like test benches, traffic generators, and performance models. But I look at it from a system point of view, and it means that there are typically multiple cores in one high-performance system, and when you have an accelerator in RISC-V five, or you have several RISC-V in one, then you immediately end up with a coherency problem because you use caches and so on. The coherency corner cases and traffic bursts can easily cause deadlocks in the fabric. It is therefore important that, from the NoC side, we do all this verification at the system level to check how the traffic can flow through and make sure that you don’t end up in a deadlock situation. In simple terms, an instruction can execute correctly, but the chip can still fail if the data burst needed for the instruction or the calculation isn’t available. That’s an area we’re addressing.