Why Physical AI Is So Hard To Engineer

Key Takeaways:

• Physical AI is the deployment of machines with autonomy, acting within the real world. It is not a single product, but a philosophy built on modular architectures.
• These systems can only be developed with a full systems approach, integrating sensors, actuators, and distributed compute systems, in a multi-physics methodology.
• Real-time constraints become much more important than in the past, and at the same time much more complex.

Physical AI is changing how chips are designed and used, integrating sensing, AI inference, and real-time control into machines that can safely and predictably interact with the physical world.

The challenges extend well beyond just performance. Engineers need to account for corner cases stemming from unpredictable environments, ensure these systems are safe and reliable, and they need to future-proof these systems by ensuring they are serviceable and upgradable. This will require new tools and methodologies, as well as some new design approaches.

“AI is leaving the screen,” says Anthony Matarazzo, head of physical AI, digital twins, simulation-driven AI, and accelerated computing at Synopsys. “It is no longer about generating text or code. It is entering the physical world. It needs to understand, interact, sense, make decisions in a physical environment. We are moving from generative AI to agentic workflow orchestration to physical AI (see figure 1.) That is literally making intelligence actionable. But the consequences of being wrong fundamentally change. Life or death, plus or minus 1%.”

Fig. 1: Three Eras of AI. Source: Synopsys

Physical AI must see and interact with the real world. “It is challenging to make artificial intelligence see the world correctly,” says Chun-Ting “Tim” Wang-Lee, signal integrity application scientist at Keysight EDA. “Real-world information comes from the sensors, and the data has to be moved somewhere and processed so that the AI can decide and then act. There are decisions that need to be made.”

This changes the design priorities. “Physical AI usually gets framed as a compute story,” says Justin Endo, director for marketing and sales at Mixel, A Silvaco Company. “But from our perspective it has really been an imaging story. Every robot, drone, and autonomous system comes down to cameras feeding an inference engine.”

Physical AI builds on some of the connectivity and sensing technologies developed for the Internet of Things. “IoT was largely about connecting and monitoring things,” says Brad Rex, head of physical AI, UX at Renesas Electronics. “Physical AI is about giving machines the ability to perceive and act autonomously in unstructured environments. This is a completely orthogonal engineering problem, and it requires a step-change in system-level content — more sensing, more real-time compute, more sophisticated power management, and more safety considerations per device.”

Within physical AI, there are two distinct classes of devices: “Those that can rely on a cloud connection and those that must operate fully locally without relying on connectivity,” says Ashley Stevens, director of product management and marketing at Arteris. “For example, vehicles cannot rely on continuous connectivity, as there are many locations where cellular coverage is inadequate. In contrast, an industrial robotic arm moves but is constrained to a particular location within a factory setting, and therefore can be connected to the cloud.”

Others agree. “Unlike traditional AI applications that primarily generate information, physical AI systems influence real-world outcomes through continuous interaction with their environment,” says William Chen, product marketing group director for design IP at Cadence. “Traditional IoT often focused on collecting and transmitting data. Physical AI adds real-time interpretation and action. The result is a shift from connected devices to intelligent systems operating within sensing and control loops.”

This requires new architectures and new methodologies. “The entire architecture is very different,” says Satish Radhakrishnan, head of semiconductor and electronics at Vinci. “Time-to-market is going to be much slower because people are still learning. Physical AI is the next step that they want, but the switch to it is not going to be easy. The easiest entry point will be a repetitive action, but people are not willing to pay much for that. They want it to do a lot more. That’s why it is going to take a more complex architecture to build, make reliable, and make it work for at least a day on a single charge.”

It also requires significant processing power. “Robotics and drones require powerful, efficient compute capable of multimodal processing and accurate, real-time responsiveness in dynamic settings,” says Sameer Sharma, associate vice president for MediaTek’s IoT business. “Because of latency and bandwidth constraints, these devices require silicon capable of handling AI workloads at the edge, including AI acceleration for machine vision and object classification.”

As systems get larger or more capable, additional challenges emerge. “You need longer reach through moving joints, aggregation of many cameras into few ports, and the functional safety expectations coming over from automotive,” says Mixel’s Endo. “But the standard is already moving in that direction. The mobile camera interface is becoming the robot’s optic nerve, and that mobile-proven maturity is hard for any alternative to match.”

How quickly physical AI is adopted by different markets will vary, but the overall projections are noteworthy. At a high level, PwC estimates the global physical AI market will reach ~€430 billion by 2030, with “significant upside potential as systems scale and mature.” Deloitte, meanwhile, found that 58% of companies report at least limited use of physical AI today, with that number projected to rise to 80% within two years.

Not all sectors will adopt this technology at the same time. “Physical AI is not just AI at the edge,” says Frank Schirrmeister, executive director of strategic programs for system solutions at Synopsys. “It is broader than just robotics, but it will be uneven. The question is how quickly each industry will adopt physical AI. It requires you to go beyond the chip to the full physics. The growth inflection comes when you have this full loop of sensing, reasoning, physical action, and then autonomy. It’s really a platform transition where everything is being elevated up to the system level.”

New markets have repeatedly driven changes in the semiconductor ecosystem. “We believe physical AI is one of the most disruptive AI opportunities of the next several years,” says Sanjay Gupta, co-founder and president at Velaura AI. “We see it as a category of its own, rather than another set of IoT products. Industry estimates vary, but several point to a multi-trillion-dollar total addressable market over the next 10 years across robotics, drones, industrial automation, logistics, healthcare, and defense.”

In many cases, it will be market pressure that dictates when a sector advances. “The clearest near-term applications are in industrial and logistics settings,” says Renesas’ Rex. “Humanoid and non-humanoid robots handling repetitive or physically demanding tasks in factories, warehouses and supply chains, where labor shortages make automation valuable, and the environment is structured enough to deploy safely today.”

While many point to safety and security concerns, there are multiple ways to look at that. “Physical AI is enabling new ways to make essential work safer, more efficient and more productive,” says Robert Day, director of go-to-market, physical AI at Arm. “We’re already seeing autonomous machines move toward practical work in manufacturing, logistics and transportation. These industries represent an estimated $200 billion annual compute opportunity in the 2030s.”

Deployment in some sectors will likely be slower than others, but the overall industry could show rapid growth. “Each category is going to be separate and see slow growth, but there will be many sectors growing in parallel,” says Vinci’s Radhakrishnan. “Each requires new-age AI, new compute, and this will impact the speed at which it’s going to develop. One area may take longer to grow deep, but before it fully penetrates one market segment, it will enter multiple new market segments. Together it’s going to grow.”

But they all require some advances in areas such as training and simulation before that happens. “We know that the Internet trained generative AI, but what’s going to train physical AI?” asks Synopsys’ Matarazzo. “This is the big question that companies are asking. Reality has rules.”

Modularity
Physical AI systems are not like mobile phones that quickly get replaced when a new model comes out. Cars, robots, and drones have much higher initial cost, and they are expected to last many years. This indicates that many physical AI systems will need to be updatable, not just in terms of software or even AI models, but hardware, sensors, and actuators, as well.

Physical AI systems will continue to evolve as models and use cases change. “A consistent compute foundation across development and deployment can help teams carry software forward instead of rebuilding individual components for each generation,” says Arm’s Day. “Turning the opportunity into growth will ultimately depend on making autonomous machines efficient, responsive and reliable enough to deploy at scale.”

“Autonomous vehicles have been there for some time, but the older ones can’t be retrofitted to the newer techniques, camera sensors, etc.,” says Radhakrishnan. “The design of physical AI needs to go into some form of modular systems. Maybe you have a compute system that can be taken out and replaced, or a new sensor. People may be okay with paying 10% of the original cost every two or three years to update certain things, and the software updates keep coming in. This is what is going to keep refreshing this.”

The ability to do this needs to be addressed at the initial design stage. “Given how quickly robot architectures are evolving, we design for headroom rather than attempting to predict a fixed end state,” says Rex. “That means greater compute margin, scalable power architectures, and modular subsystem designs that can absorb additional sensing or intelligence without a full redesign. We are developing next-generation devices and evaluating how to optimize across multiple vectors such as power, compute, scalability, and efficiency to determine the right balance. We also rely on distributed architectures, so that as capability needs grow, it is often a matter of adding or upgrading a subsystem rather than replacing the entire platform. That approach, combined with software and reference-design platforms that can be updated over time, helps customers avoid designing themselves into a corner.”

More future-proofing is required than in many previous embedded or edge systems. “Physical AI deployments often have long operational lifetimes, yet must accommodate evolving AI models, new sensors, changing workloads, and increasing autonomy requirements,” says Cadence’s Chen. “Therefore, future-proofing is less about maximizing raw performance and more about building architectural flexibility. Multi-die architectures, scalable memory subsystems, high-speed interconnects, and standards-based interfaces provide mechanisms for adapting systems as requirements change. The challenge is that many infrastructure decisions become difficult to change once they are embedded into the system. Memory architecture, communication fabrics, and interface choices frequently determine the practical limits of future expansion.”

Real-time constraints
While real-time constraints have been a fundamental aspect of many embedded systems, physical AI will take this to a much higher level of complexity. It may not be possible to make many of the simplifications used in the past, such as not using cache memory because it introduces non-predictable response times.

The scale of the compute problem alone is unprecedented. “For chips designed for physical AI, the biggest limitation is predictability under real operating conditions,” says Alex Burlak, executive vice president of engineering and customer success at proteanTecs. “Consider a humanoid robot. It may have a powerful central compute system serving as its ‘brain,’ but throughout its body there can be dozens of MCUs and other devices controlling motors, joints, sensors, power and communications. A timing issue or degradation in one of those devices may not simply mean an application runs a little slower. It could mean a joint responds differently than expected, a sensor input arrives late, or an actuator behaves outside its intended parameters.”

Translating a system’s understanding of its surroundings into precise actions is complex. “Physical environments are unpredictable, and sensor data can be noisy, so systems need to handle uncertainty and continue adapting as conditions change,” says MediaTek’s Sharma. “That requires a software stack that connects perception, navigation, decision-making, and control, and supports the application-specific models, runtimes, and tools.”

This requires a different focus. “Traditional AI infrastructure is often optimized for throughput, utilization, and scale,” says Chen. “Physical AI systems must satisfy latency, power, reliability, safety, and environmental requirements while responding to continuously changing inputs. A correct answer delivered too late may be operationally equivalent to a wrong answer. As a result, physical AI introduces a different optimization target. The goal is not simply maximizing compute performance. It is maintaining predictable system behavior across sensing, memory, compute, communication, and actuation.”

It becomes a more complex optimization problem. “Physical AI systems need to process sensor data, and reason and act in real time within tight constraints on size, power, and weight,” says Day. “They require more than capable AI models. Hardware and software must be designed together around these workloads, from low-power sensing and real-time control to demanding AI processing.”

The whole system becomes a part of the timing constraint. “This is a prime example of where hardware/software co-design matters most,” says Rex. “The complexity across multiple domains will become even more significant going forward, spanning not only electronics, but mechanical design and battery technology, as well. Many of these disciplines, which were once addressed somewhat independently, now represent a larger challenge because they must be solved concurrently rather than in isolation. Because physical AI systems have to reconcile AI inference with deterministic, safety-critical control, the silicon and the software cannot be optimized separately and expected to integrate successfully. The system architecture, the real-time software, and the underlying compute and power hardware all have to be designed together from the outset.”

This methodology change has been brewing for a long time. “Physical AI arguably represents one of the strongest cases for hardware/software co-design,” says Chen. “Control-loop behavior emerges from interactions between sensing, memory access, inference execution, communication, and actuation. Improving software alone cannot overcome infrastructure bottlenecks, and faster hardware does not automatically improve system behavior if data delivery, synchronization, or timing remain constrained. The architecture must be considered end-to-end.”

What does real-time AI even mean? “Will AI always be fast enough to get this control back to the actuator?” asks Synopsys’ Schirrmeister. “The engineering challenges are behavioral. Get those real scenarios done, then deal with the uncertainty of making sure that it’s not leading to catastrophic failures. Hardware software will become even more critical. You really need to be flexible in being able to upgrade and then also, for safety reasons, stop where you can’t go.”

The challenge is not simply building faster chips or more capable AI models. Physical AI systems must coordinate sensing, computation, and actuation while remaining predictable under changing operating conditions. That will require new approaches to system architecture, hardware/software co-design, and verification, particularly as these machines become more autonomous and remain in service for longer periods.

Editor’s note: Future stories in this series will examine how physical AI systems learn and adapt, how engineers account for aging and degradation, and how verification methodologies can help prevent incorrect or unsafe behavior.

Related Articles
EDA’s Future Is Evidence-Driven Automation
AI may accelerate semiconductor design, but users still need formal proof, semantic continuity, and auditable workflows to trust automation.

Hardware-Software Co-Design In The AI Era
Even though the benefits are accepted, a combined hardware/software development flow is hampered by a lot of challenges.

When AI Agents Cross Chip Design Silos
Specialization and orchestration are becoming more important as the role of AI agents in chip design widens, but coordination, control, and trust remain challenging.