From Functional Monitoring To In-System Test: Extending Voltage-Aware Observability

By Aakash Jani, Henrique Mendes, and Robert Wilcox

In a previous blog, we highlighted the importance of combining physical and functional sensors to plan for new workloads. This use case remains relevant for high-performance AI and HPC systems because ‘spiky’ workloads induce severe droops that dominate power budgets.

Your $1 billion ASIC just had a scan failure during an In-System Test. How can you tell whether the failure is an actual structural failure, or one caused by physical phenomena like voltage droop? Scan patterns can often generate intense switching activity when long or multiple chains are toggled, which may induce a voltage droop. If the failure is due to a voltage droop, you have a few options: slow the shift clock, raise the voltage, or repartition your scan patterns. Without physical sensors that report conditions during the test, teams can’t tell the two cases apart and might throw away good parts on false failures.

Co-mingling physical and functional sensors is necessary across the system lifecycle because functional and physical failures are connected. Let’s move further down the system lifecycle into silicon deployment for another example.

The previous demo combined the Movellus Aeonic Insight Droop Detector with Siemens Tessent Embedded Analytics to correlate voltage events with processor and interconnect activity. Today, we are proud to announce an extension of physical and functional monitoring for system lifecycle management: adding Tessent In-System Test (IST) to the demo. Running structural test on deployed systems means the false-failure question follows the part off the tester and into the field, where a wrong call can get a healthy node pulled from service. Combined with voltage monitoring, IST helps engineers separate false failures caused by droop from real structural defects.

Why add in-system test?

Meta ran silent-error tests across hundreds of thousands of fleet machines and flagged hundreds of CPUs that corrupted data without reporting an error.^[1] Google has reported a similar rate in its own fleet.^[2] Amazon also presented its results from using In-System Test technology on AWS silicon.^[3] Tessent IST incorporates selected structural tests into deployed systems, allowing testing without removing the device, so faults like these can be investigated early.

However, structural testing can cause intense switching activity, which can lead to voltage droops during tests. These events depend on test patterns, sensor locations, and device conditions. A pass/fail result alone lacks context.

Combining IST with telemetry enables more detailed analysis: Did the droop coincide with testing? Which core was tested? Was other traffic active?

Demonstrating physical and functional sensors across the system lifecycle

The updated demonstration integrates Tessent IST, alongside the previous solution, on a Siemens Veloce proFPGA platform: Movellus Aeonic Insight voltage-droop sensors, Tessent Embedded Analytics functional monitors and message infrastructure, and Siemens scan-and-test infrastructure for IST. The target design contains multiple DFT-instrumented RISC-V cores, which drive the workload observed in the functional mode, and then receive scan patterns when brought into test mode.

Fig. 1: Demo system including Tessent Embedded Analytics, the Movellus Aeonic Insight voltage-droop sensor, and scan-and-test infrastructure.

Functional Mode: a Movellus voltage droop detector is configured through an APB path and can issue a hardware event when a programmed threshold is crossed. Tessent Embedded Analytics receives the event through its trigger and message infrastructure. A Status Monitor records sensor and system signals with timestamps, while a Bus Monitor reports transaction activity on the SoC interconnect. The workflow shows how a physical event initiates collection of the functional evidence needed to investigate its root cause.

Fig. 2: Functional-mode workflow correlating voltage-droop events with timestamped processor and interconnect activity.

Test Mode: The Tessent IST support adds test control and test-state visibility. Test patterns are run on the demonstration platform, and test start and stop activity is observed through Tessent Embedded Analytics Status monitoring, which monitors an internal test-mode state, so that the test window can be aligned with voltage data. This gives the demonstration time-aligned test execution and sensor behavior.

From pass/fail to actionable context

The value of the combined approach is not that voltage telemetry replaces structural test, or that functional monitors determine whether a test pattern passed. The benefit comes from placing their outputs on a common timeline. Structural test identifies whether the targeted logic responds as expected. The voltage monitor reports the physical conditions during that test. Functional monitors reveal concurrent system activity that may have contributed to those conditions.

For example, if a voltage excursion appears only when a particular pattern group is active, the engineer has a focused starting point for investigation. If the excursion coincides instead with heavy traffic from a neighboring functional core, the test scheduler may avoid running that workload and the test concurrently. Repeated observations across sensor locations can also help determine whether an event is local or shared across part of the power-delivery network. These are diagnostic leads rather than conclusions, but they narrow the search space and reduce dependence on worst-case assumptions.

Fig. 3: Test-mode workflow aligning In-System test execution with voltage-sensor data using Status Monitor.

One sensor, two operating domains

The previous blog showed how a voltage droop detector and functional monitoring can bridge the physical and functional domains. Adding Tessent IST extends that observability into test operation. The same sensor can contribute evidence during normal workloads and while structural test content is being applied, while Tessent Embedded Analytics supplies the timestamped context needed to relate those observations.

The current FPGA demonstration establishes the end-to-end concept by running IST patterns and correlating test start and stop with voltage and functional-monitor data. The next step is to complete direct sensor access through iJTAG, providing a path in which test control, sensor observability and functional telemetry can be coordinated through the deployed system.

Together, these capabilities move in-system diagnosis beyond isolated measurements and toward a repeatable detect-diagnose-test workflow.

References

H. D. Dixit et al., “Silent data corruptions at scale,” 2021, arXiv:2102.11245.
P. H. Hochschild et al., “Cores that don’t count,” in Proc. 18th Workshop on Hot Topics in Operating Systems (HotOS ’21), 2021.
D. Trock, S. Mahadevan, N. Mukherjee, L. Harrison, J. Rajski, and J. Tyszer, “Deterministic In-Fleet Scan Test for a Cloud Computing Platform,” in 2024 IEEE International Test Conference (ITC), 2024, pp. 391–399.

Henrique Mendes is a product engineer for Tessent Embedded Analytics at Siemens EDA.

Robert Wilcox is a product engineer for Tessent Embedded Analytics at Siemens EDA.