
Last week Anthropic published research showing that an AI model can produce formal proofs for production at scale. Working with the fuzzing tests sourced from 123 OS libraries, Claude used mathematics – just like TrustInSoft Analyzer does – to prove that none of them can ever crash for any input up to 64 MB. And in doing so, it also found and fixed bugs that years of fuzzing had missed.
AI models are getting very good at finding bugs – and chaining small ones into serious exploits. Testing “harder” is not the fix – it simply can’t keep pace. Teams – especially in embedded development – need a way to show that whole classes of bugs are absent.
At TrustInSoft, we’ve been having this discussion with embedded teams for more than a decade, and so we’re excited to hear it made this loudly. It’s also worth looking at what this research shows, what it doesn’t yet cover and what embedded teams writing C, C++ and Rust can do about it today.
Where we agree with Anthropic’s results
- Your tests can become proof
Anthropic started with fuzzing. A fuzzer throws huge numbers of generated inputs at a program and waits for a crash. It can find a lot, but it can’t find everything because the input space is effectively infinite. Many of the bugs found by the Anthropic research team only appeared on inputs of 20 MB or more – and this is beyond the general range most fuzzers test at.
The biggest challenge when it comes to formal verification is the spec: Writing down what “correct” means. Anthropic skipped this step, which is their right as an organization running a “test”. They took the fuzz tests developers had already written and proved that “this test always passes”.
TrustInSoft customers know this pattern. TrustInSoft Analyzer replays the test suite, ensuring there is no undefined behavior. The input values of those same tests are then generalized so they cover the full input range, no formal spec required. The tests your team maintain already do the job.
- Memory safety is the spec almost everyone shares
Most programs don’t come with a precise definition of correct behavior, but almost all of them require one requirement: Don’t read or write memory that is off limits. Anthropic leans on this broad property because it applies nearly universally, and these errors drive a large portion of real vulnerabilities.
TrustInSoft Analyzer starts here too, using the C and C++ standards and the Rust language specification as the definition. Buffer overflows, use-after-free, uninitialized reads, signed integer overflow, invalid pointer arithmetic, division by zero – these are all examples of undefined behavior and TrustInSoft Analyzer checks each one on every path.
- A failed proof is a bug report
The research presented by Anthropic noted that even an unfinished proof attempt is a result. A stuck proof usually points to a real defect. The researchers found hundreds of bugs based on failed attempts alone.
And this is also how TrustInSoft Analyzer alarms work in practice. Where the analysis cannot show an operation is safe, it raises an alarm with the path and values that lead there. The Root Cause Investigator – our own “super debugger” – points the engineer to the exact line that caused the alarm, saving a lot of guesswork.
Where the Anthropic research falls short for embedded teams – and where TISA already is
The research was done on a set of open-source libraries running on general-purpose computers and that comes with a set of assumptions.
Your target isn’t a Linux process
The proofs model a single-threaded, statically linked program on a RISC-V CPU running on Linux. The researchers also acknowledge that this is a gap: most of the world runs something else, and closing it needs models of other machines.
Embedded code is running on bare metal, on an RTOS, on a hypervisor partition, on hardware with its own word sizes, alignment rules and memory map. TISA is configured for the target: Its data model, its endianness, its physical memory layout. The Analyzer is hardware-agnostic: It runs on a standard development PC, with no board, simulator or emulator. The results are based on the hardware you ship on, even before that hardware is on your desk. Built-in support for AUTOSAR, ARINC 653 APIs means the platform your application sits on is part of the model
Your code isn’t static
Each proof in Anthropic’s research is tied to one executable. Change the source, a dependency, the compiler or a build flag, and the link to the proved binary breaks. When the researchers patched the bugs they found, they regenerated the specifications and reproved the binaries. The full run took a few weeks, and checking a proof is a manual, documented procedure.
Embedded code changes with every sprint, every product variant and every toolchain update. TrustInSoft Analyzer works on the source code and runs inside your CI pipeline, in Agile and V-model workflows alike, so the analysis reruns every time the code changes. Each run produces a report your team, and your auditors, can trace.
A trust base that comes with a certificate
Each proof rests on what you have to trust. In this research, that is the RISC-V model, the Linux process model and the Rocq proof checker. The authors say hardening those things is real work, though far smaller than reviewing the code itself.
For safety-critical teams, “trust the tool” has weight. TrustInSoft Analyzer comes with a trust base of its own and we hold it to the same standard. Qualification is one part – it’s qualified by TÜV SÜD to ISO 26262 ASIL D for C. That qualification goes into the safety manual – something your teams functional safety manager can put in a safety case. The other part is the toolchain itself, which we have been testing and proving for more than a decade.
That work is ongoing and pairing AI with formal verification is making it faster than ever.
What this research has revealed
At TrustInSoft, we don’t just read this report as validation for the work we’ve been carrying out since 2014. There were two segments that got us excited.
Proof on the binary
Anthropic’s proofs verify compiled machine code, covering the output of a specific compiler. This is both a strength and a limitation: undefined behavior originates in source code but becomes some concrete behavior after compilation. It may appear harmless in one build yet cause failures in the next. Source-level analysis detects this latent undefined behavior across compilers and targets and identifies the relevant source lines. In safety-critical work, a qualified toolchain addresses the compiler separately. The approaches answer different questions, but source code is where undefined behavior must be identified.
AI writes the proof
In both the Anthropic testing and in TrustInSoft Analyzer’s own product development, a LLM generated the proofs and then also created the fix when the proof failed. This changes the economics of formal verification, making it easier to deploy and scale across the entire source code, even legacy code.
How TrustInSoft Analyzer works is the same direction that Anthropic is describing: AI does the heavy lifting, and a sound, checkable method decides whether the results hold.
Proof delivered today
The pressure is already in the pipeline. More code is being written with AI and it’s shipping in systems where a crash is a recall, a field update or a safety incident.
TrustInSoft Analyzer answers that question with proof on your target, in your pipeline, starting from the tests you’re already deploying. We’re clear on the boundary – it proves the absence of runtime errors and undefined behavior.
And always has.
See it on your code today. Reach out to us for a proof of concept and we can show you what TrustInSoft Analyzer can find.