TRACES moves beyond static datasets and answer keys, testing whether AI can navigate real-world scientific environments, adapt to feedback and produce verifiable discoveries
REDWOOD CITY, Calif., Aug. 20, 2026 /PRNewswire/ — Apodex today introduced TRACES, a novel benchmark designed to evaluate AI on one of the hardest challenges in artificial intelligence: working on real-world problems where the answer may not yet be known.
Most AI benchmarks begin with an answer key. They measure whether a model can retrieve established knowledge, solve a predefined problem, write correct code or reproduce a known result. Scientific discovery is different. Researchers often begin without knowing the answer, and sometimes without knowing whether their initial hypothesis is correct. Progress may require searching vast bodies of evidence, using specialized tools, maintaining competing hypotheses, running experiments, learning from failure and repeatedly revising a line of inquiry before reaching a conclusion that can withstand verification. In some scientific problems, the ultimate answer may not become known until months or years later.
TRACES is designed for that world. Rather than treating a benchmark as a static dataset with hidden answers, TRACES transforms real-world problems into executable environments where AI systems can observe, act, use tools, learn from feedback and work toward verifiable outcomes. The process of such AI systems interacting with executable environments are also evaluated along six TRACES capabilities, novel evaluation metrics which we design specifically for the purpose of evaluating discoverative AI. Together, these real-world problems, executable environments and evaluation frameworks form the TRACES benchmark.
“TRACES is a benchmark designed specifically to evaluate progress in discoverative AI,” said Dr. Sheng Wang, Lead Scientist at Apodex, who leads discovery and evaluation. “It brings together sophisticated efforts in scouting high-value real-world problems, assembling the tools and data needed to build executable environments, and developing a novel scoring system that evaluates not only outcomes but also the discovery process. We believe TRACES represents an important milestone in the development of discoverative AI.”
Beyond Static Benchmarks: Toward Executable Environments
TRACES is built around a different conception of AI evaluation. Instead of presenting a model with a question and scoring only its final answer, a TRACES environment gives an AI system an environment in which to work. Depending on the problem, that environment may include scientific literature, structured datasets, code execution, specialized scientific tools, simulators, folding engines, experimental feedback or other interfaces. The system must decide what to do, interpret what happens, update its approach and continue working toward a verifiable outcome. The full trajectory can then be evaluated alongside the final result.
TRACES: Six Capabilities for Discovery
Discoverative AI aims at answering open-ended, high-value questions, many of which may lack outcome feedback. It is therefore essential to evaluate and verify the process of such systems, essential for the success of discoverative AI. To this end, within these environments, TRACES evaluates six capabilities Apodex considers fundamental to long-horizon, verifiable problem solving:
- T — Tools: selecting, calling and correctly interpreting external tools
- R — Repair: locating and correcting its own errors once feedback arrives
- A — Alternatives: laying out competing hypotheses and keeping or discarding them as evidence accumulates
- C — Coherence: holding state, constraints and logic intact across a long chain of work
- E — Evidence: grounding every conclusion in observation, data, experiment or citation
- S — Scope: stating the conditions under which a conclusion holds, and where it does not apply
“The TRACES process verification is what makes the benchmark unique,” said Brian Wang, AI Research Scientist at Apodex. “In scientific discovery, the answer is one line at the end of hundreds of judgments — what to try next, when the evidence is enough, when to abandon a hypothesis. The capability lives there, and scoring only the last line throws away almost all of it. Our TRACES verifier systematically scores the quality of process, which is essential for evaluating and improving AI systems for long-horizon, open-ended discoverative jobs.”
Together, these dimensions evaluate more than whether an AI produced the right final answer. They evaluate how the system got there. A scientifically plausible answer is not enough if the system used the wrong tool, ignored contradictory evidence, failed to consider alternative explanations or made a claim broader than its evidence supports. The objective is to evaluate both outcome and process — what the system discovered and whether the path it took supports the conclusion.
Verifying the Path, Not Only the Answer
In TRACES, AI systems are evaluated both by outcome and process verifiers. The outcome verifier grades a submission against hidden ground truth; the process evaluation asks whether the conclusion was earned — whether the path that produced it is one that would hold up on the next problem. Both matter, but for discoverative AI, process verification is arguably more important, evaluating submissions along the six fundamental TRACES capabilities.
TRACES makes each of the generic capabilities concrete for different environments. Evidence fidelity (E), for instance, could take different forms: where a solver must propose a candidate that will later be tested in the physical world, it asks whether the proposal rests on a prediction the solver actually ran under its limited feedback budget rather than on asserted plausibility; where the deliverable is a report that must survive audit, it asks whether every figure traces to a procedure genuinely executed on the population it claims to describe. Decomposing TRACES capabilities into problem-specific atomic skills or subrubrics, TRACES verification offers targeted process evaluation and diagnostic feedback.
Evaluators do not assign a free-form score — they match observed evidence against explicit written descriptions of each scoring band, and every finding is anchored to specific steps in the recorded trajectory. An independent model then reviews the result, and disagreement triggers a re-score and final adjudication. Calibrations between process verification and hidden outcomes, and verification-repair loops are also carried out to ensure the validity of our process verifier.
Beyond What AI Knows: Measuring What It Can Discover
As AI moves deeper into science, the distinction between knowing and discovering becomes increasingly important. A model can summarize what humanity already knows about a disease and still fail to identify a new therapeutic opportunity. It can understand published protein science and still fail to design a biological candidate that works when tested. It can produce an elegant scientific explanation that collapses when confronted with new evidence. Discovery begins where the answer key ends. TRACES is Apodex’s attempt to evaluate AI at that boundary.
About Apodex
Apodex is building Discoverative AI: artificial intelligence that discovers unknown things from known knowledge, rather than just generating outputs from material already in the training distribution. At the core is the company’s Self-Evolving Solver, a system built for real-world problems: questions with no answer in any dataset, where every reasoning step can be audited, and every improvement is earned through verified discovery.
Apodex’s founding thesis is that scale and data alone cannot cross the barrier from pattern-matching to genuine discovery. Crossing it requires new ideas from neuroscience, information theory, physics, and formal verification — and the people sharp enough to bring them. Apodex is hiring across research, engineering, and open-source contributions. Learn more at www.apodex.com.
Participation and Access
TRACES is open for participation, and Apodex is actively seeking both solver systems to evaluate and new problems to build. Teams building a solver system — a model, a harness, an agent loop, or all three — can submit it for evaluation on our benchmark, including on the TRACES capabilities, through https://discovery.apodex.com/submit-solver.html. No integration work is required from the submitting team: every system enters through the same fixed episode interface, and Apodex performs that work in collaboration with the submitter. Researchers and organizations holding a consequential problem from their own field — one they believe an AI system should be able to attack — can propose it through https://discovery.apodex.com/submit-problem.html, and Apodex will work with them to turn it into an executable environment with both outcome and process verifiers.
For More Information
The complete framework, including the problem-scouting process behind the 423 high-value problems assembled from a survey of 561 industries across 16 sectors, the environment and episode design, the process metric, and full results, is documented in the following technical report and websites:
Explore TRACES and its environments
Read the TRACES technical paper on arXiv
Media Contact
TEAM LEWIS: apodex@teamlewis.com
https://www.allianz-trade.com/en_HK.html
https://www.linkedin.com/company/allianz-trade-apac/
