OpenAI's GPT-6 Astra scored 62.7% on ARC-AGI-3's neutral test -- and 99.9% on a version only Astra can run

ARC Prize, the nonprofit behind the benchmark most AI researchers treat as the hardest general-reasoning test available, published two very different numbers for OpenAI's newest model on the same results page this week. Under its standard, provider-neutral harness -- the same interface every other lab's model is tested on -- GPT-6 Astra solved 62.7% of ARC-AGI-3's Semi-Private evaluation set. Under a second harness built specifically around OpenAI's own infrastructure, the same model hit 99.9%. Both figures came from ARC Prize's own testing, not from OpenAI's marketing team -- and the gap between them is bigger than the improvement most model generations manage in a full year.

ARC-AGI-3 does not ask multiple-choice questions. It drops an AI agent into an unfamiliar, video-game-like environment with no instructions, and scores it on whether the agent can work out the rules, the goal, and a workable plan purely by acting and observing what happens -- reasoning under genuine novelty, the thing the benchmark is designed to resist memorizing its way around. ARC Prize co-founder François Chollet, who designed the underlying test, reviewed Astra's own reasoning traces and said the model was performing highly efficient, on-the-fly symbolic world modeling -- inventing its own shorthand notation to represent in-game situations as it played.

The difference between 62.7% and 99.9% is not raw reasoning power -- it is memory. The standard harness treats every model the same way: whatever notes it wants to keep between moves, it has to write out and carry forward itself, through the same plain interface Claude Opus 5 or Gemini get tested on. The Provider Adapter instead lets Astra hold its own internal reasoning state across an entire game and compress it as needed -- state that ARC Prize itself cannot inspect, and that OpenAI has not built for any other benchmark to use. Across the tasks solved under both setups, ARC Prize found the adapter runs were 3.66 times faster, used 49% fewer total tokens, and cost less in absolute terms -- $18,817 per max-reasoning task versus $26,098 under the standard harness -- a real efficiency gain, measured on a track no competitor's model is running. ARC Prize's own leaderboard also runs Claude Opus 5, Claude Fable 5, Gemini 3.5 and 3.7, GPT-5.6, and Grok 4.5 and 4.6 through that same standard harness -- which is exactly why 62.7%, not 99.9%, is the number that tells a reader anything about where Astra actually stands next to its rivals. (For comparison, ARC Prize's human testers earned an average of $12.78 per attempted game; Astra's own max-reasoning run cost $26,098 per task under the standard harness -- roughly 2,000 times as much.)

Neither of ARC Prize's own numbers is the one that spread widest. Screenshots citing a flat 99.9%, with no mention of which harness produced it, circulated within hours, alongside a separate, widely shared claim that Astra scored 98.6% -- a figure that matches no confirmed ARC Prize result. The New Stack flagged the gap directly, under the headline "the asterisk matters more than the score." The asterisk is the harness.

ARC Prize's own co-founders read the result as genuine progress, not a marketing artifact -- while disagreeing on how far it goes. Chollet called the run a major breakthrough in model intelligence, describing it in a separate post as representing a step-function change in interactive-reasoning capability.

“GPT-6 Astra represents a step-function change in model capability for interactive reasoning problems.” — François Chollet, ARC Prize co-founder

Co-founder Mike Knoop was more careful about what the result proves. He called the provider-adapter run "a qualitatively large leap towards AGI" -- but added, in the same post, that "we lack evidence to call this AGI yet," and noted that ARC-AGI-3 was never built to test everything the AGI label implies. "Open-ended invention is unsolved," he wrote, "and this will form the new basis for ARC-AGI-4."

The one figure ARC Prize highlighted without any caveat is about efficiency, not accuracy: under the Provider Adapter, Astra needed fewer moves than the median human tester on 96% of levels, using 51.7% fewer actions on average to reach the same goals. Frontier models have beaten specific human benchmarks on accuracy before; a documented efficiency edge against a human baseline, on a benchmark built to resist exactly this kind of shortcut, has not been reported here before.

96% (of ARC-AGI-3 levels where Astra needed fewer moves than the median human tester)

The result lands the same week OpenAI confirmed Astra crossed its own "Critical" cybersecurity threshold under a separate, gated access program -- a different capability claim, evaluated by a different method, but the same pattern: a real number, produced under conditions no outside party can fully replicate, arriving well ahead of any public release date. OpenAI has not said when, or whether, the version of Astra that scored 99.9% will be available outside ARC Prize's own test environment. The pattern reaches beyond Astra: as frontier labs increasingly test unreleased models through bespoke, vendor-built harnesses before any public launch, the reader-facing question shifts from what a model scored to under what conditions, and who else could reproduce them.