Continual-ARCby unify

Figures

The paper’s figures and tables

The Continual-ARC paper leads with Figure 1, every learner on the open models on its two frozen tests, one of the tasks it met and one of 100 it never met, read here beside the same learners in absolute terms. Most of the rest of its main text follows, and after it, from its appendices, the views for further introspection: the benchmark’s details, what the harnesses kept, and GPT-6 Luna’s 228 runs, on which no harness left to itself saves a measurable share of what it leaves to learn and a deliberately simple program library saves 0.49 to 0.62 of its own. Each element carries the paper’s number. Point at a chart to read it, switch effort where the paper has panels, turn learners off and on in the legend, and open the run behind any number in the replay.

Two kinds of learning

An agent can get better with experience in two ways. It can remember the rule of a task it met before, which the paper calls memorization, or it can get better at tasks it has never met, which it calls generalization. After the stream the learner is frozen and plays a test of each, and Figure 1 puts every learner on both.

Figure 1

Every learner on the two frozen tests

the student
−0.2−0.100.10.20.30.40.50.60.70.80.9−0.2−0.100.10.20.30.40.5seen test, share saved: memorizationheld-out test, share saved: generalizationbettermemorizationwithin chancegeneralization within chanceHHHH-EHH-THH-T-EHH-E-T-EBprogram libraryWDAggerWDAgger, 2 usersWDAgger, 4 usersBdemo-cache floorBtold the page
Every learner on the open models after the stream, frozen, at one user, Qwen3.5-9B and GLM-5.3 Flash playing its own harness one at a time. Across is the share of its reset’s spend it saves on fresh instances of the tasks it met, the seen test, and up is the share it saves on 100 tasks it never met, the held-out test, each against the learner itself reset before each instance, on the same instances, with its 95 percent interval as a faint cross. The gray cross is the interval of a learner that keeps nothing between instances, widened for the number of learners on the panel: how far chance moves the panel’s points on each test, so a point inside its upright arm is within chance on the seen test, and one inside its level arm on the held-out test.
Beside Figure 1

The two tests in absolute terms

the student
seen test, budget used (%)0255075100held-out test, budget used (%)0255075100← betterprogram librarydemo-cache floorHH-EH-TH-T-EH-E-T-EDAggerDAgger, 2 usersDAgger, 4 userstold the page
The same learners on the budget each used on each test, its reset’s as a hollow disc and its own filled, lower is better: what Figure 1’s shares are shares of, which the paper gives as its Table 16. A learner whose reset already spends little can save a large share of very little, so the two read together.

The benchmark

A learner meets a long stream of ARC instances. The same tasks recur after dormant gaps, each time with a fresh input, while new tasks keep arriving, and every instance is charged for the feedback it uses. Then come the two tests, each a single round of 100 instances played from where the stream left the learner.

Figure 2

One instance

environmentserves the task id and a fresh input gridlearnerkeeps whatever it likes between instancessubmit a gridwrong: cost 1 · cap 8correct / incorrectrequest a demonstrationcost 1 each · cap 8one more pairAn instance ends on acorrect answer or theeighth wrong one, andthere is no giving up.A ninth request for ademonstration isrefused at no cost.budget used100 × cost / 16 %cost: wrong answersplus pairs bought ifsolved, else all 16;compute is logged,not scored
The learner sees the task id and a fresh input grid. It may submit a grid, and hears only correct or incorrect, or pay for one more demonstration pair of the task. On an instance it solves, each wrong answer and each demonstration costs one; an instance it does not solve costs the whole budget of sixteen. The budget used is that cost as a share of the sixteen, so lower is better. What the learner carries between instances is recorded but does not enter the score.
Figure 3, right

The stream

first meeting with the taskthe task recurstask active, in a spell
workstreams125050100150
Point at a column to see which task it served; the task’s marks light up.
The 200 instances of the ARC-AGI-1 stream at seed 0, one row per task and one mark per instance, with a thin line for each spell in which a task is active. Four tasks are worked at a time, in spells of about three instances with gaps of about thirty between them: a learner has met 14 tasks by step 50 and all 25 by step 100, and a task recurs after a median gap of 5 instances, at most 141. Point at a column to see which task it served.

Learners

The paper asks where learning should go: into the notes an agent harness keeps, or into the weights of the model that drives it. Either can be taught by the model itself or by a stronger one, so every learner is one student, Qwen3.5-9B, placed by those two choices, with GLM-5.3 Flash as the teacher.

Table 3, in the benchmark’s details below, places each row, and the post describes each.

Results

Table 1 gives each row that keeps something at one user on the open models, on the stream and on the two tests, and the table here adds the resets the rows are read against.

Table 1

Every row on Qwen3.5-9B and GLM-5.3 Flash

rowQwen3.5-9BGLM-5.3 Flash
Lseenheld-outLseenheld-out
notes in the harness
H−0.02 −0.09 to +0.04−0.01 −0.04 to +0.03−0.01 −0.05 to +0.03+0.12 −0.16 to +0.36+0.12 −0.03 to +0.27+0.00 −0.11 to +0.10
H-E+0.05 −0.06 to +0.16+0.03 −0.08 to +0.13−0.01 −0.05 to +0.03+0.62 +0.49 to +0.75+0.74 +0.64 to +0.84+0.07 −0.02 to +0.16
notes the teacher writes
H-T+0.05 −0.02 to +0.14+0.06 −0.01 to +0.14+0.03 −0.02 to +0.07–––
H-T-E+0.07 −0.03 to +0.16+0.01 −0.05 to +0.07−0.02 −0.07 to +0.03–––
H-E-T-E+0.29 +0.19 to +0.40+0.34 +0.26 to +0.42−0.05 −0.10 to −0.00–––
weights
DAgger+0.43 +0.33 to +0.54+0.57 +0.48 to +0.67+0.29 +0.23 to +0.35–––
baselines
program library+0.43 +0.25 to +0.64+0.65 +0.46 to +0.86+0.03 −0.02 to +0.09+0.68 +0.49 to +0.87+0.81 +0.59 to +1.00+0.05 −0.06 to +0.14
stateless floor–+0.06 −0.03 to +0.15−0.02 −0.10 to +0.05–−0.06 −0.16 to +0.04+0.03 −0.10 to +0.14
demo-cache floor+0.31 +0.19 to +0.45+0.37 +0.26 to +0.51−0.01 −0.05 to +0.03+0.41 +0.29 to +0.53+0.44 +0.34 to +0.57−0.02 −0.11 to +0.06
H0–+0.00 −0.05 to +0.07+0.03 −0.03 to +0.08–−0.05 −0.23 to +0.11+0.13 −0.01 to +0.26
H-E0–−0.06 −0.14 to +0.01+0.00 −0.05 to +0.06–−0.06 −0.24 to +0.07−0.01 −0.15 to +0.12
PL0–+0.03 −0.07 to +0.12+0.01 −0.06 to +0.07–+0.00 −0.16 to +0.13−0.05 −0.22 to +0.09
Every row at one user on the two open models. L is the share learned on the stream, against the row itself reset before every instance: its wiped twin, and for a trained row the untrained student in the same harness. A row that keeps nothing between instances is its own reset, and its L is 0. The seen and held-out columns are the share of the reset’s spend on each test that the row no longer pays, from four plays of each test against four of the reset’s. Higher is better for all three, and small type is the 95 percent interval over task resamples.

Further introspection

Everything above reads a learner after the stream, on the two tests. What follows is from the paper’s appendices: the benchmark’s details, how the harnesses ran and what each kept, and the stream itself on GPT-6 Luna, whose runs cover three harnesses at three reasoning efforts, with the share learned while each played. These views help explain a place on the plane rather than rank anyone.

Benchmark details

Appendix A of the paper: the words, the comparison, the models, the configurations and the tasks.

Table 2

The terms

termmeaning
taskan ARC rule, known to the learner by a name drawn for the run
instanceone input of a task to solve: generated fresh on the stream and the seen test, a task's own test pair on the held-out test
slotone entry of a schedule, naming a task, an instance seed and a phase
phasethe stream, or one of the two tests after it
seen test100 fresh instances of the stream's tasks, four of each, played at once by the learner as the stream left it, with nothing learned on the way: memorization, across on the plane
held-out testone instance of each of 100 evaluation tasks the stream never served, played the same way: generalization, up on the plane
schedulethe ordered list of slots a learner is served, fixed by a configuration, a seed and the number of users
configurationthe track, the number of tasks, the stream's length, the number of held-out tasks and the stream's parameters, such as arc1 (Table 5); the number of users is a setting of the run
regimea setting of the number of users, open to any learner: individual (one user, whose learner may change between any two instances) or collective (2, 4, 8 or 16, served in rounds, with changes only between rounds, each user bringing 25 tasks of their own, none shared: 50 tasks at two users, 100 at four, 200 at eight and 400 at sixteen)
trackthe fixed schedule on ARC-AGI-1: its configuration's stream with its two tests, one user, seed 0
exposure indexthe number of times the instance's task had been met before
budget usedU, the mean share of the cap an instance was charged, in percent, lower is better
learnable budgetthe budget used by the learner reset before every instance less that of perfect memory on the same system: the most that memory can save
share learnedL, the share of the learnable budget a learner saves over itself reset on the same stream: memorization while it is built
share savedS, the share of its reset's spend on a test that a learner no longer pays, against itself reset before each instance on the same instances: a coordinate of the plane
gapthe number of instances since the task was last met
rowa kind of learner: the floors (F), the program library (PL), a harness as shipped (H) and wiped (H0), and the rows of Table 3
preseta model at one reasoning effort
cellone run: a row at one preset
The words the paper and these pages use for the parts of the benchmark.
Table 3

The comparison on Qwen3.5-9B

kept in the harness’s noteskept in the student’s weights
taught by the student
  • H the harness as shipped
  • H-E given expert guidance, the program library's procedure
–
taught by GLM-5.3 Flash
  • H-T the teacher plays the stream and writes the notes, and the student plays from them
  • H-T-E the teacher given the guidance
  • H-E-T-E both given it
  • DAgger imitates the teacher's move in each state the student met, the teacher reading a page of playing rules
Every row plays the same stream in the same harness, Prime Agent, and the rows of one line share their teacher. What persists between instances, in notes or in weights, is the continual-learning method under test. Each row's wiped twin plays beside it, and so do the floors and the program library, on GLM-5.3 Flash as well as on the student. We also trained the student's weights by a policy gradient on each game's outcome and by self-distillation from the teacher's move, but got neither learning reliably, so neither has a row.
Table 4

The models

model, effortverifiedUSD per taskwhat ran, in which orders
GPT-6 Luna, low37.7%0.003every learner: workstreams, fresh
GPT-6 Luna, medium61.0%0.006every learner: workstreams, fresh
GPT-6 Luna, high †70.3%0.009every learner: workstreams, fresh
The presets, chosen from the ARC Prize leaderboard: by default the track runs the efforts of one model whose verified zero-shot score lies between about 15 and 60 percent, the band in which a learner has room to learn. Marked † is the one outside it, GPT-6 Luna at high effort. Verified is the leaderboard’s semi-private score at that effort and the cost per task the leaderboard’s. What ran is read from the run records; every harness ran as shipped and wiped.
Figure 5

An input grid does not say which task it belongs to

A feca6190B 7fe24cdd
A 91413438B 007bbfb7
A c3e719e8B ac0a08a4
Each row is one input generated for training task A, with the output of A’s rule and of task B’s rule on the same grid. Both are valid ARC tasks, and only the demonstrations or the id say which rule applies, which is why every instance shows its task id. Top: A draws diagonal lines from each colored cell on a larger black square, and B lays out the input and its three rotations as four quadrants. Middle: A tiles copies of the input, one per colored cell, and B places the input’s pattern in every tile whose cell is colored. Bottom: A places the whole input in every tile whose cell has the most frequent colour, and B enlarges every cell into a block. The pairs were found automatically: an input generated for A on which B’s verifier runs and returns a different valid grid.
Table 5

The ARC-AGI-1 track's configuration

configurationtasksstreamheld outusersactivespellrestnew task every
arc125200100143302
The track is arc1’s stream, one user, seed 0. Users is the number served at once, and the stream’s length is per user. Held out is the number of evaluation tasks the held-out test serves, once each, and the seen test serves four fresh instances of each of the stream’s tasks. The last four columns set how the stream interleaves its tasks: tasks active at once, the mean spell, the median rest and the mean time between new tasks, all in instances. Every configuration caps an instance at 8 wrong answers and 8 demonstrations.
Figure 6

The tasks the track serves

The ARC-AGI-1 track: the 25 stream tasks of arc1, drawn from the 400 training tasks, and the 100 evaluation tasks of its held-out test. Each thumbnail is the task's first demonstration pair, input beside output. Every task opens on the Tasks page with all its pairs and how each learner did on each visit.

Running the harnesses

Appendix D: each harness runs as a black box in a sandbox, with a fresh conversation for every instance and nothing carried over but what it wrote itself.

Table 8

How the adapter drives a harness

rulePrime AgentHermes Agent, OpenClaw
action budget300 s or 15 model calls a turn600 s a turn
after a turn cut for budget300 s or 6 calls, with a notice600 s, with a notice
longest one turn may run1,800 sthe action budget
silence before a restart600 s with no event and no model traffic300 s with no update and no model traffic
model errorsthe harness's own retries, then five resends with backoff from 60 sthe harness's own retries
a reply with no readable actiontwo re-prompts, then a give-upthe same
empty repliesunreadable; a restart after four in a rowthe same
restartstwo an action, files kept, then a give-upthe same
end of an instancethe verdict, then the harness's finishthe same; OpenClaw then idles 40 s
context window declared, reply cap262,144 and 65,536 tokensthe same
The adapter's rules. A turn cut for budget does not fail the instance: the harness keeps what it said and its next message tells it so. A give-up is scored as a failed instance, charged the rest of the instance's budget, and no learner may choose one. A restart opens a new session, which under one conversation per instance costs the harness nothing it would have kept.

What the harnesses kept

Appendix E reads everything each harness stored at the end of the stream in its three runs as shipped, one at each effort. Left to decide, none stored what would answer a returning task. The post takes the harnesses one at a time.

Table 9

What each learner is built to keep

learnerwhat it keepswritten byhow it reaches the model
program libraryone Python program per task, kept once it reproduces every pair the model has, and the pairs bought, up to eight a taskthe learner, whenever a program passes its checksrun first; the model is asked only when it fails
Prime Agentmemories, prompt notes, functions, skills and sub-agentsthe agent, when it chooses; an automatic refinement every 25 turnsa digest of the store, capped in size, at the start of a session; a search tool
Hermes Agentskill files and a memory about its usera background review after a turn with ten or more tool calls; the agent, when it choosesthe agent reads its skills at the start of a session or when it wants them
OpenClawa consolidated memory, skills and workspace filesa scheduled consolidation over its transcripts and a skill workshop; the agent, when it choosesskills listed at the start of a session; search over its memory and over the conversation of every earlier instance
What each learner is built to keep between instances, who writes it, and how it reaches the model when a task returns, read from the designs rather than from any run. The three harnesses are built for lessons that carry across unrelated work, and none of them files what it keeps under the task id, as the program library does.
Table 10

What memory changes in how a harness spends

pairs boughtwrong answerssolved (%)budget used (points)
harnesseffortfirstreturnsreturnsreturnsfirstreturns
Prime Agentlow+0.64−0.04−0.25+2.9+3.5 ±9.1−2.0 ±2.2
medium+0.12+0.07+0.04+0.6+2.3 ±4.3+0.7 ±1.3
high+0.32+0.09−0.02+1.1+4.8 ±4.1+0.4 ±1.4
Hermes Agentlow+0.36+0.72−0.12+2.3+2.3 ±7.2+1.8 ±2.7
medium−0.44+0.02−0.05−0.6−2.8 ±4.8+0.5 ±1.5
high+0.16+0.29+0.02+0.6+2.8 ±3.8+1.4 ±1.7
OpenClawlow+0.20+0.57−0.26+2.3−3.3 ±8.7−1.1 ±2.7
medium+0.48−0.01+0.39−4.0+3.0 ±5.2+3.8 ±2.4
high+0.52−0.08−0.03−0.6+4.3 ±3.6−0.8 ±1.3
ARC-AGI-1: each harness as shipped minus itself wiped, on the same instances of the stream, at first meetings (25 instances) and at returns (175), per instance. A positive budget is a cost of memory, and ± is one standard error over tasks. Pairs and wrong answers are counts, and on a solved instance one of either is 6.25 points of budget. A change at a task’s first meeting, such as the extra demonstrations Hermes Agent and OpenClaw buy at low effort, cannot come from memory of the task: it comes from what the harness kept about how to work.
Beside Table 9

What each harness stored, and what it saved

harnesswhat it held after the streamwhat reached the model on a returnsaved, lowsaved, medium
Prime Agentnothing beyond the session: no global storea digest of the empty store+1.3 ±2.2−0.9 ±1.2
Hermes Agenta memory file of three entries, none naming a task; 5 skills with 92 reference files, none named for a taska skill, on 175 of 175 returns; nothing naming the task−1.9 ±2.2−0.1 ±1.5
OpenClawno memory file; a dream diary of 16,857 bytes; no skillno earlier correct answer, on 122 returns to a solved task+1.4 ±2.2−3.7 ±2.3
What each harness held after the medium-effort stream, what of it reached the model when a task returned, and what its memory saved against itself wiped, in points of the budget on the stream, ± one standard error over tasks.

Results on GPT-6 Luna

Appendix I: to check whether what the harnesses do on the open models holds on a stronger one, GPT-6 Luna at each reasoning effort drives the two floors, the program library, and each harness as shipped and with everything it wrote wiped before every instance. Every learner meets the same instances in the same order, and each is scored by the share learned: read against itself reset, the same system wiped before every instance, the share of the budget left to learn that it saves. The figures after it are the site’s own, drawn from the runs’ records beside that table: the share learned by effort, and the stream read by the number of times a task had been met before.

After its stream a run went on to some or all of the 100 held-out tasks while it could still learn, which the frozen tests replace, so nothing from those tasks is drawn here.

Table 18

Share learned on GPT-6 Luna

GPT-6 LunaARC-AGI-1
effortlowmediumhigh
learnable budget, model alone (points)50.832.526.5
Stateless floor000
Demo-cache floor+0.19 ±0.05+0.24 ±0.08+0.28 ±0.08
Hermes Agent, memory wiped000
Hermes Agent, as shipped−0.05 ±0.060.00 ±0.06−0.08 ±0.08
OpenClaw, memory wiped000
OpenClaw, as shipped+0.04 ±0.06−0.15 ±0.10+0.01 ±0.06
Prime Agent, memory wiped000
Prime Agent, as shipped+0.03 ±0.05−0.04 ±0.05−0.04 ±0.06
Program library+0.49 ±0.11+0.60 ±0.13+0.62 ±0.15
The share learned of each learner at each reasoning effort, on the stream: the share of the learnable budget it saves over itself reset before every instance (for a harness, itself wiped; for the demo-cache floor, the stateless floor; for the program library, its programs wiped), where 0 is the reset and 1 is perfect memory, every returning task answered at no cost. A learner that keeps nothing between instances is its own reset and reads 0. The first row is the learnable budget of the model alone, in points: the stateless floor’s budget used less perfect memory’s. Small type is the standard error over tasks, and the best share learned in each column is bold, even within error bars. Every number opens the replay of its run.
Beside Table 18

Share learned by reasoning effort

ARC-AGI-1share learned−0.50.00.51.0the learner resetperfect memoryto learn51low33medium27highreasoning effort · learnable budget in points
ARC-AGI-1 · GPT-6 Luna, highlearnable budget of the model alone 26.5 points
learnershare learned95% intervalsaved over its reset
Demo-cache floor+0.28+0.11 to +0.44+7.3 pts
Program library+0.62+0.37 to +0.93+11.3 pts
Prime Agent, as shipped−0.04−0.16 to +0.07−0.9 pts
Hermes Agent, as shipped−0.08−0.23 to +0.08−1.6 pts
OpenClaw, as shipped+0.01−0.10 to +0.13+0.2 pts
The share learned of each learner on GPT-6 Luna that keeps something between instances, by reasoning effort, higher is better, with the learnable budget the model alone leaves at each effort under it. Zero is the learner itself reset before every instance and one is perfect memory on it; the band is the 95 percent interval over task resamples. No harness saves a measurable share at any effort, the program library saves 0.49 to 0.62 of its own, and the demo-cache floor, which only keeps the pairs it bought, 0.19 to 0.28. The learnable budget falls as the effort rises: a stronger model pays less on every instance, returns included, and leaves less to learn.
Beside Table 18

Budget used by exposure

GPT-6 Luna, lowbudget used (%)02040608001234567times the task was met before
GPT-6 Luna, mediumbudget used (%)0102030405001234567times the task was met before
GPT-6 Luna, highbudget used (%)01020304001234567times the task was met before
demo-cache floor after its first meeting
GPT-6 Luna, low · exposure 7met 7 times before
learnerbudget used16th to 84th percentileinstances
Stateless floor53.044.2 to 60.825
Demo-cache floor42.833.8 to 51.225
Program library20.812.8 to 28.725
Prime Agent, as shipped50.042.2 to 58.025
Prime Agent, memory wiped50.043.0 to 58.225
Hermes Agent, as shipped43.836.2 to 50.725
Hermes Agent, memory wiped46.238.2 to 54.025
OpenClaw, as shipped47.838.8 to 55.825
OpenClaw, memory wiped50.742.2 to 58.825
ARC-AGI-1, on the stream: budget used by the number of times the task had been met before, one panel per effort, lower is better, each panel on its own scale. A learner that relearns every task draws a flat line, and the demo-cache floor falls only by the demonstrations it no longer has to pay for; the dotted line is its mean after the first meeting. A learner that has learned a task falls below it. The program library does, using 13.9 to 23.5 points less of the budget than every harness at every effort, and no harness beats the demo-cache floor at any effort.
Beside Table 18

Budget saved against the demo-cache floor, by exposure

GPT-6 Luna, lowbudget saved (points)-40-20020406001234567times the task was met before
GPT-6 Luna, mediumbudget saved (points)-40-200204001234567times the task was met before
GPT-6 Luna, highbudget saved (points)-20-10010203001234567times the task was met before
GPT-6 Luna, low · exposure 7met 7 times before
learnersaved16th to 84th percentile
Stateless floor−10.2−14.0 to −5.8
Program library+22.0+14.5 to +28.8
Prime Agent, as shipped−7.2−13.5 to −1.0
Prime Agent, memory wiped−7.2−15.8 to −0.5
Hermes Agent, as shipped−1.0−7.5 to +5.5
Hermes Agent, memory wiped−3.5−11.5 to +3.5
OpenClaw, as shipped−5.0−11.5 to +1.5
OpenClaw, memory wiped−8.0−13.8 to −2.8
ARC-AGI-1, on the stream: budget saved against the demo-cache floor on the same instance, by the number of times the task had been met before, one panel per effort, higher is better, and zero is the floor. On a first meeting the floor has cached nothing and is the bare model: learners with tools save against it at low effort, and at medium and high effort about as much as the stateless floor does, which is the bare model too. As its cache fills, every harness falls to its level or below, and only the program library stays above. Pairing on the instance takes out what the schedule itself does, such as how hard the drawn instances are.
Beside Table 18

Solved by exposure

GPT-6 Luna, lowsolved within the cap (%)2040608010001234567times the task was met before
GPT-6 Luna, mediumsolved within the cap (%)2040608010001234567times the task was met before
GPT-6 Luna, highsolved within the cap (%)2040608010001234567times the task was met before
GPT-6 Luna, low · exposure 7met 7 times before
learnersolved16th to 84th percentileinstances
Stateless floor60.052.0 to 72.025
Demo-cache floor68.060.0 to 76.025
Program library80.072.0 to 88.025
Prime Agent, as shipped68.056.0 to 76.025
Prime Agent, memory wiped72.064.0 to 80.025
Hermes Agent, as shipped72.064.0 to 80.025
Hermes Agent, memory wiped72.064.0 to 84.025
OpenClaw, as shipped68.060.0 to 76.025
OpenClaw, memory wiped64.052.0 to 76.025
ARC-AGI-1, on the stream: the share of instances solved within the cap by the number of times the task had been met before, one panel per effort, all on one scale; the solve-rate twin of the budget used by exposure above. The leftmost point is the first-meeting solve rate, the check that a preset has room to learn. The two views differ where a learner learns to solve but keeps paying for attempts and demonstrations on the way, and where it fails outright.

The second and third tracks

Appendix K: ARC-AGI-2 and ARC-AGI-3 under the first track’s protocol, each checked end to end with four learners on Kimi K3.1 Pro. The Overview says what each track serves and quotes the paper’s Table 21, how often three models solve each one at first sight.

Table 22

Four learners on each new track

streamfrozen tests
trackrowsolvedLseenheld-out
ARC-AGI-2stateless floor61.0%0−0.01 −0.13 to +0.10−0.01 −0.05 to +0.03
demo-cache floor71.0%+0.35 +0.22 to +0.48+0.37 +0.26 to +0.48+0.01 −0.02 to +0.04
Prime Agent, as shipped64.5%0.00 −0.09 to +0.08+0.01 −0.06 to +0.09−0.02 −0.04 to +0.01
Prime Agent, wiped65.0%0−0.04 −0.21 to +0.09+0.01 −0.03 to +0.06
ARC-AGI-3stateless floor25.8%0−0.06 −0.12 to 0.00+0.01 −0.05 to +0.06
demo-cache floor32.5%+0.11 +0.01 to +0.23+0.13 +0.04 to +0.24−0.02 −0.10 to +0.04
Prime Agent, as shipped60.0%+0.04 −0.07 to +0.14+0.03 −0.05 to +0.09−0.04 −0.11 to +0.02
Prime Agent, wiped52.5%0+0.01 −0.04 to +0.06+0.06 −0.03 to +0.17
Kimi K3.1 Pro at medium effort and one user, on arc2 and arc3 at seed 0. Solved is the share of the stream’s instances solved. L is the share learned on the stream, and the seen and held-out columns the share of the reset’s spend on each frozen test that the row no longer pays, read as in Table 1: the demo-cache floor against the stateless floor and Prime Agent against itself wiped, from four plays of each test by the row and four by its reset. A row that is its own reset learns 0. Higher is better for all three, and small type is the 95 percent interval over task resamples.

In the paper and not drawn here: the open models’ share solved and budget used by visit over the stream and on each test (Figure 4); every row on Qwen3.5-9B that keeps notes in the harness and every row on GLM-5.3 Flash, row H-A among them (Tables 11 and 12); how often DAgger’s phases changed its adapter (Figure 7); DAgger at more users (Figure 8, Table 17); every row in absolute terms as a table (Table 16), drawn above as a chart beside Figure 1; the budget a solved instance used (Table 15), whose curves the blog post draws; GPT-6 Luna’s share solved and budget used by visit at each effort, and its budget on the instances it solves (Figures 9 to 12); GPT-6 Luna’s frozen tests (Table 19), which the leaderboard draws; the ablations (Table 13); the stream process written out (Algorithm 1); where the code keeps every other text a learner is shown (Table 6); the software behind the harness rows (Table 7); the comparison with related benchmarks (Table 14); the first-sight solve rates on the private held-out season (Table 20); and those on the second and third tracks (Table 21), which the Overview quotes.