Hermes Agent, told to keep a program library, memory wiped
Told to keep a program library, with everything it wrote deleted before every instance.
- Share learned
- 0 its own reset
- Stream, budget used
- 47.3 solved 72%
- Epilogue, budget used
- 44.8 solved 67%
- Model spend
- $10.02 17,851 calls
- Instances
- 206 19.6 hours
Budget used is the share of the cap of 16 that an instance was charged, averaged over the instances: a solved instance costs 1 for each wrong answer and each demonstration, an unsolved one all 16, so lower is better. The share learned reads the stream's budget against the learner itself reset before every instance, and this learner keeps nothing between instances, so it is its own reset and reads 0.
- As shipped
- Memory wiped
- Asked to store
- Told to keep a program library
- Told to keep a program library, memory wiped
- Workstreams
- Fresh