Hermes Agent, told to keep a program library
Told to keep one tested program per task and run it first when the task returns.
- Share learned
- +0.44 ±0.10
- Stream, budget used
- 16.9 solved 93%
- Epilogue, budget used
- 31.3 solved 100%
- Model spend
- $12.30 11,256 calls
- Instances
- 201 19.5 hours
Budget used is the share of the cap of 16 that an instance was charged, averaged over the instances: a solved instance costs 1 for each wrong answer and each demonstration, an unsolved one all 16, so lower is better. The share learned reads the stream's budget against the learner itself reset, the same system wiped before every instance of the same stream: the share of the budget left to learn that the learner saves, 0 for a learner that keeps nothing and 1 for perfect memory.
- As shipped
- Memory wiped
- Asked to store
- Told to keep a program library
- Told to keep a program library, memory wiped
- Workstreams
- Fresh