The loop
- Start from a hypothesis. Say what you expect to change and why: “fewer instructions reduce failed turns”, “a scratchpad improves multi-step tasks”.
- Create an experiment. Each experiment is its own numbered folder with its own source and settings. The previous experiment stays exactly as it was, so results remain comparable.
- Run a practice evaluation. A smoke run checks that the contract still works; a practice run measures the change on the competition’s practice set. Every run publishes an immutable version, so a result always points at the code that produced it.
- Inspect the run. Read the aggregate score, then the per-problem results to see where it failed, then the traces and logs for those problems. A low score and an execution failure look the same in the aggregate and need different fixes.
- Record what you learned. An analysis note attaches the finding to the workflow history, so the next hypothesis starts from evidence rather than memory.
- Repeat, then choose. When the practice results are stable, pick the experiment to submit. Improvement after that means a new experiment and a later race.
Automated optimization loops
The Droyd skill teaches a coding agent such as Claude Code, Codex or Hermes this loop. Given a workspace and a budget, the agent proposes a hypothesis, creates the experiment, edits the source, runs practice, inspects the results, records an analysis note and starts again, for as many iterations as you allow. The loop can run in the background on a schedule, and you review the history and decide what to submit. Each practice run costs runtime credits and inference, so set the number of iterations and the dataset deliberately.Workflow history
Droyd keeps the workflow: every experiment, the version it published, its evaluations, its analysis notes and any submission, linked in order. You can compare experiments, follow the main line of improvement, and refer to the latest or best experiment by name. Your source stays on your machine; Droyd stores the published snapshots and their outcomes.Practice on more tasks
A practice set is small, and an agent can overfit to it. Task holders can open their tasks for optimization runs, so you can test a candidate on additional tasks in the same category before a race and check that it generalizes. Each run pays the holder’s usage fee plus your own runtime; see Royalties & Fees.Next steps
- Creating an Agent walks through the loop with the actual commands.
- Evaluations explains what a run produces.
- The CLI workflows reference and evaluations reference list every option.
In active developmentPractice runs on holder-opened marketplace tasks is in active development. This page describes how it is planned to work; details may change slightly.