Creating Tasks

Creating a task is design work before it is code. The author decides what a finished result looks like and how a verifier will recognize it; the tooling turns that decision into a folder Droyd can run.

​

Start from the check

A task is only as good as its verifier. Before writing anything, state in one sentence what the agent must produce and in another how a program will decide it is right without trusting the agent. If the second sentence cannot be written, the task cannot be scored. An agent’s own report that it finished is never evidence.

​

What you write

• The description and instructions the agent reads, phrased as an outcome.
• The workspace: the files the agent may read, and for repository tasks the checkout it receives.
• Task rows, tools and reward: a small Python package that declares the task, any custom tools, and the reward logic that compares the result with private data. Hidden tests live alongside for repository tasks.
• The world, when the task needs it: service fixtures and a registry of the people and organizations that appear in them. See World Data.

Everything the agent may see is separated from everything only the verifier may see, by file role, so a leak has to be authored rather than accidental.

​

Revisions and debug runs

Each push saves an immutable private revision; nothing runs. A debug run then attempts a revision with a model from Droyd’s catalog and records a trajectory for every rollout. Reading successful trajectories is how an author finds shortcuts: an answer quoted in a README, a tool that returns the whole result, a filename that gives the game away. Add a complication only when it changes the solution path, push, rerun, and compare solve rates between revisions. Debug runs are never official measurements.

​

What makes a good task

• It asks for a business outcome and a destination, not a script of tool calls.
• A strong model solves it sometimes, not always, and not by luck.
• Its data looks like real work: consistent people, plausible history, a starting state that does not already pass the checks.
• Its answer is nowhere in the agent-visible world.
• Its verifier is independent, deterministic, and scores an empty attempt as a failure.

​

Automating task creation

Task creation does not have to be a sit-down exercise. A scheduled action for your standard agent, whether Claude Code, Codex or Hermes, can inspect recent chat history to find real problems where the agent had to fix or optimize something, then use the Droyd CLI and skills to create a task from it, anonymize it, refine it against debug runs, and submit it, all in a scheduled background run. Tasks drawn from real work this way tend to be harder and more realistic than invented puzzles.

​

Next steps

• Initialize Task is the step-by-step guide.
• Submitting Tasks explains what happens when a draft is ready.