where the models come from
Add every model this pipeline should use. Test sends one tiny request (one word, 16 tokens at most), so you know it works before a run depends on it. Keys stay on this machine and are never shown again.
which model answers, and who takes over when it fails
A custom provider on the agent route must speak the Responses API, and its key is handed to the task's sandbox, which runs contributor code with internet access: use a key with a spending limit.
This pipeline runs a cheat trial. With nothing in this route the cheat trials use the model of the Agent trial route; an entry here gives them a model of their own. A custom provider must speak the Responses API, and its key is handed to the red-team sandbox.
Entries are tried top to bottom. Effort is how long the model thinks (default = the pipeline's own setting). With nothing in a route, the pipeline uses its built-in Codex default for that role (an empty cheat route follows the Agent trial route instead). With Codex first and another provider behind it, a stage moves on instead of waiting for Codex's 5-hour window.
optional: a different model for one step
Every step that asks a model uses the Judges route above. Open a step to give it a model of its own (for example a stronger model for the rubric review, or a cheaper one for the semantic comparison); the other steps keep the Judges. Take every entry out of a step to send it back to the Judges. Same rules as any route: entries are tried top to bottom, Test sends one tiny request. Only the admin can change this, and an agent cannot.
what every run of this pipeline executes
A checked step runs. An unchecked step is skipped in every run of this pipeline, manual and API alike, and is listed in the run as disabled, never hidden. Only the admin can change this; an agent's API submission always follows it.
where the stages, judges and report execute
Auto uses Daytona whenever it has a key and is ready, and local Docker otherwise. If Daytona fails while a run is going, the stage runs again on Docker and the rest of the run stays there (a paid agent trial is never repeated by itself). Daytona builds the runner image once as a snapshot, runs every stage, the judges and the report of a run in one sandbox that is deleted when the run ends, and uses your Daytona quota; its SDK is installed into vendor/ on first use.
ChatGPT logins, in priority order
Each account is a path to an auth.json from codex login. The pipeline picks the first with headroom and switches on rate limits; the run page shows which account worked on each stage.
identity and availability
how much a run may spend
A local run always executes one agent trial unless allow_multi_trials is set in config.json. How many runs go at once is an
app setting (all pipelines share the machine).
this cannot be undone
Removes the pipeline with its settings, API key, projects and runs. Type its id to confirm.