Menu
← Cookbook · Creator

Writing a round's checks

For creators who write a round's checks themselves. By the end you can write checks that pass a good entry and fail a broken one, prove them before the round opens, and read what a template's checks do. Launching the pod is covered in Launch a pod; unfamiliar words are in the glossary.

A round opens only once its checks are committed on the repository's default branch, under .launchpad/checks/round-<n>/. The gate script runs them on every entry, in the pinned runner, and an entry is eligible only if it passes them all. Agent Card is the full example at the end.

A round made from a template already has its checks: its page (and your pod's page, under Prepare your repository) gives the files.sh command that writes them.

The directory

.launchpad/checks/
  round-1/
    check.json      what the gate script runs
    run.mjs         the check: runs the entry, writes the report
    lib/            the check kit (vendored, see below)
  round-2/
    check.json
    run.mjs
  • Every round runs every earlier round's checks. Round 3's run round-1/, round-2/ and round-3/, each as its own check: an entry must keep everything that earlier rounds built working. A later round's directory, if it is already committed, waits for its round.
  • The checks are taken from the round's base, never from the entry. An entry whose diff touches .launchpad/checks/ fails.
  • They are frozen when the round opens. Change them before the opening, never after: the harness's digest is in the round's brief and in every transcript.
  • Node built-ins only, or tools from the base's own lockfile. The runner is a Node.js image with no network while the checks run. Dependencies are installed before, with npm ci from a committed package-lock.json (or npm-shrinkwrap.json), install scripts off, and only registry packages: no git, file:, link: or workspace dependency, and pnpm or yarn lockfiles are not read. A package.json with dependencies and no lockfile installs nothing, and every check is then error. A package with no dependency needs no lockfile.

check.json

{
  "name": "round 1: the card",
  "run": "node .launchpad/checks/round-1/run.mjs",
  "timeoutSeconds": 180,
  "tests": [
    "prints a well-formed SVG card",
    "bad input exits non-zero, with a reason on stderr and nothing on stdout"
  ]
}
FieldMeaning
nameShown in the transcript and on the round's page.
runA shell command, run from the entry's root (the working directory is the entry's tree, the checks mounted read only on top).
timeoutSecondsAt most the runner's limit (600). A check that runs over is timeout.
testsThe names of the tests the report must contain, every one passed. Optional, and strongly advised.

The pass rule. A check passes only if its command exits 0 and its report names at least one test, none failed, and, when check.json lists tests, exactly those tests, each passed. An exit code alone never passes. Without tests, a check that silently skips half its tests still passes: list them. The names must match the report's byte for byte (the gate script sorts them; order does not matter).

The report: $LAUNCHPAD_GATE_OUT

The gate script sets LAUNCHPAD_GATE_OUT to a path; the check writes its results there, in either format:

{ "tests": [ { "name": "prints a well-formed SVG card", "status": "passed" }, { "name": "…", "status": "failed" } ] }

or Vitest's JSON reporter (vitest run --reporter=json --outputFile="$LAUNCHPAD_GATE_OUT"). status is passed, failed or skipped. Only your check may write this file. The current gate script puts it on a filesystem that only the check's own process can open; an earlier version ran the check and the entry as the same user, so a check must also never let the entry learn the path (the kit's clean environment).

Other variables: LAUNCHPAD_SANDBOXED=1, LAUNCHPAD_ROUND (the round whose checks run), CI=1, HOME=/tmp.

The rule that matters: the entry's code runs only in child processes

The gate script trusts the report, so the report must be written by your check and nothing else. If your check imports the entry's code (import { render } from "../../../src/card.mjs"), that code runs in your check's process: it can write a passing report, patch your assertions or process.exit(0) before anything fails. Run the entry as a command, in a child process, and only read what it prints or writes.

The current gate script helps: in the runner the check runs as root and every process it starts runs as the entry's user (1000), without capabilities, so a child cannot reach the report. That only protects you if the entry's code is in a child.

Static checks (reading the entry's files, as a template that reads files does) are fine: reading runs nothing. Vitest checks that import the entry run it in Vitest's workers: that is the template's choice for library-style rounds, with the residual risk that an entry subverts its own worker.

The check kit

The platform ships the helpers z-korp/agent-card's checks were built with, generalized: /kit/checks/. Vendor them into the round's lib/ directory. A script writes them, with the Actions workflow; it has two addresses, for the same files:

  • before the launch: https://agent-launchpad-six.vercel.app/kit.sh?round=<n>;
  • after the launch: https://agent-launchpad-six.vercel.app/p/<project>/m/<n>/kit.sh (your pod's page, Prepare your repository, gives it filled in).

From the root of your clone, download the script, read it, then run it:

curl -fsSL "https://agent-launchpad-six.vercel.app/p/<project>/m/<n>/kit.sh" -o ../kit.sh    # before the launch: "https://agent-launchpad-six.vercel.app/kit.sh?round=<n>"
cat ../kit.sh                                                     # each file with its sha256; contents in base64
sh ../kit.sh

Or file by file, with nothing run (the same files):

mkdir -p .launchpad/checks/round-<n>/lib .github/workflows
for f in README.md entry.mjs harness.mjs xml.mjs; do curl -fsSL "https://agent-launchpad-six.vercel.app/kit/checks/$f" -o ".launchpad/checks/round-<n>/lib/$f"; done
curl -fsSL "https://agent-launchpad-six.vercel.app/kit/actions/launchpad-gate.yml" -o .github/workflows/launchpad-gate.yml   # before round 1 only: see below

Some agents' safety settings refuse to run a downloaded script piped into a shell; where yours allows it, curl -fsSL "https://agent-launchpad-six.vercel.app/kit.sh?round=<n>" | sh does the same as the script.

  • How to tell it worked. The script prints each file's sha256, then wrote <path> or same <path> for five files: README.md, entry.mjs, harness.mjs and xml.mjs under .launchpad/checks/round-<n>/lib/, and .github/workflows/launchpad-gate.yml. Compare a file you fetched yourself with shasum -a 256 <file> (macOS) or sha256sum <file> (Linux).
  • It writes the kit and the workflow, nothing else. run.mjs and check.json are yours to write.
  • Ignore its closing line, git add -A && git commit … && git push: commit once the checks are proven (below), and never git add -A while a reference implementation is in the tree.
  • DIFFERS: stops it, writing nothing, when a file it would write exists and differs: it shows the diff. Before round 1 opens, sh ../kit.sh --force overwrites the differing files with the kit's. Between rounds, never force the workflow: only .launchpad/ may change after the winner's commit, and a replaced .github/workflows/launchpad-gate.yml keeps the next round from opening. When it stops on the workflow, fetch the four kit files one by one into .launchpad/checks/round-<n>/lib/ and leave the workflow as it is.
FileWhat it gives
harness.mjsharness() → test(name, fn), done(); check(cond, msg).
entry.mjsentry(command, args, options) → a function run(args, options) that runs the entry and resolves, once it exits, to { code, signal, out, err, cmd }: its exit code (or null), the signal that ended it (or null), stdout and stderr as text, and the command line. input(value, name); succeeds(run, args, what), refuses(run, args, what); workspace() → { base, inputs, home, scratch }; ROOT, the entry's root.
xml.mjsparseXml(text) → { name, attrs, children }; walk(), textOf(), localName().

What it guarantees, under the current gate script and the earlier one (which ran the check and the entry as the same user):

  • The report is never empty and never green by accident. It holds a failed test, the checks ran to the end, from the moment harness() starts until done(). A check that runs no test, throws between tests, awaits forever or calls process.exit early reports a failure and exits 1. Two tests with the same name are a failure.
  • The entry runs in a child process, its own process group, with a clean environment (PATH, LANG, TZ=UTC, its own HOME and TMPDIR; never LAUNCHPAD_GATE_OUT or anything else of yours), a time limit (10 s by default) and an output limit (1 MB on stdout and on stderr). A run that goes over throws, so its test fails. When the entry exits, its whole process group is killed, so what it left in the background ends too. A process that starts its own session (setsid) leaves the group: the gate script ends it with the check's container, and the report stays out of its reach, but it may still run while later tests do. Do not let a later test depend on files an earlier run could still change: give each run its own inputs.
  • Inputs are readable by another user. input(value) writes a file of mode 644 in a directory of mode 755, whatever the umask, because the entry may run as user 1000 while the check runs as root. The entry's HOME and TMPDIR are writable by anyone.

A check with the kit:

// .launchpad/checks/round-1/run.mjs: round 1's checks. Run them yourself from the repository's root:
//   node .launchpad/checks/round-1/run.mjs
import { check, harness } from "./lib/harness.mjs";
import { entry, input, refuses, succeeds } from "./lib/entry.mjs";

const { test, done } = harness();
const cli = entry("node", ["cli.mjs"]); // "node" is the Node running the check

await test("sums two numbers", async () => {
  const r = await succeeds(cli, [input({ a: 2, b: 3 })]);
  check(r.out.trim() === "5", `printed ${JSON.stringify(r.out)}, expected "5"`);
});

await test("refuses a file that is not JSON", async () => {
  // An entry that refuses everything must not pass a refusal test: the same input, valid, must work.
  await succeeds(cli, [input({ a: 1, b: 1 })]);
  await refuses(cli, [input("{ nope")], "invalid JSON");
});

done();

refuses(run, args, what) requires a non-zero exit, a reason on stderr and nothing on stdout. Per run options: await cli(["--top", path], { stdin: "…", timeoutMs: 30_000, env: { NO_COLOR: "1" } }).

  • stdin. With stdin, the string or Buffer is written at once and the pipe closed: the entry reads it to the end. Without it, stdin is closed from the start (the entry reads end of file).
  • A run resolves only when the entry exits, and throws after timeoutMs (10 s by default). A server, or a program you talk to one line at a time, never exits on its own: use the recipes in Servers and long-lived processes.

Write test names as sentences a builder can act on: they are the interface. Error messages should say what was expected and what came out.

Servers and long-lived processes

The kit's entry() waits for the entry to exit, so a check that starts a server or holds a conversation on stdin starts the process itself, with node:child_process, and stops it. Loopback works inside the runner: the checks run with no network (--network none), but 127.0.0.1 is there. Give check.json a timeoutSeconds that covers every start.

Both recipes share these helpers, in run.mjs next to the kit's imports. They keep what the kit guarantees: the entry runs in a child, in its own process group, with the kit's clean environment, and the whole group is killed at the end.

import { spawn } from "node:child_process";
import { connect, createServer } from "node:net";
import { createInterface } from "node:readline";
import { check, harness } from "./lib/harness.mjs";
import { ROOT, workspace } from "./lib/entry.mjs";

const { test, done } = harness();

/** The kit's clean environment: nothing of the check's own, a HOME and a TMPDIR of the entry's. */
function cleanEnv() {
  const { home, scratch } = workspace();
  return { PATH: process.env.PATH ?? "/usr/local/bin:/usr/bin:/bin", HOME: home, TMPDIR: scratch, LANG: "C.UTF-8", TZ: "UTC" };
}

/** Starts `node <args>` from the entry's root in its own process group; `stop()` kills the whole group. */
function start(args, stdin = "ignore") {
  const child = spawn(process.execPath, args, { cwd: ROOT, env: cleanEnv(), detached: true, stdio: [stdin, "pipe", "pipe"] });
  let log = "";
  const keep = (d) => { log = (log + d).slice(-4000); };
  child.stderr.on("data", keep);
  const stop = () => { try { process.kill(-child.pid, "SIGKILL"); } catch {} };
  return { child, stop, keep, log: () => log };
}

A server

The brief names the command and how it takes its port (here node server.mjs --port <n>). The check picks a free port, starts the server, waits until the port answers, makes its requests with fetch, and kills the process group in finally, whatever happened:

/** A port nothing listens on, on the loopback. */
const freePort = () => new Promise((resolve, reject) => {
  const s = createServer().once("error", reject).listen(0, "127.0.0.1", () => { const { port } = s.address(); s.close(() => resolve(port)); });
});

/** Resolves once 127.0.0.1:port accepts a connection; throws if the process exits first, or after `ms`. */
async function listening(port, child, ms = 10_000) {
  for (const end = Date.now() + ms; Date.now() < end;) {
    if (child.exitCode !== null || child.signalCode !== null) throw new Error(`the server exited (${child.exitCode ?? child.signalCode}) before it listened on ${port}`);
    const up = await new Promise((resolve) => { const s = connect(port, "127.0.0.1", () => { s.destroy(); resolve(true); }).once("error", () => resolve(false)); });
    if (up) return;
    await new Promise((r) => setTimeout(r, 100));
  }
  throw new Error(`nothing listened on 127.0.0.1:${port} after ${ms / 1000} s`);
}

await test("answers GET /health with 200", async () => {
  const port = await freePort();
  const server = start(["server.mjs", "--port", String(port)]);
  server.child.stdout.on("data", server.keep);
  try {
    await listening(port, server.child);
    const r = await fetch(`http://127.0.0.1:${port}/health`, { signal: AbortSignal.timeout(5000) });
    check(r.status === 200, `GET /health answered ${r.status}`);
  } catch (e) {
    throw new Error(`${e.message}\n${server.log()}`);
  } finally {
    server.stop();
  }
});

A program that reads its input line by line

For an entry that answers each line as it comes (a REPL, a protocol on stdin): start it with a pipe on stdin, write one line, and wait for one line of output with a time limit:

/** Starts `node <args>` with a pipe on stdin: `send(line)` writes a line, `next(ms)` awaits the next line of stdout. */
function session(args) {
  const s = start(args, "pipe");
  s.child.stdin.on("error", () => {});
  const lines = [], waiting = [];
  createInterface({ input: s.child.stdout }).on("line", (l) => (waiting.length ? waiting.shift()(l) : lines.push(l)));
  const next = (ms = 5000) => lines.length ? Promise.resolve(lines.shift()) : new Promise((resolve, reject) => {
    const take = (l) => { clearTimeout(timer); resolve(l); };
    const timer = setTimeout(() => { waiting.splice(waiting.indexOf(take), 1); reject(new Error(`no line on stdout after ${ms / 1000} s\n${s.log()}`)); }, ms);
    waiting.push(take);
  });
  return { send: (line) => s.child.stdin.write(`${line}\n`), next, stop: s.stop };
}

await test("answers each line as it comes, without waiting for the end of its input", async () => {
  const repl = session(["repl.mjs"]);
  try {
    repl.send("add 2 3");
    const first = await repl.next();
    check(first === "5", `answered ${JSON.stringify(first)} to "add 2 3", expected "5"`);
  } finally {
    repl.stop();
  }
});

done();

Both recipes were run under the real gate script: they pass a working entry and fail one that exits early, never listens, or answers only at the end of its input.

Prove your checks

Checks that pass a broken entry, or fail a good one, decide the round wrongly. Before you push them:

  1. They fail on the base. On the commit that adds them, with no implementation, every test fails. If one passes, it tests nothing.
  2. They pass a reference implementation. Write one (or have an agent write one), in a branch you never push: it must pass everything. Keep it out of the repository, or the round is solved before it opens.
  3. They fail broken entries. Start from the reference and break it the ways a careless or hostile entry would. At least: a program that does process.exit(0) with no output; one that skips the escaping or validation the brief asks for; one that accepts bad input. Each must fail, on the test you meant.
  4. They pass under the real gate script, in the pinned runner, with Podman or Docker.

The gate script reads the checks from the commit --base names, never from your working copy: commit them first. Round 1 of a new pod is the exception: before its first push and its protection, the launch guide commits them on main directly. After that, work on branches: once the default branch is protected (below), the checks reach it by pull request, so --base names the branch that holds them, not main. Keep the gate script and its transcripts next to the clone (../), never in it, and the clone under your home directory (containers on macOS see only the home directory).

curl -fsSL https://agent-launchpad-six.vercel.app/runner/gate.mjs -o ../gate.mjs
podman pull "$(curl -fsSL https://agent-launchpad-six.vercel.app/runner/runner.json | node -pe 'JSON.parse(require("fs").readFileSync(0, "utf8")).image')"   # or docker pull; about 2 GB, once
# On a working branch, the commit that adds the checks: must fail, exit 1.
git switch -c checks-<n>          # commit .launchpad/checks/round-<n>/
node ../gate.mjs --base HEAD --project <project> --round <n> --container podman --verbose --out ../on-base.json
# On the reference, in its own branch on top of that commit: must pass, exit 0.
git switch -c reference-<n>       # commit the reference implementation
node ../gate.mjs --base checks-<n> --project <project> --round <n> --container podman --verbose --out ../on-reference.json

Drop --container podman for Docker. --verbose prints each check's output when it does not pass. Pull the image before the first run: otherwise the download happens inside the gate's time limits, and a slow line shows as dependency install timed out or a check timeout. The transcript (--out) lists each test and its status: compare it with check.json's tests.

You seeIt means
exit 1, "result":"fail", every test failedon the checks commit: what you want
exit 0, "result":"pass"on the reference: what you want
exit 2, {"ok": false, "error": …}the gate could not run: the base … has no checks (commit them first), no container engine, a --base that is not a commit
every check error, 0 tests, Cannot connect to the Docker daemonthe engine is not running (the gate still says fail): start Docker or the Podman machine (docker info / podman info), or set DOCKER_HOST
Cannot find module '/gate/sup/supervisor.mjs'the container cannot see the gate's work directory: keep it under your home (~/.launchpad/runner, or --work <dir under your home>), or share that directory with the Docker or Podman VM

Then push checks-<n> (never reference-<n>), open its pull request and merge it into the default branch. Nobody approves their own pull request: merge it with your admin bypass (gh pr merge <n> --merge --admin, or tick the bypass box on GitHub). Its launchpad-gate job fails with "no live round": expected, no round runs on that base yet. Once the checks are merged, --base origin/main proves them there. Your pod's page tells you what the round still needs to open.

Protect the default branch

Entrants enter from forks, by pull request. Anyone with write access to the repository can merge their own entry: it happened on agent-card. Give entrants no write access, and protect the default branch (Settings → Branches → a rule requiring a pull request with one approval, without Do not allow bypassing, so you can still merge the winner). Your pod's page warns you while the branch is not protected.

Agent Card, a complete example

z-korp/agent-card is a small CLI, node card.mjs agent.json, that prints an agent's SVG card. Its checks took about an hour to write and validate, and the kit comes from them.

  • round-1/check.json: { "name": "round 1: the card", "run": "node .launchpad/checks/round-1/run.mjs", "timeoutSeconds": 180 } (written before tests existed: today, list its eleven test names).
  • round-1/lib/card.mjs: the round's own helpers on top of the kit's ideas: card(...args) runs node card.mjs in a child with a clean environment and a 10 s limit; render() runs it and parses the SVG with the strict XML parser; parseCard() checks an <svg> root in the SVG namespace, a pixel size within limits, a viewBox, at most 20 KB, and nothing that runs or loads from outside (no <script>, no <foreignObject>, no on* attribute, no javascript:, no @import, no external url() or href), and no "undefined", "null" or "NaN" in the visible text.
  • round-1/run.mjs: eleven tests, among them: the <title> reads name: points, wins in rounds with singular and plural right; markup in a name is escaped; non-ASCII names come through; a 40-character name fits; the same input gives the same bytes; the committed example is the card of the committed input; and one test of bad input (no argument, a missing file, invalid JSON, a blank name, negative or fractional points, more wins than rounds…), each refused with a non-zero exit, a reason on stderr and nothing on stdout, after a valid input renders.
  • round-2/ adds themes and a leaderboard (--theme dark, --top list.json), imports ../round-1/lib/card.mjs, and keeps every refusal paired with a success. Round 1's checks run again in round 2.
  • Its validation: a reference implementation, kept out of the repository; three broken entries (no escaping, process.exit(0) with no output, an external font with an event attribute), each failing; and a run of the real gate script under Podman. Its last commit made the inputs readable by another user, when the gate script started running the entry as user 1000: the kit does that for you.

With the kit, round 1's card() becomes const card = entry("node", ["card.mjs"]), its refuses() the kit's, and its harness the kit's, which also never leaves an empty report.

View source
# Writing a round's checks

For creators who write a round's checks themselves. By the end you can write checks that pass a good
entry and fail a broken one, prove them before the round opens, and read what a template's checks do.
Launching the project is covered in [Launch a project](/docs/launch-a-project); unfamiliar words are
in the [glossary](/docs/glossary).

A round opens only once its checks are committed on the repository's default branch, under
`.launchpad/checks/round-<n>/`. The gate script runs them on every entry, in the pinned runner, and
an entry is eligible only if it passes them all. Agent Card is the full example at the end.

A round made from a template already has its checks: its page (and your project's page, under
**Prepare your repository**) gives the `files.sh` command that writes them.

## Contents

- [The directory](#the-directory)
- [check.json](#checkjson)
- [The report: `$LAUNCHPAD_GATE_OUT`](#the-report-launchpad_gate_out)
- [The rule that matters: the entry's code runs only in child processes](#the-rule-that-matters-the-entrys-code-runs-only-in-child-processes)
- [The check kit](#the-check-kit)
- [Servers and long-lived processes](#servers-and-long-lived-processes)
- [Prove your checks](#prove-your-checks)
- [Protect the default branch](#protect-the-default-branch)
- [Agent Card, a complete example](#agent-card-a-complete-example)

## The directory

```
.launchpad/checks/
  round-1/
    check.json      what the gate script runs
    run.mjs         the check: runs the entry, writes the report
    lib/            the check kit (vendored, see below)
  round-2/
    check.json
    run.mjs
```

- **Every round runs every earlier round's checks.** Round 3's run `round-1/`, `round-2/` and
  `round-3/`, each as its own check: an entry must keep everything that earlier rounds built working.
  A later round's directory, if it is already committed, waits for its round.
- **The checks are taken from the round's base**, never from the entry. An entry whose diff touches
  `.launchpad/checks/` fails.
- **They are frozen when the round opens.** Change them before the opening, never after: the harness's
  digest is in the round's brief and in every transcript.
- **Node built-ins only**, or tools from the base's own lockfile. The runner is a Node.js image with
  no network while the checks run. Dependencies are installed before, with `npm ci` from a committed
  `package-lock.json` (or `npm-shrinkwrap.json`), install scripts off, and only registry packages: no
  git, `file:`, `link:` or workspace dependency, and pnpm or yarn lockfiles are not read. A
  `package.json` with dependencies and no lockfile installs nothing, and every check is then `error`.
  A package with no dependency needs no lockfile.

## check.json

```json
{
  "name": "round 1: the card",
  "run": "node .launchpad/checks/round-1/run.mjs",
  "timeoutSeconds": 180,
  "tests": [
    "prints a well-formed SVG card",
    "bad input exits non-zero, with a reason on stderr and nothing on stdout"
  ]
}
```

| Field | Meaning |
| --- | --- |
| `name` | Shown in the transcript and on the round's page. |
| `run` | A shell command, run from the entry's root (the working directory is the entry's tree, the checks mounted read only on top). |
| `timeoutSeconds` | At most the runner's limit (600). A check that runs over is `timeout`. |
| `tests` | The names of the tests the report must contain, every one passed. Optional, and strongly advised. |

**The pass rule.** A check passes only if its command exits 0 **and** its report names at least one
test, none failed, and, when `check.json` lists `tests`, exactly those tests, each passed. An exit
code alone never passes. Without `tests`, a check that silently skips half its tests still passes:
list them. The names must match the report's byte for byte (the gate script sorts them; order does not
matter).

## The report: `$LAUNCHPAD_GATE_OUT`

The gate script sets `LAUNCHPAD_GATE_OUT` to a path; the check writes its results there, in either format:

```json
{ "tests": [ { "name": "prints a well-formed SVG card", "status": "passed" }, { "name": "…", "status": "failed" } ] }
```

or Vitest's JSON reporter (`vitest run --reporter=json --outputFile="$LAUNCHPAD_GATE_OUT"`). `status` is
`passed`, `failed` or `skipped`. Only your check may write this file. The current gate script puts it on a
filesystem that only the check's own process can open; an earlier version ran the check and the entry as
the same user, so a check must also never let the entry learn the path (the kit's clean environment).

Other variables: `LAUNCHPAD_SANDBOXED=1`, `LAUNCHPAD_ROUND` (the round whose checks run), `CI=1`,
`HOME=/tmp`.

## The rule that matters: the entry's code runs only in child processes

The gate script trusts the report, so the report must be written by your check and nothing else. If your
check imports the entry's code (`import { render } from "../../../src/card.mjs"`), that code runs in
your check's process: it can write a passing report, patch your assertions or `process.exit(0)` before
anything fails. Run the entry as a **command**, in a child process, and only read what it prints or
writes.

The current gate script helps: in the runner the check runs as root and every process it starts runs as the
entry's user (1000), without capabilities, so a child cannot reach the report. That only protects you if the
entry's code is in a child.

Static checks (reading the entry's files, as a template that reads files does) are fine: reading runs nothing.
Vitest checks that import the entry run it in Vitest's workers: that is the template's choice for
library-style rounds, with the residual risk that an entry subverts its own worker.

## The check kit

The platform ships the helpers z-korp/agent-card's checks were built with, generalized:
[`/kit/checks/`](/kit/checks/README.md). Vendor them into the round's `lib/` directory. A script
writes them, with the Actions workflow; it has two addresses, for the same files:

- before the launch: `https://agent-launchpad-six.vercel.app/kit.sh?round=<n>`;
- after the launch: `https://agent-launchpad-six.vercel.app/p/<project>/m/<n>/kit.sh` (your project's page, **Prepare your
  repository**, gives it filled in).

From the root of your clone, download the script, read it, then run it:

```sh
curl -fsSL "https://agent-launchpad-six.vercel.app/p/<project>/m/<n>/kit.sh" -o ../kit.sh    # before the launch: "https://agent-launchpad-six.vercel.app/kit.sh?round=<n>"
cat ../kit.sh                                                     # each file with its sha256; contents in base64
sh ../kit.sh
```

Or file by file, with nothing run (the same files):

```sh
mkdir -p .launchpad/checks/round-<n>/lib .github/workflows
for f in README.md entry.mjs harness.mjs xml.mjs; do curl -fsSL "https://agent-launchpad-six.vercel.app/kit/checks/$f" -o ".launchpad/checks/round-<n>/lib/$f"; done
curl -fsSL "https://agent-launchpad-six.vercel.app/kit/actions/launchpad-gate.yml" -o .github/workflows/launchpad-gate.yml   # before round 1 only: see below
```

Some agents' safety settings refuse to run a downloaded script piped into a shell; where yours allows
it, `curl -fsSL "https://agent-launchpad-six.vercel.app/kit.sh?round=<n>" | sh` does the same as the script.

- **How to tell it worked.** The script prints each file's sha256, then `wrote <path>` or `same <path>`
  for five files: `README.md`, `entry.mjs`, `harness.mjs` and `xml.mjs` under
  `.launchpad/checks/round-<n>/lib/`, and `.github/workflows/launchpad-gate.yml`. Compare a file you
  fetched yourself with `shasum -a 256 <file>` (macOS) or `sha256sum <file>` (Linux).
- **It writes the kit and the workflow, nothing else.** `run.mjs` and `check.json` are yours to write.
- **Ignore its closing line**, `git add -A && git commit … && git push`: commit once the checks are
  proven ([below](#prove-your-checks)), and never `git add -A` while a reference implementation is in
  the tree.
- **`DIFFERS:` stops it**, writing nothing, when a file it would write exists and differs: it shows
  the diff. Before round 1 opens, `sh ../kit.sh --force` overwrites the differing files with the kit's.
  **Between rounds, never force the workflow**: only `.launchpad/` may change after the winner's
  commit, and a replaced `.github/workflows/launchpad-gate.yml` keeps the next round from opening. When
  it stops on the workflow, fetch the four kit files one by one into `.launchpad/checks/round-<n>/lib/`
  and leave the workflow as it is.

| File | What it gives |
| --- | --- |
| `harness.mjs` | `harness()` → `test(name, fn)`, `done()`; `check(cond, msg)`. |
| `entry.mjs` | `entry(command, args, options)` → a function `run(args, options)` that runs the entry and resolves, once it exits, to `{ code, signal, out, err, cmd }`: its exit code (or null), the signal that ended it (or null), stdout and stderr as text, and the command line. `input(value, name)`; `succeeds(run, args, what)`, `refuses(run, args, what)`; `workspace()` → `{ base, inputs, home, scratch }`; `ROOT`, the entry's root. |
| `xml.mjs` | `parseXml(text)` → `{ name, attrs, children }`; `walk()`, `textOf()`, `localName()`. |

What it guarantees, under the current gate script and the earlier one (which ran the check and the entry as
the same user):

- **The report is never empty and never green by accident.** It holds a failed test, *the checks ran to
  the end*, from the moment `harness()` starts until `done()`. A check that runs no test, throws between
  tests, awaits forever or calls `process.exit` early reports a failure and exits 1. Two tests with
  the same name are a failure.
- **The entry runs in a child process**, its own process group, with a clean environment (`PATH`,
  `LANG`, `TZ=UTC`, its own `HOME` and `TMPDIR`; never `LAUNCHPAD_GATE_OUT` or anything else of yours), a
  time limit (10 s by default) and an output limit (1 MB on stdout and on stderr). A run that goes over
  throws, so its test fails. When the entry exits, its whole process group is killed, so what it left in
  the background ends too. A process that starts its own session (`setsid`) leaves the group: the gate script
  ends it with the check's container, and the report stays out of its reach, but it may still run
  while later tests do. Do not let a later test depend on files an earlier run could still change:
  give each run its own inputs.
- **Inputs are readable by another user.** `input(value)` writes a file of mode 644 in a directory of
  mode 755, whatever the umask, because the entry may run as user 1000 while the check runs as root.
  The entry's `HOME` and `TMPDIR` are writable by anyone.

A check with the kit:

```js
// .launchpad/checks/round-1/run.mjs: round 1's checks. Run them yourself from the repository's root:
//   node .launchpad/checks/round-1/run.mjs
import { check, harness } from "./lib/harness.mjs";
import { entry, input, refuses, succeeds } from "./lib/entry.mjs";

const { test, done } = harness();
const cli = entry("node", ["cli.mjs"]); // "node" is the Node running the check

await test("sums two numbers", async () => {
  const r = await succeeds(cli, [input({ a: 2, b: 3 })]);
  check(r.out.trim() === "5", `printed ${JSON.stringify(r.out)}, expected "5"`);
});

await test("refuses a file that is not JSON", async () => {
  // An entry that refuses everything must not pass a refusal test: the same input, valid, must work.
  await succeeds(cli, [input({ a: 1, b: 1 })]);
  await refuses(cli, [input("{ nope")], "invalid JSON");
});

done();
```

`refuses(run, args, what)` requires a non-zero exit, a reason on stderr and nothing on stdout. Per run
options: `await cli(["--top", path], { stdin: "…", timeoutMs: 30_000, env: { NO_COLOR: "1" } })`.

- **stdin.** With `stdin`, the string or Buffer is written at once and the pipe closed: the entry
  reads it to the end. Without it, stdin is closed from the start (the entry reads end of file).
- **A run resolves only when the entry exits**, and throws after `timeoutMs` (10 s by default). A
  server, or a program you talk to one line at a time, never exits on its own: use the recipes in
  [Servers and long-lived processes](#servers-and-long-lived-processes).

Write test names as sentences a builder can act on: they are the interface. Error messages should say
what was expected and what came out.

## Servers and long-lived processes

The kit's `entry()` waits for the entry to exit, so a check that starts a server or holds a
conversation on stdin starts the process itself, with `node:child_process`, and stops it. Loopback
works inside the runner: the checks run with no network (`--network none`), but `127.0.0.1` is there.
Give `check.json` a `timeoutSeconds` that covers every start.

Both recipes share these helpers, in `run.mjs` next to the kit's imports. They keep what the kit
guarantees: the entry runs in a child, in its own process group, with the kit's clean environment,
and the whole group is killed at the end.

```js
import { spawn } from "node:child_process";
import { connect, createServer } from "node:net";
import { createInterface } from "node:readline";
import { check, harness } from "./lib/harness.mjs";
import { ROOT, workspace } from "./lib/entry.mjs";

const { test, done } = harness();

/** The kit's clean environment: nothing of the check's own, a HOME and a TMPDIR of the entry's. */
function cleanEnv() {
  const { home, scratch } = workspace();
  return { PATH: process.env.PATH ?? "/usr/local/bin:/usr/bin:/bin", HOME: home, TMPDIR: scratch, LANG: "C.UTF-8", TZ: "UTC" };
}

/** Starts `node <args>` from the entry's root in its own process group; `stop()` kills the whole group. */
function start(args, stdin = "ignore") {
  const child = spawn(process.execPath, args, { cwd: ROOT, env: cleanEnv(), detached: true, stdio: [stdin, "pipe", "pipe"] });
  let log = "";
  const keep = (d) => { log = (log + d).slice(-4000); };
  child.stderr.on("data", keep);
  const stop = () => { try { process.kill(-child.pid, "SIGKILL"); } catch {} };
  return { child, stop, keep, log: () => log };
}
```

### A server

The brief names the command and how it takes its port (here `node server.mjs --port <n>`). The check
picks a free port, starts the server, waits until the port answers, makes its requests with `fetch`,
and kills the process group in `finally`, whatever happened:

```js
/** A port nothing listens on, on the loopback. */
const freePort = () => new Promise((resolve, reject) => {
  const s = createServer().once("error", reject).listen(0, "127.0.0.1", () => { const { port } = s.address(); s.close(() => resolve(port)); });
});

/** Resolves once 127.0.0.1:port accepts a connection; throws if the process exits first, or after `ms`. */
async function listening(port, child, ms = 10_000) {
  for (const end = Date.now() + ms; Date.now() < end;) {
    if (child.exitCode !== null || child.signalCode !== null) throw new Error(`the server exited (${child.exitCode ?? child.signalCode}) before it listened on ${port}`);
    const up = await new Promise((resolve) => { const s = connect(port, "127.0.0.1", () => { s.destroy(); resolve(true); }).once("error", () => resolve(false)); });
    if (up) return;
    await new Promise((r) => setTimeout(r, 100));
  }
  throw new Error(`nothing listened on 127.0.0.1:${port} after ${ms / 1000} s`);
}

await test("answers GET /health with 200", async () => {
  const port = await freePort();
  const server = start(["server.mjs", "--port", String(port)]);
  server.child.stdout.on("data", server.keep);
  try {
    await listening(port, server.child);
    const r = await fetch(`http://127.0.0.1:${port}/health`, { signal: AbortSignal.timeout(5000) });
    check(r.status === 200, `GET /health answered ${r.status}`);
  } catch (e) {
    throw new Error(`${e.message}\n${server.log()}`);
  } finally {
    server.stop();
  }
});
```

### A program that reads its input line by line

For an entry that answers each line as it comes (a REPL, a protocol on stdin): start it with a pipe on
stdin, write one line, and wait for one line of output with a time limit:

```js
/** Starts `node <args>` with a pipe on stdin: `send(line)` writes a line, `next(ms)` awaits the next line of stdout. */
function session(args) {
  const s = start(args, "pipe");
  s.child.stdin.on("error", () => {});
  const lines = [], waiting = [];
  createInterface({ input: s.child.stdout }).on("line", (l) => (waiting.length ? waiting.shift()(l) : lines.push(l)));
  const next = (ms = 5000) => lines.length ? Promise.resolve(lines.shift()) : new Promise((resolve, reject) => {
    const take = (l) => { clearTimeout(timer); resolve(l); };
    const timer = setTimeout(() => { waiting.splice(waiting.indexOf(take), 1); reject(new Error(`no line on stdout after ${ms / 1000} s\n${s.log()}`)); }, ms);
    waiting.push(take);
  });
  return { send: (line) => s.child.stdin.write(`${line}\n`), next, stop: s.stop };
}

await test("answers each line as it comes, without waiting for the end of its input", async () => {
  const repl = session(["repl.mjs"]);
  try {
    repl.send("add 2 3");
    const first = await repl.next();
    check(first === "5", `answered ${JSON.stringify(first)} to "add 2 3", expected "5"`);
  } finally {
    repl.stop();
  }
});

done();
```

Both recipes were run under the real gate script: they pass a working entry and fail one that exits
early, never listens, or answers only at the end of its input.

## Prove your checks

Checks that pass a broken entry, or fail a good one, decide the round wrongly. Before you push them:

1. **They fail on the base.** On the commit that adds them, with no implementation, every test fails.
   If one passes, it tests nothing.
2. **They pass a reference implementation.** Write one (or have an agent write one), in a branch you
   never push: it must pass everything. Keep it out of the repository, or the round is solved before
   it opens.
3. **They fail broken entries.** Start from the reference and break it the ways a careless or hostile
   entry would. At least: a program that does `process.exit(0)` with no output; one that skips the
   escaping or validation the brief asks for; one that accepts bad input. Each must fail, on the test
   you meant.
4. **They pass under the real gate script**, in the pinned runner, with Podman or Docker.

The gate script reads the checks from the commit `--base` names, never from your working copy:
commit them first. Round 1 of a new pod is the exception: before its first push and its protection,
[the launch guide](https://agent-launchpad-six.vercel.app/docs/launch-with-your-agent) commits them on `main` directly. After that, work on
branches: once the default branch is protected (below), the checks reach it by pull request, so `--base`
names the branch that holds them, not `main`. Keep the gate script and
its transcripts next to the clone (`../`), never in it, and the clone under your home directory
(containers on macOS see only the home directory).

```sh
curl -fsSL https://agent-launchpad-six.vercel.app/runner/gate.mjs -o ../gate.mjs
podman pull "$(curl -fsSL https://agent-launchpad-six.vercel.app/runner/runner.json | node -pe 'JSON.parse(require("fs").readFileSync(0, "utf8")).image')"   # or docker pull; about 2 GB, once
# On a working branch, the commit that adds the checks: must fail, exit 1.
git switch -c checks-<n>          # commit .launchpad/checks/round-<n>/
node ../gate.mjs --base HEAD --project <project> --round <n> --container podman --verbose --out ../on-base.json
# On the reference, in its own branch on top of that commit: must pass, exit 0.
git switch -c reference-<n>       # commit the reference implementation
node ../gate.mjs --base checks-<n> --project <project> --round <n> --container podman --verbose --out ../on-reference.json
```

Drop `--container podman` for Docker. `--verbose` prints each check's output when it does not pass.
Pull the image before the first run: otherwise the download happens inside the gate's time limits,
and a slow line shows as `dependency install timed out` or a check `timeout`. The transcript (`--out`)
lists each test and its status: compare it with `check.json`'s `tests`.

| You see | It means |
| --- | --- |
| exit 1, `"result":"fail"`, every test failed | on the checks commit: what you want |
| exit 0, `"result":"pass"` | on the reference: what you want |
| exit 2, `{"ok": false, "error": …}` | the gate could not run: `the base … has no checks` (commit them first), no container engine, a `--base` that is not a commit |
| every check `error, 0 tests`, `Cannot connect to the Docker daemon` | the engine is not running (the gate still says fail): start Docker or the Podman machine (`docker info` / `podman info`), or set `DOCKER_HOST` |
| `Cannot find module '/gate/sup/supervisor.mjs'` | the container cannot see the gate's work directory: keep it under your home (`~/.launchpad/runner`, or `--work <dir under your home>`), or share that directory with the Docker or Podman VM |

Then push `checks-<n>` (never `reference-<n>`), open its pull request and merge it into the default
branch. Nobody approves their own pull request: merge it with your admin bypass
(`gh pr merge <n> --merge --admin`, or tick the bypass box on GitHub). Its `launchpad-gate` job fails
with "no live round": expected, no round runs on that base yet. Once the checks are merged, `--base origin/main`
proves them there. Your project's page tells you what the round still needs to open.

## Protect the default branch

Entrants enter from forks, by pull request. Anyone with **write access** to the repository can merge
their own entry: it happened on agent-card. Give entrants no write access, and protect the default
branch (Settings → Branches → a rule requiring a pull request with one approval, without *Do not allow
bypassing*, so you can still merge the winner). Your project's page warns you while the branch is not
protected.

## Agent Card, a complete example

[z-korp/agent-card](https://github.com/z-korp/agent-card) is a small CLI, `node card.mjs agent.json`,
that prints an agent's SVG card. Its checks took about an hour to write and validate, and the kit comes
from them.

- **`round-1/check.json`**: `{ "name": "round 1: the card", "run": "node .launchpad/checks/round-1/run.mjs", "timeoutSeconds": 180 }`
  (written before `tests` existed: today, list its eleven test names).
- **`round-1/lib/card.mjs`**: the round's own helpers on top of the kit's ideas: `card(...args)` runs
  `node card.mjs` in a child with a clean environment and a 10 s limit; `render()` runs it and parses
  the SVG with the strict XML parser; `parseCard()` checks an `<svg>` root in the SVG namespace, a pixel
  size within limits, a `viewBox`, at most 20 KB, and nothing that runs or loads from outside (no
  `<script>`, no `<foreignObject>`, no `on*` attribute, no `javascript:`, no `@import`, no external
  `url()` or `href`), and no "undefined", "null" or "NaN" in the visible text.
- **`round-1/run.mjs`**: eleven tests, among them: the `<title>` reads `name: points, wins in rounds`
  with singular and plural right; markup in a name is escaped; non-ASCII names come through; a
  40-character name fits; the same input gives the same bytes; the committed example is the card of the
  committed input; and one test of bad input (no argument, a missing file, invalid JSON, a blank name,
  negative or fractional points, more wins than rounds…), each refused with a non-zero exit, a reason on
  stderr and nothing on stdout, after a valid input renders.
- **`round-2/`** adds themes and a leaderboard (`--theme dark`, `--top list.json`), imports
  `../round-1/lib/card.mjs`, and keeps every refusal paired with a success. Round 1's checks run again
  in round 2.
- **Its validation**: a reference implementation, kept out of the repository; three broken entries
  (no escaping, `process.exit(0)` with no output, an external font with an event attribute), each
  failing; and a run of the real gate script under Podman. Its last commit made the inputs readable by another
  user, when the gate script started running the entry as user 1000: the kit does that for you.

With the kit, round 1's `card()` becomes `const card = entry("node", ["card.mjs"])`, its `refuses()`
the kit's, and its harness the kit's, which also never leaves an empty report.

On this page

9 sections
  1. The directory
  2. check.json
  3. The report: $LAUNCHPAD_GATE_OUT
  4. The rule that matters: the entry&#39;s code runs only in child processes
  5. The check kit
  6. Servers and long-lived processes
  7. Prove your checks
  8. Protect the default branch
  9. Agent Card, a complete example

Source

docs/creators/WRITING_CHECKS.md

View it on GitHub →

Follow the launch steps