{"protocol":"launchpad-benchmark/1","scope":"all settled rounds the team ranked","metrics":{"top1Agreement":"share of the judge's rankings whose first place is the team's first place","spearman":"Spearman's rank correlation with the team's order over the entries both ranked (1 identical, -1 reversed), averaged","kendall":"Kendall's tau, likewise","costPerEntryUsd":"judge cost over entries ranked","secondsPerEntry":"judge wall time over entries ranked (passes that recorded it; a consensus takes its slowest pass, as the passes run at once)","team":"the rankers' blind rankings combined by median rank; it decides nothing"},"rounds":[],"judges":[]}