Run success rate

Run success rate

Tasks the run solved on its last attempt / tasks attempted in this run.

Formula: COUNT(distinct tasks where last attempt passed) / COUNT(distinct tasks attempted in this run)

Per-run metric for the model's "final answer" on each task. Differs from leaderboard pass_at_n: this denominator is the run's own attempted-task count, not the task set size, so partial runs are not penalised for unattempted tasks.

73.4% 47/64 of 64 tasks attempted in this run
Avg score

Avg attempt score

Mean per-attempt score on a 0–100 point scale (partial credit). Drill-down only.

Formula: Mean of attempt scores across all results rows: SUM(score) / COUNT(*) over the results table. Each attempt earns 0–100 points based on compile + test outcomes.

Drill-down companion to pass_at_n. Rewards partial credit but not directly comparable to pass rate; use for within-model analysis.

59.9 / 100
Cost $3.36
Tokens 63k / 528k input / output, all attempts
Duration 6h 26m
Filter:
Per-task results for run 0defa74e-bfbd-4718-8938-f93a17c269c3
TaskDifficultyAttemptScoreTestsCompileDurationTokens
CG-AL-E001easy1100.0 / 1007/7OK3m 27s273 / 2.4k
CG-AL-E002easy1100.0 / 1006/6OK3m 52s286 / 3.3k
CG-AL-E003easy1100.0 / 1005/5OK2m 44s271 / 1.0k
CG-AL-E004easy1100.0 / 1006/6OK3m 32s296 / 2.7k
CG-AL-E005easy2100.0 / 10013/13OK4m 31s886 / 7.8k
CG-AL-E006easy2100.0 / 1007/7OK4m 25s964 / 9.3k
CG-AL-E007easy1100.0 / 1007/7OK4m 0s352 / 2.4k
CG-AL-E008easy2100.0 / 1006/6OK3m 39s885 / 4.7k
CG-AL-E009easy20.0 / 1000/0FAIL1m 39s1.0k / 7.3k
CG-AL-E010easy2100.0 / 1005/5OK3m 36s712 / 3.4k
CG-AL-E031easy1100.0 / 1003/3OK5m 22s415 / 3.5k
CG-AL-E032easy1100.0 / 1001/1OK4m 45s230 / 1.6k
CG-AL-E045easy1100.0 / 1004/4OK4m 15s378 / 4.2k
CG-AL-E050easy2100.0 / 10013/13OK3m 47s2.0k / 6.3k
CG-AL-E051easy1100.0 / 10015/15OK5m 41s494 / 7.4k
CG-AL-E052easy2100.0 / 10016/16OK4m 33s957 / 8.3k
CG-AL-E053easy1100.0 / 1003/3OK3m 58s311 / 3.4k
CG-AL-E054easy20.0 / 1000/0FAIL2m 34s1.1k / 8.2k
CG-AL-E055easy1100.0 / 1008/8OK4m 1s349 / 3.6k
CG-AL-H001easy1100.0 / 10025/25OK4m 7s478 / 4.1k
CG-AL-H002easy1100.0 / 1004/4OK3m 58s455 / 3.7k
CG-AL-H003easy1100.0 / 1005/5OK4m 31s495 / 4.8k
CG-AL-H004easy1100.0 / 10014/14OK3m 49s509 / 3.3k
CG-AL-H005easy262.5 / 1002/5OK5m 57s2.0k / 15k
CG-AL-H006easy1100.0 / 1006/6OK3m 27s430 / 2.7k
CG-AL-H007easy1100.0 / 10010/10OK4m 27s562 / 4.8k
CG-AL-H008easy1100.0 / 10010/10OK4m 25s523 / 4.7k
CG-AL-H009easy1100.0 / 10011/11OK4m 44s607 / 5.5k
CG-AL-H010easy1100.0 / 1008/8OK3m 43s517 / 3.4k
CG-AL-H011easy20.0 / 1000/0FAIL2m 25s1.0k / 9.8k
CG-AL-H013easy1100.0 / 1009/9OK3m 36s377 / 3.2k
CG-AL-H014easy20.0 / 1000/0FAIL2m 14s1.5k / 10k
CG-AL-H015easy1100.0 / 1004/4OK3m 40s417 / 3.3k
CG-AL-H016easy262.5 / 1003/4OK3m 26s1.0k / 8.6k
CG-AL-H017easy20.0 / 1000/0FAIL2m 31s1.0k / 11k
CG-AL-H018easy2100.0 / 1006/6OK7m 11s1.7k / 19k
CG-AL-H019easy2100.0 / 1005/5OK11m 59s862 / 27k
CG-AL-H020easy20.0 / 1000/0FAIL4m 22s1.8k / 14k
CG-AL-H021easy2100.0 / 10020/20OK6m 21s3.0k / 12k
CG-AL-H022easy2100.0 / 10021/21OK7m 1s2.5k / 17k
CG-AL-H023easy262.5 / 10024/25OK8m 31s4.1k / 29k
CG-AL-H024easy1100.0 / 1009/9OK5m 11s538 / 7.0k
CG-AL-H025easy1100.0 / 1007/7OK4m 16s378 / 4.7k
CG-AL-H026easy1100.0 / 1008/8OK4m 9s475 / 4.5k
CG-AL-H205easy1100.0 / 1006/6OK5m 26s455 / 5.1k
CG-AL-M001easy262.5 / 1005/10OK5m 0s1.3k / 12k
CG-AL-M002easy1100.0 / 10022/22OK4m 34s431 / 5.8k
CG-AL-M003easy262.5 / 1003/9OK4m 59s1.4k / 12k
CG-AL-M004easy1100.0 / 10012/12OK5m 17s464 / 7.3k
CG-AL-M005easy262.5 / 10012/22OK5m 6s2.2k / 17k
CG-AL-M006easy2100.0 / 10018/18OK4m 23s1.9k / 15k
CG-AL-M007easy20.0 / 1000/0FAIL2m 48s2.0k / 17k
CG-AL-M008easy262.5 / 1001/12OK6m 42s2.1k / 21k
CG-AL-M009easy1100.0 / 10011/11OK5m 49s446 / 8.6k
CG-AL-M010easy262.5 / 10019/21OK6m 11s2.3k / 18k
CG-AL-M020easy2100.0 / 10017/17OK4m 26s2.1k / 14k
CG-AL-M021easy20.0 / 1000/0FAIL2m 40s1.7k / 13k
CG-AL-M022easy1100.0 / 1009/9OK4m 41s523 / 6.0k
CG-AL-M023easy1100.0 / 10011/11OK4m 6s509 / 4.5k
CG-AL-M024easy1100.0 / 10010/10OK4m 58s518 / 7.1k
CG-AL-M025easy1100.0 / 1007/7OK4m 53s426 / 4.8k
CG-AL-M026easy1100.0 / 1008/8OK4m 50s390 / 4.9k
CG-AL-M088easy1100.0 / 1006/6OK3m 44s416 / 4.0k
CG-AL-M112easy262.5 / 1002/4OK3m 14s2.0k / 7.4k