Run success rate

Run success rate

Tasks the run solved on its last attempt / tasks attempted in this run.

Formula: COUNT(distinct tasks where last attempt passed) / COUNT(distinct tasks attempted in this run)

Per-run metric for the model's "final answer" on each task. Differs from leaderboard pass_at_n: this denominator is the run's own attempted-task count, not the task set size, so partial runs are not penalised for unattempted tasks.

71.9% 46/64 of 64 tasks attempted in this run
Avg score

Avg attempt score

Mean per-attempt score on a 0–100 point scale (partial credit). Drill-down only.

Formula: Mean of attempt scores across all results rows: SUM(score) / COUNT(*) over the results table. Each attempt earns 0–100 points based on compile + test outcomes.

Drill-down companion to pass_at_n. Rewards partial credit but not directly comparable to pass rate; use for within-model analysis.

55.7 / 100
Cost $3.87
Tokens 67k / 609k input / output, all attempts
Duration 6h 52m
Filter:
Per-task results for run 8dec1145-326b-4b54-92b3-6093a70b31dd
TaskDifficultyAttemptScoreTestsCompileDurationTokens
CG-AL-E001easy1100.0 / 1007/7OK3m 43s273 / 3.1k
CG-AL-E002easy1100.0 / 1006/6OK3m 41s286 / 3.0k
CG-AL-E003easy2100.0 / 1005/5OK4m 4s844 / 3.3k
CG-AL-E004easy1100.0 / 1006/6OK3m 59s296 / 3.8k
CG-AL-E005easy2100.0 / 10013/13OK4m 5s1.1k / 8.4k
CG-AL-E006easy20.0 / 1000/0FAIL1m 40s977 / 7.4k
CG-AL-E007easy1100.0 / 1007/7OK4m 8s352 / 2.3k
CG-AL-E008easy2100.0 / 1006/6OK3m 43s865 / 4.8k
CG-AL-E009easy1100.0 / 1005/5OK5m 33s282 / 4.5k
CG-AL-E010easy1100.0 / 1005/5OK5m 7s286 / 3.2k
CG-AL-E031easy1100.0 / 1003/3OK5m 19s415 / 3.4k
CG-AL-E032easy1100.0 / 1001/1OK4m 51s230 / 1.5k
CG-AL-E045easy1100.0 / 1004/4OK3m 48s378 / 3.3k
CG-AL-E050easy20.0 / 1000/0FAIL1m 51s2.0k / 8.1k
CG-AL-E051easy1100.0 / 10015/15OK5m 28s494 / 7.5k
CG-AL-E052easy2100.0 / 10016/16OK4m 18s957 / 8.3k
CG-AL-E053easy1100.0 / 1003/3OK3m 53s311 / 3.8k
CG-AL-E054easy20.0 / 1000/0FAIL2m 20s1.1k / 9.0k
CG-AL-E055easy1100.0 / 1008/8OK3m 27s349 / 2.8k
CG-AL-H001easy1100.0 / 10025/25OK4m 9s478 / 4.4k
CG-AL-H002easy2100.0 / 1004/4OK3m 23s1.5k / 6.3k
CG-AL-H003easy1100.0 / 1005/5OK4m 14s495 / 4.7k
CG-AL-H004easy1100.0 / 10014/14OK3m 47s509 / 3.8k
CG-AL-H005easy262.5 / 1002/5OK6m 2s1.9k / 15k
CG-AL-H006easy1100.0 / 1006/6OK3m 24s430 / 2.7k
CG-AL-H007easy1100.0 / 10010/10OK4m 18s562 / 4.7k
CG-AL-H008easy1100.0 / 10010/10OK4m 25s523 / 4.7k
CG-AL-H009easy1100.0 / 10011/11OK5m 23s607 / 7.4k
CG-AL-H010easy1100.0 / 1008/8OK3m 56s517 / 3.9k
CG-AL-H011easy20.0 / 1000/0FAIL8m 29s942 / 26k
CG-AL-H013easy1100.0 / 1009/9OK3m 55s377 / 3.9k
CG-AL-H014easy20.0 / 1000/0FAIL2m 14s1.5k / 12k
CG-AL-H015easy1100.0 / 1004/4OK3m 55s417 / 3.9k
CG-AL-H016easy262.5 / 1003/4OK4m 12s1.1k / 8.0k
CG-AL-H017easy20.0 / 1000/0FAIL2m 20s1.2k / 11k
CG-AL-H018easy2100.0 / 1006/6OK5m 43s1.7k / 91k
CG-AL-H019easy1100.0 / 1005/5OK3m 58s325 / 3.7k
CG-AL-H020easy2100.0 / 10010/10OK4m 41s1.7k / 12k
CG-AL-H021easy2100.0 / 10020/20OK4m 50s2.2k / 10k
CG-AL-H022easy2100.0 / 10021/21OK6m 32s2.5k / 15k
CG-AL-H023easy20.0 / 1000/0FAIL5m 48s3.6k / 28k
CG-AL-H024easy2100.0 / 1009/9OK4m 54s1.8k / 11k
CG-AL-H025easy2100.0 / 1007/7OK3m 45s1.1k / 7.4k
CG-AL-H026easy1100.0 / 1008/8OK4m 27s475 / 4.3k
CG-AL-H205easy1100.0 / 1006/6OK4m 52s455 / 4.6k
CG-AL-M001easy262.5 / 1005/10OK6m 9s1.4k / 14k
CG-AL-M002easy1100.0 / 10022/22OK5m 20s431 / 6.3k
CG-AL-M003easy262.5 / 1003/9OK4m 1s1.6k / 11k
CG-AL-M004easy1100.0 / 10012/12OK5m 21s464 / 7.2k
CG-AL-M005easy20.0 / 1000/0FAIL3m 4s2.1k / 15k
CG-AL-M006easy2100.0 / 10018/18OK5m 18s2.0k / 15k
CG-AL-M007easy20.0 / 1000/0FAIL4m 49s2.1k / 16k
CG-AL-M008easy262.5 / 1003/12OK8m 3s3.1k / 24k
CG-AL-M009easy1100.0 / 10011/11OK6m 49s446 / 9.6k
CG-AL-M010easy262.5 / 10019/21OK5m 53s2.2k / 18k
CG-AL-M020easy20.0 / 1000/0FAIL3m 28s1.9k / 15k
CG-AL-M021easy20.0 / 1000/0FAIL3m 37s1.7k / 12k
CG-AL-M022easy1100.0 / 1009/9OK5m 48s523 / 5.8k
CG-AL-M023easy1100.0 / 10011/11OK5m 47s509 / 4.6k
CG-AL-M024easy2100.0 / 10010/10OK4m 12s2.2k / 11k
CG-AL-M025easy1100.0 / 1007/7OK4m 51s426 / 5.8k
CG-AL-M026easy1100.0 / 1008/8OK4m 6s390 / 4.0k
CG-AL-M088easy1100.0 / 1006/6OK3m 45s416 / 4.0k
CG-AL-M112easy262.5 / 1002/4OK3m 58s2.0k / 9.2k