Run success rate

Run success rate

Tasks the run solved on its last attempt / tasks attempted in this run.

Formula: COUNT(distinct tasks where last attempt passed) / COUNT(distinct tasks attempted in this run)

Per-run metric for the model's "final answer" on each task. Differs from leaderboard pass_at_n: this denominator is the run's own attempted-task count, not the task set size, so partial runs are not penalised for unattempted tasks.

75.8% 47/62 of 62 tasks attempted in this run
Avg score

Avg attempt score

Mean per-attempt score on a 0–100 point scale (partial credit). Drill-down only.

Formula: Mean of attempt scores across all results rows: SUM(score) / COUNT(*) over the results table. Each attempt earns 0–100 points based on compile + test outcomes.

Drill-down companion to pass_at_n. Rewards partial credit but not directly comparable to pass rate; use for within-model analysis.

66.3 / 100
Cost $3.34
Tokens 114k / 111k input / output, all attempts
Duration 2h 54m
Filter:
Per-task results for run 6d0288a4-707c-40c9-8ef4-b714845ad967
TaskDifficultyAttemptScoreTestsCompileDurationTokens
CG-AL-E001easy1100.0 / 1007/7OK2m 32s418 / 433
CG-AL-E002easy1100.0 / 1006/6OK2m 24s433 / 378
CG-AL-E003easy1100.0 / 1005/5OK2m 35s410 / 166
CG-AL-E004easy1100.0 / 1006/6OK2m 39s470 / 367
CG-AL-E005easy1100.0 / 10013/13OK2m 40s491 / 700
CG-AL-E006easy1100.0 / 1007/7OK2m 43s470 / 596
CG-AL-E007easy1100.0 / 1007/7OK2m 47s558 / 242
CG-AL-E008easy1100.0 / 1006/6OK2m 49s470 / 465
CG-AL-E009easy1100.0 / 1005/5OK2m 48s442 / 254
CG-AL-E010easy1100.0 / 1005/5OK2m 50s467 / 188
CG-AL-E031easy1100.0 / 1003/3OK2m 58s678 / 659
CG-AL-E032easy1100.0 / 1001/1OK2m 55s377 / 89
CG-AL-E045easy1100.0 / 1004/4OK2m 25s565 / 434
CG-AL-E050easy1100.0 / 10013/13OK2m 26s635 / 295
CG-AL-E051easy1100.0 / 10015/15OK2m 34s776 / 1.2k
CG-AL-E052easy2100.0 / 10016/16OK2m 26s1.6k / 763
CG-AL-E053easy1100.0 / 1003/3OK2m 26s461 / 176
CG-AL-E054easy2100.0 / 1009/9OK2m 25s1.7k / 714
CG-AL-E055easy1100.0 / 1008/8OK2m 25s566 / 306
CG-AL-H001hard1100.0 / 10025/25OK2m 26s726 / 425
CG-AL-H002hard1100.0 / 1004/4OK2m 28s747 / 785
CG-AL-H003hard2100.0 / 1005/5OK2m 27s2.3k / 1.7k
CG-AL-H004hard1100.0 / 10014/14OK2m 28s779 / 625
CG-AL-H005hard262.5 / 1002/5OK2m 30s3.0k / 2.6k
CG-AL-H006hard1100.0 / 1006/6OK2m 25s715 / 365
CG-AL-H007hard1100.0 / 10010/10OK2m 31s952 / 963
CG-AL-H008hard1100.0 / 10010/10OK2m 27s886 / 586
CG-AL-H009hard1100.0 / 10011/11OK2m 43s1.0k / 1.3k
CG-AL-H010hard1100.0 / 1008/8OK2m 27s901 / 632
CG-AL-H011hard20.0 / 1000/0FAIL11.9s1.5k / 583
CG-AL-H013hard1100.0 / 1009/9OK2m 25s607 / 393
CG-AL-H014hard20.0 / 1000/0FAIL14.5s2.4k / 1.4k
CG-AL-H015hard1100.0 / 1004/4OK2m 26s663 / 449
CG-AL-H016hard20.0 / 1000/0FAIL13.7s2.0k / 1.0k
CG-AL-H017hard20.0 / 1000/0FAIL12.3s1.6k / 708
CG-AL-H018hard1100.0 / 1006/6OK2m 26s732 / 808
CG-AL-H019hard1100.0 / 1005/5OK2m 21s533 / 233
CG-AL-H020hard2100.0 / 10010/10OK2m 26s2.8k / 2.2k
CG-AL-H021hard262.5 / 1006/20OK2m 32s3.3k / 2.1k
CG-AL-H023hard20.0 / 1000/0FAIL2m 18s22k / 22k
CG-AL-H025hard1100.0 / 1007/7OK2m 25s575 / 302
CG-AL-H026hard20.0 / 1000/0FAIL15.9s3.2k / 1.2k
CG-AL-H205hard1100.0 / 1006/6OK2m 27s784 / 664
CG-AL-M001medium262.5 / 1004/10OK2m 31s2.6k / 3.1k
CG-AL-M002medium1100.0 / 10022/22OK2m 23s730 / 877
CG-AL-M003medium262.5 / 1008/9OK2m 31s2.4k / 2.2k
CG-AL-M004medium2100.0 / 10012/12OK2m 38s2.8k / 3.1k
CG-AL-M005medium262.5 / 10012/22OK2m 45s3.6k / 5.7k
CG-AL-M006medium1100.0 / 10018/18OK2m 42s710 / 1.9k
CG-AL-M007medium20.0 / 1000/0FAIL40.6s4.2k / 7.8k
CG-AL-M008medium20.0 / 1000/0FAIL43.8s8.4k / 8.9k
CG-AL-M009medium1100.0 / 10011/11OK2m 43s751 / 2.1k
CG-AL-M010medium262.5 / 10019/21OK2m 42s3.5k / 6.9k
CG-AL-M020medium2100.0 / 10017/17OK2m 31s3.0k / 2.0k
CG-AL-M021medium2100.0 / 10014/14OK2m 39s3.4k / 4.3k
CG-AL-M022medium2100.0 / 1009/9OK2m 31s2.9k / 2.2k
CG-AL-M023medium1100.0 / 10011/11OK2m 29s808 / 741
CG-AL-M024medium1100.0 / 10010/10OK2m 35s762 / 1.5k
CG-AL-M025medium1100.0 / 1007/7OK2m 25s660 / 508
CG-AL-M026medium2100.0 / 1008/8OK2m 26s2.3k / 1.3k
CG-AL-M088medium1100.0 / 1006/6OK2m 28s678 / 643
CG-AL-M112medium262.5 / 1002/4OK2m 23s2.9k / 2.2k