Business Logic
Codeunits, calculations, algorithms, and string manipulation
Report generated: February 25, 2026 at 7:49 AM
Benchmark data: Feb 18, 2026 – Feb 25, 2026
15
Models
11
Tasks
70.3%
Pass Rate
Model Rankings
Model Performance
anthropic/claude-opus-4-6
Runs:3
pass@1:81.8%
pass@3:81.8%
Consistency:100.0%
1st: 232nd: 4Failed: 29/11 passed
Temperature:0.1
Thinking:-
Tokens/run:54,143
Cost/run:$0.97
Known Shortcomings (8)
- reserved-keyword-as-parameter-name 1x
- cross-join-dataitem-link 1x
- incomplete-procedure-body 1x
- flowfield-calcfields-requirement 1x
- parse-failure 1x
+3 more View all 8
openai/gpt-5.3-codex
Runs:3
pass@1:78.8%
pass@3:81.8%
Consistency:90.9%
1st: 202nd: 6Failed: 29/11 passed
Temperature:0.1
Thinking:-
Tokens/run:27,385
Cost/run:$0.24
Known Shortcomings (14)
- query-object-syntax 2x
- empty-or-missing-code-generation 2x
- parse-failure 2x
- dictionary-keys-method-signature 1x
- al-syntax-basics 1x
+9 more View all 14
anthropic/claude-sonnet-4-6
Runs:3
pass@1:75.8%
pass@3:81.8%
Consistency:90.9%
1st: 202nd: 5Failed: 29/11 passed
Temperature:0.1
Thinking:-
Tokens/run:62,401
Cost/run:$0.67
Known Shortcomings (5)
- event-subscriber-event-name 1x
- count-method-column-syntax 1x
- code-truncation-incomplete-output 1x
- jsonvalue-type-checking-api 1x
- flowfield-calcsums-restriction 1x
openai/gpt-5.2-2025-12-11@thinking=high
Runs:3
pass@1:75.8%
pass@3:81.8%
Consistency:90.9%
1st: 192nd: 6Failed: 29/11 passed
Temperature:0.1
Thinking:high
Tokens/run:26,692
Cost/run:$0.25
Known Shortcomings (13)
- interface-definition-syntax 2x
- table-field-caption-property 2x
- query-object-syntax 2x
- query-crossjoin-syntax 2x
- jsonvalue-type-checking-methods 2x
+8 more View all 13
openrouter/x-ai/grok-code-fast-1
Runs:3
pass@1:72.7%
pass@3:72.7%
Consistency:100.0%
1st: 202nd: 4Failed: 38/11 passed
Temperature:0.1
Thinking:-
Tokens/run:39,270
Cost/run:$0.39
Known Shortcomings (22)
- query-object-syntax 2x
- jsonobject-selecttoken-vs-get 2x
- httpclient-getheaders-usage 2x
- multiline-string-literals 1x
- page-extension-cardpageid-override 1x
+17 more View all 22
anthropic/claude-opus-4-5-20251101@thinking=50000
Runs:3
pass@1:72.7%
pass@3:72.7%
Consistency:100.0%
1st: 212nd: 3Failed: 38/11 passed
Temperature:0.1
Thinking:50,000
Tokens/run:36,504
Cost/run:$0.61
Known Shortcomings (10)
- page-extension-with-table-extension 1x
- reserved-keyword-as-parameter-name 1x
- dictionary-iteration-syntax 1x
- empty-or-malformed-code-generation 1x
- temporary-table-parameter-handling 1x
+5 more View all 10
anthropic/claude-sonnet-4-5-20250929
Runs:3
pass@1:72.7%
pass@3:72.7%
Consistency:100.0%
1st: 152nd: 9Failed: 38/11 passed
Temperature:0.1
Thinking:-
Tokens/run:33,078
Cost/run:$0.33
Known Shortcomings (14)
- multiline-string-literals 1x
- query-filter-element-syntax 1x
- jsonobject-get-method-signature 1x
- cross-join-dataitem-link-constraints 1x
- reserved-keyword-as-variable-name 1x
+9 more View all 14
gemini/gemini-3.1-pro-preview
Runs:3
pass@1:66.7%
pass@3:72.7%
Consistency:81.8%
1st: 182nd: 4Failed: 38/11 passed
Temperature:0.1
Thinking:-
Tokens/run:133,962
Cost/run:$0.10
Known Shortcomings (16)
- empty-or-missing-code-generation 2x
- empty-or-missing-code-generation 2x
- empty-or-missing-code-generation 1x
- table-extension-with-page-extension 1x
- al-syntax-structure 1x
+11 more View all 16
openrouter/deepseek/deepseek-v3.2
Runs:3
pass@1:63.6%
pass@3:72.7%
Consistency:81.8%
1st: 102nd: 11Failed: 38/11 passed
Temperature:0.1
Thinking:-
Tokens/run:34,687
Cost/run:$0.35
Known Shortcomings (29)
- interface-definition-syntax 3x
- application-area-in-page-extension-field 2x
- reserved-keyword-as-variable-name 2x
- query-cross-join-syntax 2x
- jsonobject-get-vs-selecttoken 2x
+24 more View all 29
openrouter/z-ai/glm-5
Runs:3
pass@1:60.6%
pass@3:72.7%
Consistency:81.8%
1st: 182nd: 2Failed: 38/11 passed
Temperature:0.1
Thinking:-
Tokens/run:68,155
Cost/run:$0.82
Known Shortcomings (25)
- query-object-syntax 2x
- list-dictionary-of-interface-clear-method 1x
- event-subscriber-event-name 1x
- al-string-literal-escaping 1x
- fluent-api-return-self-codeunit 1x
+20 more View all 25
openrouter/moonshotai/kimi-k2.5
Runs:3
pass@1:60.6%
pass@3:63.6%
Consistency:90.9%
1st: 192nd: 1Failed: 47/11 passed
Temperature:0.1
Thinking:-
Tokens/run:62,198
Cost/run:$0.71
Known Shortcomings (11)
- query-object-syntax 2x
- event-subscriber-parameter-syntax 1x
- page-extension-cardpageid-override 1x
- parse-failure 1x
- multiline-string-literals 1x
+6 more View all 11
openrouter/minimax/minimax-m2.5
Runs:3
pass@1:57.6%
pass@3:63.6%
Consistency:90.9%
1st: 142nd: 5Failed: 47/11 passed
Temperature:0.1
Thinking:-
Tokens/run:41,217
Cost/run:$0.44
Known Shortcomings (32)
- json-object-key-iteration 3x
- interface-definition-syntax 2x
- multiline-string-literals 2x
- text-char-conversion-copystr 1x
- page-object-definition 1x
+27 more View all 32
openrouter/qwen/qwen3-max-thinking
Runs:3
pass@1:48.5%
pass@3:63.6%
Consistency:72.7%
1st: 132nd: 3Failed: 47/11 passed
Temperature:0.1
Thinking:-
Tokens/run:29,455
Cost/run:$0.29
Known Shortcomings (27)
- option-field-optionmembers-required 2x
- variant-type-argument-and-interface-definition 2x
- query-object-syntax 2x
- enum-frominteger-syntax 1x
- list-iteration-pattern 1x
+22 more View all 27
openrouter/qwen/qwen3-coder-next
Runs:3
pass@1:48.5%
pass@3:54.5%
Consistency:81.8%
1st: 142nd: 2Failed: 56/11 passed
Temperature:0.1
Thinking:-
Tokens/run:33,569
Cost/run:$0.33
Known Shortcomings (35)
- codeunit-generation-empty-output 5x
- interface-definition-syntax 4x
- query-object-syntax 2x
- reserved-keyword-as-parameter-name 2x
- initvalue-vs-defaultvalue 1x
+30 more View all 35
gemini/gemini-3-pro-preview
Runs:3
pass@1:39.4%
pass@3:45.5%
Consistency:90.9%
1st: 13Failed: 65/11 passed
Temperature:0.1
Thinking:-
Tokens/run:215,428
Cost/run:$0.25
Known Shortcomings (23)
- codeunit-syntax-structure 3x
- multiline-string-literals 1x
- inherent-permissions-syntax 1x
- query-crossjoin-column-datasource 1x
- complete-codeunit-generation 1x
+18 more View all 23
Task Results Matrix
N/M = passed N of M runs (hover for details)
| Task | Description | Claude Opus 4.6 | GPT-5.3 Codex | Claude Sonnet 4 6 | GPT-5.2 | Grok Code Fast 1 | Claude Opus 4.5 (50K) | Claude Sonnet 4.5 | Gemini 3.1 Pro Preview | Deepseek V3.2 | Glm 5 | Kimi K2.5 | Minimax M2.5 | Qwen3 Max Thinking | Qwen3 Coder Next | Gemini 3 Pro |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CG-AL-E005 | Create a simple AL codeunit called "Text Utilities" with ID 70000. The codeunit should have the following public procedures: - CapitalizeFirstLetter(InputText: Text): Text - capitalizes the first letter of input text - CountWords(InputText: Text): Integer - counts the number of words in the input text - IsValidEmail(EmailAddress: Text): Boolean - basic validation to check if text contains @ and . | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 1/3 | 3/3 | 3/3 | 1/3 | 1/3 | 2/3 | 3/3 |
| CG-AL-E050 | Create an AL codeunit named "CG Text Builder" with ID 70050 that uses multiline string literals. | 3/3 | 2/3 | 1/3 | 1/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| CG-AL-E051 | Create an AL codeunit named "CG Number Sequence" with ID 70051 that increments string values with numeric portions. | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 1/3 | 2/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| CG-AL-E052 | Create an AL codeunit named "CG Type Converter" with ID 70052 for converting simple types to text. | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 2/3 | 1/3 | 0/3 | 3/3 | 3/3 | 2/3 | 0/3 |
| CG-AL-H001 | Create a codeunit called "Tax Calculator" with ID 70100. The codeunit should implement international tax calculation with the following procedure: | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 0/3 |
| CG-AL-H009 | Create a codeunit that implements proper BC currency rounding patterns. | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 1/3 |
| CG-AL-H013 | Create an AL codeunit named "CG Loop Utilities" with ID 70013 that demonstrates the continue keyword: | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| CG-AL-M002 | Create a comprehensive business logic codeunit called "Sales Order Calculator" with ID 70001. The codeunit should implement sales calculations including: - CalculateLineTotal(Quantity: Decimal; UnitPrice: Decimal): Decimal - basic calculation - CalculateLineTotal(Quantity: Decimal; UnitPrice: Decimal; DiscountPercent: Decimal): Decimal - with discount - CalculateOrderTotal(LineTotals: List of [Decimal]): Decimal - sum all line totals from list - ApplyVolumeDiscount(OrderTotal: Decimal): Decimal - apply tiered discounts: | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 1/3 | 3/3 | 3/3 |
| CG-AL-M008 | Create a workflow implementation codeunit called "Purchase Approval Workflow" with ID 70003. | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| CG-AL-M010 | Create a complete multi-object scenario for a "Project Management" system with related objects: | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| CG-AL-M088 | 1. Create an Enum with ID 70188 named "Subscription Plan". | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 0/3 | 3/3 |