User Interface
Pages, page extensions, reports, XMLports, and API pages
Report generated: February 25, 2026 at 7:49 AM
Benchmark data: Feb 18, 2026 – Feb 25, 2026
15
Models
7
Tasks
68.6%
Pass Rate
Model Rankings
Model Performance
openrouter/moonshotai/kimi-k2.5
Runs:3
pass@1:76.2%
pass@3:85.7%
Consistency:85.7%
1st: 142nd: 2Failed: 16/7 passed
Temperature:0.1
Thinking:-
Tokens/run:38,363
Cost/run:$0.49
Known Shortcomings (11)
- query-object-syntax 2x
- event-subscriber-parameter-syntax 1x
- page-extension-cardpageid-override 1x
- parse-failure 1x
- multiline-string-literals 1x
+6 more View all 11
openai/gpt-5.3-codex
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 112nd: 4Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:15,422
Cost/run:$0.15
Known Shortcomings (14)
- query-object-syntax 2x
- empty-or-missing-code-generation 2x
- parse-failure 2x
- dictionary-keys-method-signature 1x
- al-syntax-basics 1x
+9 more View all 14
openai/gpt-5.2-2025-12-11@thinking=high
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 15Failed: 25/7 passed
Temperature:0.1
Thinking:high
Tokens/run:16,246
Cost/run:$0.17
Known Shortcomings (13)
- interface-definition-syntax 2x
- table-field-caption-property 2x
- query-object-syntax 2x
- query-crossjoin-syntax 2x
- jsonvalue-type-checking-methods 2x
+8 more View all 13
anthropic/claude-opus-4-6
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 15Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:45,101
Cost/run:$0.87
Known Shortcomings (8)
- reserved-keyword-as-parameter-name 1x
- cross-join-dataitem-link 1x
- incomplete-procedure-body 1x
- flowfield-calcfields-requirement 1x
- parse-failure 1x
+3 more View all 8
anthropic/claude-opus-4-5-20251101@thinking=50000
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 15Failed: 25/7 passed
Temperature:0.1
Thinking:50,000
Tokens/run:27,124
Cost/run:$0.49
Known Shortcomings (10)
- page-extension-with-table-extension 1x
- reserved-keyword-as-parameter-name 1x
- dictionary-iteration-syntax 1x
- empty-or-malformed-code-generation 1x
- temporary-table-parameter-handling 1x
+5 more View all 10
anthropic/claude-sonnet-4-5-20250929
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 142nd: 1Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:17,179
Cost/run:$0.19
Known Shortcomings (14)
- multiline-string-literals 1x
- query-filter-element-syntax 1x
- jsonobject-get-method-signature 1x
- cross-join-dataitem-link-constraints 1x
- reserved-keyword-as-variable-name 1x
+9 more View all 14
openrouter/qwen/qwen3-max-thinking
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 112nd: 4Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:13,284
Cost/run:$0.13
Known Shortcomings (27)
- option-field-optionmembers-required 2x
- variant-type-argument-and-interface-definition 2x
- query-object-syntax 2x
- enum-frominteger-syntax 1x
- list-iteration-pattern 1x
+22 more View all 27
anthropic/claude-sonnet-4-6
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 15Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:45,335
Cost/run:$0.53
Known Shortcomings (5)
- event-subscriber-event-name 1x
- count-method-column-syntax 1x
- code-truncation-incomplete-output 1x
- jsonvalue-type-checking-api 1x
- flowfield-calcsums-restriction 1x
openrouter/x-ai/grok-code-fast-1
Runs:3
pass@1:66.7%
pass@3:71.4%
Consistency:85.7%
1st: 14Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:22,130
Cost/run:$0.25
Known Shortcomings (22)
- query-object-syntax 2x
- jsonobject-selecttoken-vs-get 2x
- httpclient-getheaders-usage 2x
- multiline-string-literals 1x
- page-extension-cardpageid-override 1x
+17 more View all 22
openrouter/z-ai/glm-5
Runs:3
pass@1:61.9%
pass@3:71.4%
Consistency:85.7%
1st: 13Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:41,321
Cost/run:$0.55
Known Shortcomings (25)
- query-object-syntax 2x
- list-dictionary-of-interface-clear-method 1x
- event-subscriber-event-name 1x
- al-string-literal-escaping 1x
- fluent-api-return-self-codeunit 1x
+20 more View all 25
gemini/gemini-3-pro-preview
Runs:3
pass@1:61.9%
pass@3:71.4%
Consistency:85.7%
1st: 13Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:83,112
Cost/run:$0.11
Known Shortcomings (23)
- codeunit-syntax-structure 3x
- multiline-string-literals 1x
- inherent-permissions-syntax 1x
- query-crossjoin-column-datasource 1x
- complete-codeunit-generation 1x
+18 more View all 23
gemini/gemini-3.1-pro-preview
Runs:3
pass@1:57.1%
pass@3:71.4%
Consistency:71.4%
1st: 112nd: 1Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:70,492
Cost/run:$0.05
Known Shortcomings (16)
- empty-or-missing-code-generation 2x
- empty-or-missing-code-generation 2x
- empty-or-missing-code-generation 1x
- table-extension-with-page-extension 1x
- al-syntax-structure 1x
+11 more View all 16
openrouter/minimax/minimax-m2.5
Runs:3
pass@1:52.4%
pass@3:71.4%
Consistency:57.1%
1st: 52nd: 6Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:27,808
Cost/run:$0.31
Known Shortcomings (32)
- json-object-key-iteration 3x
- interface-definition-syntax 2x
- multiline-string-literals 2x
- text-char-conversion-copystr 1x
- page-object-definition 1x
+27 more View all 32
openrouter/deepseek/deepseek-v3.2
Runs:3
pass@1:42.9%
pass@3:42.9%
Consistency:100.0%
1st: 9Failed: 43/7 passed
Temperature:0.1
Thinking:-
Tokens/run:20,468
Cost/run:$0.23
Known Shortcomings (29)
- interface-definition-syntax 3x
- application-area-in-page-extension-field 2x
- reserved-keyword-as-variable-name 2x
- query-cross-join-syntax 2x
- jsonobject-get-vs-selecttoken 2x
+24 more View all 29
openrouter/qwen/qwen3-coder-next
Runs:3
pass@1:38.1%
pass@3:42.9%
Consistency:85.7%
1st: 52nd: 3Failed: 43/7 passed
Temperature:0.1
Thinking:-
Tokens/run:17,530
Cost/run:$0.17
Known Shortcomings (35)
- codeunit-generation-empty-output 5x
- interface-definition-syntax 4x
- query-object-syntax 2x
- reserved-keyword-as-parameter-name 2x
- initvalue-vs-defaultvalue 1x
+30 more View all 35
Task Results Matrix
N/M = passed N of M runs (hover for details)
| Task | Description | Kimi K2.5 | GPT-5.3 Codex | GPT-5.2 | Claude Opus 4.6 | Claude Opus 4.5 (50K) | Claude Sonnet 4.5 | Qwen3 Max Thinking | Claude Sonnet 4 6 | Grok Code Fast 1 | Glm 5 | Gemini 3 Pro | Gemini 3.1 Pro Preview | Minimax M2.5 | Deepseek V3.2 | Qwen3 Coder Next |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CG-AL-E002 | Create a simple AL card page called "Product Category Card" with ID 70001. The page should be based on the existing "Product Category" table (ID 69001) and display the following fields: - Code - Description - Active - Created Date | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| CG-AL-E006 | Create a page extension called "Customer Card Extension" with ID 70000 that extends the Customer Card page (page 21). Add the following controls to the General group: - Preferred Contact Method (Option field with values: Email, Phone, Mail, SMS) - Customer Notes (Text field for internal notes) - VIP Customer (Boolean field to mark important customers) | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 1/3 | 2/3 | 0/3 | 0/3 |
| CG-AL-E007 | Create a simple AL report called "Customer List Report" with ID 70000. The report should use the Customer table as its data source and display: - Customer No. (column name: No_Customer, source: "No.") - Customer Name (column name: Name_Customer, source: Name) - City (column name: City_Customer, source: City) - Phone No. (column name: PhoneNo_Customer, source: "Phone No.") | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 |
| CG-AL-E053 | Create a page extension named "CG Item List Extension" with ID 70053 that extends the "Item List" page (Page 31). | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 3/3 | 3/3 | 1/3 | 0/3 | 3/3 |
| CG-AL-M001 | Create a comprehensive API page called "Product API" with ID 70100 that provides full CRUD operations for a Product table. The API should: - Support GET, POST, PATCH, DELETE operations - Include proper error handling and validation - Have fields: id (SystemId), productCode, description, unitPrice, stockQuantity, categoryId - Include proper OData annotations and API versioning (v1.0) - Add business logic validation (price must be positive, stock must be non-negative) - Include proper captions, publisher info (mycompany), and API group (products) | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| CG-AL-M004 | Create an interactive page called "Sales Order Workspace" with ID 70101 that provides comprehensive order management. The page should be PageType Card, based on Sales Header table (filtered to Document Type::Order), and include: | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 1/3 | 1/3 | 2/3 | 2/3 | 3/3 | 0/3 |
| CG-AL-M007 | Create a complex report called "Sales Performance Analysis" with ID 70001 that performs advanced data processing. | 1/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 |