← Back to Benchmark Results

Data Modeling

Tables, fields, enums, keys, FlowFields, CalcFormulas, and table extensions

Report generated: February 25, 2026 at 7:49 AM

Benchmark data: Feb 18, 2026 – Feb 25, 2026

15
Models
10
Tasks
75.3%
Pass Rate

Model Rankings

pass@1 pass@3 (additional)
Claude Opus 4.5 (50K, t0.1)
80%
80.0%
Claude Opus 4.6 (t0.1)
80%
80.0%
Gemini 3.1 Pro Preview (t0.1)
77%
80.0%
Deepseek V3.2 (t0.1)
77%
80.0%
Claude Sonnet 4.5 (t0.1)
77%
80.0%
GPT-5.3 Codex (t0.1)
73%
80.0%
Claude Sonnet 4 6 (t0.1)
73%
80.0%
Grok Code Fast 1 (t0.1)
73%
80.0%
Kimi K2.5 (t0.1)
73%
80.0%
Glm 5 (t0.1)
70%
70.0%
Minimax M2.5 (t0.1)
70%
70.0%
GPT-5.2 (high, t0.1)
70%
70.0%
Gemini 3 Pro (t0.1)
70%
70.0%
Qwen3 Max Thinking (t0.1)
70%
70.0%
Qwen3 Coder Next (t0.1)
60%
60.0%

Model Performance

anthropic/claude-opus-4-5-20251101@thinking=50000

Runs:3
pass@1:80.0%
pass@3:80.0%
Consistency:100.0%
1st: 212nd: 3Failed: 28/10 passed
Temperature:0.1
Thinking:50,000
Tokens/run:18,293
Cost/run:$0.29

Known Shortcomings (10)

  • page-extension-with-table-extension 1x
  • reserved-keyword-as-parameter-name 1x
  • dictionary-iteration-syntax 1x
  • empty-or-malformed-code-generation 1x
  • temporary-table-parameter-handling 1x
+5 more View all 10
View details ›

anthropic/claude-opus-4-6

Runs:3
pass@1:80.0%
pass@3:80.0%
Consistency:100.0%
1st: 212nd: 3Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:15,617
Cost/run:$0.24

Known Shortcomings (8)

  • reserved-keyword-as-parameter-name 1x
  • cross-join-dataitem-link 1x
  • incomplete-procedure-body 1x
  • flowfield-calcfields-requirement 1x
  • parse-failure 1x
+3 more View all 8
View details ›

gemini/gemini-3.1-pro-preview

Runs:3
pass@1:76.7%
pass@3:80.0%
Consistency:90.0%
1st: 212nd: 2Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:58,328
Cost/run:$0.05

Known Shortcomings (16)

  • empty-or-missing-code-generation 2x
  • empty-or-missing-code-generation 2x
  • empty-or-missing-code-generation 1x
  • table-extension-with-page-extension 1x
  • al-syntax-structure 1x
+11 more View all 16
View details ›

openrouter/deepseek/deepseek-v3.2

Runs:3
pass@1:76.7%
pass@3:80.0%
Consistency:90.0%
1st: 122nd: 11Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:22,913
Cost/run:$0.20

Known Shortcomings (29)

  • interface-definition-syntax 3x
  • application-area-in-page-extension-field 2x
  • reserved-keyword-as-variable-name 2x
  • query-cross-join-syntax 2x
  • jsonobject-get-vs-selecttoken 2x
+24 more View all 29
View details ›

anthropic/claude-sonnet-4-5-20250929

Runs:3
pass@1:76.7%
pass@3:80.0%
Consistency:90.0%
1st: 212nd: 2Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:17,169
Cost/run:$0.16

Known Shortcomings (14)

  • multiline-string-literals 1x
  • query-filter-element-syntax 1x
  • jsonobject-get-method-signature 1x
  • cross-join-dataitem-link-constraints 1x
  • reserved-keyword-as-variable-name 1x
+9 more View all 14
View details ›

openai/gpt-5.3-codex

Runs:3
pass@1:73.3%
pass@3:80.0%
Consistency:90.0%
1st: 212nd: 1Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:20,321
Cost/run:$0.18

Known Shortcomings (14)

  • query-object-syntax 2x
  • empty-or-missing-code-generation 2x
  • parse-failure 2x
  • dictionary-keys-method-signature 1x
  • al-syntax-basics 1x
+9 more View all 14
View details ›

anthropic/claude-sonnet-4-6

Runs:3
pass@1:73.3%
pass@3:80.0%
Consistency:90.0%
1st: 22Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:17,181
Cost/run:$0.16

Known Shortcomings (5)

  • event-subscriber-event-name 1x
  • count-method-column-syntax 1x
  • code-truncation-incomplete-output 1x
  • jsonvalue-type-checking-api 1x
  • flowfield-calcsums-restriction 1x
View all 5
View details ›

openrouter/x-ai/grok-code-fast-1

Runs:3
pass@1:73.3%
pass@3:80.0%
Consistency:90.0%
1st: 202nd: 2Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:33,321
Cost/run:$0.37

Known Shortcomings (22)

  • query-object-syntax 2x
  • jsonobject-selecttoken-vs-get 2x
  • httpclient-getheaders-usage 2x
  • multiline-string-literals 1x
  • page-extension-cardpageid-override 1x
+17 more View all 22
View details ›

openrouter/moonshotai/kimi-k2.5

Runs:3
pass@1:73.3%
pass@3:80.0%
Consistency:90.0%
1st: 192nd: 3Failed: 28/10 passed
Temperature:0.1
Thinking:-
Tokens/run:44,763
Cost/run:$0.52

Known Shortcomings (11)

  • query-object-syntax 2x
  • event-subscriber-parameter-syntax 1x
  • page-extension-cardpageid-override 1x
  • parse-failure 1x
  • multiline-string-literals 1x
+6 more View all 11
View details ›

openrouter/z-ai/glm-5

Runs:3
pass@1:70.0%
pass@3:70.0%
Consistency:100.0%
1st: 202nd: 1Failed: 37/10 passed
Temperature:0.1
Thinking:-
Tokens/run:65,571
Cost/run:$0.86

Known Shortcomings (25)

  • query-object-syntax 2x
  • list-dictionary-of-interface-clear-method 1x
  • event-subscriber-event-name 1x
  • al-string-literal-escaping 1x
  • fluent-api-return-self-codeunit 1x
+20 more View all 25
View details ›

openrouter/minimax/minimax-m2.5

Runs:3
pass@1:70.0%
pass@3:70.0%
Consistency:100.0%
1st: 162nd: 5Failed: 37/10 passed
Temperature:0.1
Thinking:-
Tokens/run:91,170
Cost/run:$1.24

Known Shortcomings (32)

  • json-object-key-iteration 3x
  • interface-definition-syntax 2x
  • multiline-string-literals 2x
  • text-char-conversion-copystr 1x
  • page-object-definition 1x
+27 more View all 32
View details ›

openai/gpt-5.2-2025-12-11@thinking=high

Runs:3
pass@1:70.0%
pass@3:70.0%
Consistency:100.0%
1st: 21Failed: 37/10 passed
Temperature:0.1
Thinking:high
Tokens/run:14,862
Cost/run:$0.12

Known Shortcomings (13)

  • interface-definition-syntax 2x
  • table-field-caption-property 2x
  • query-object-syntax 2x
  • query-crossjoin-syntax 2x
  • jsonvalue-type-checking-methods 2x
+8 more View all 13
View details ›

gemini/gemini-3-pro-preview

Runs:3
pass@1:70.0%
pass@3:70.0%
Consistency:100.0%
1st: 21Failed: 37/10 passed
Temperature:0.1
Thinking:-
Tokens/run:109,359
Cost/run:$0.14

Known Shortcomings (23)

  • codeunit-syntax-structure 3x
  • multiline-string-literals 1x
  • inherent-permissions-syntax 1x
  • query-crossjoin-column-datasource 1x
  • complete-codeunit-generation 1x
+18 more View all 23
View details ›

openrouter/qwen/qwen3-max-thinking

Runs:3
pass@1:70.0%
pass@3:70.0%
Consistency:100.0%
1st: 122nd: 9Failed: 37/10 passed
Temperature:0.1
Thinking:-
Tokens/run:18,418
Cost/run:$0.16

Known Shortcomings (27)

  • option-field-optionmembers-required 2x
  • variant-type-argument-and-interface-definition 2x
  • query-object-syntax 2x
  • enum-frominteger-syntax 1x
  • list-iteration-pattern 1x
+22 more View all 27
View details ›

openrouter/qwen/qwen3-coder-next

Runs:3
pass@1:60.0%
pass@3:60.0%
Consistency:100.0%
1st: 112nd: 7Failed: 46/10 passed
Temperature:0.1
Thinking:-
Tokens/run:20,587
Cost/run:$0.17

Known Shortcomings (35)

  • codeunit-generation-empty-output 5x
  • interface-definition-syntax 4x
  • query-object-syntax 2x
  • reserved-keyword-as-parameter-name 2x
  • initvalue-vs-defaultvalue 1x
+30 more View all 35
View details ›

Task Results Matrix

N/M = passed N of M runs (hover for details)

TaskDescriptionClaude Opus 4.5 (50K)Claude Opus 4.6Gemini 3.1 Pro PreviewDeepseek V3.2Claude Sonnet 4.5GPT-5.3 CodexClaude Sonnet 4 6Grok Code Fast 1Kimi K2.5Glm 5Minimax M2.5GPT-5.2Gemini 3 ProQwen3 Max ThinkingQwen3 Coder Next
CG-AL-E001Create a simple AL table called "Product Category" with ID 70000. The table should have the following fields: - Code (Code[20], primary key) - Description (Text[100]) - Active (Boolean, default true) - Created Date (Date)3/33/33/33/33/33/33/33/33/33/33/33/33/33/30/3
CG-AL-E003Create a simple AL enum called "Priority Level" with ID 70000. The enum should have the following values: - Low (value 0) - Medium (value 1) - High (value 2) - Critical (value 3)3/33/33/33/33/33/33/33/33/33/33/33/33/33/33/3
CG-AL-E004Create a table extension called "Item Extension" with ID 70000 that extends the standard Item table. Add the following new fields: - Warranty Period (Integer, representing months) - Supplier Rating (Option with values: Not Rated, Bronze, Silver, Gold, Platinum) - Last Maintenance Date (Date) - Special Instructions (Text[250])3/33/33/33/33/33/33/33/33/33/33/33/33/33/33/3
CG-AL-E031Create a table called "CG Subscription Plan" with ID 70031.3/33/33/33/33/33/33/33/33/33/33/33/33/33/33/3
CG-AL-E045Create a table called "Vehicle Log" with ID 70045.3/33/33/33/33/33/33/33/33/33/33/33/33/33/33/3
CG-AL-H002Create two tables to demonstrate FlowField with CalcFormula:3/33/33/33/33/33/33/33/33/33/33/33/33/33/33/3
CG-AL-H004Create an enum and codeunit that demonstrate correct enum ordinal handling.3/33/33/33/33/33/33/33/33/33/33/33/33/33/33/3
CG-AL-M003Create a complex table called "Sales Contract" with ID 70002 that includes comprehensive validation. Fields should include: - Contract No. (Code[20], primary key, auto-generated) - Customer No. (Code[20], with TableRelation to Customer) - Start Date and End Date (Date fields with validation) - Contract Value (Decimal with minimum value validation) - Status (Option: Draft, Active, Suspended, Terminated, Closed) - Payment Terms (Code[10] with TableRelation)0/30/30/30/30/30/30/30/30/30/30/30/30/30/30/3
CG-AL-M006Create an advanced table extension called "Advanced Customer Extension" with ID 70001 that extends the Customer table.3/33/32/32/32/31/31/31/31/30/30/30/30/30/30/3
CG-AL-M112Create two tables:0/30/30/30/30/30/30/30/30/30/30/30/30/30/30/3