← Back to Benchmark Results

openrouter/x-ai/grok-code-fast-1

Pass Rate:48.2%
Tasks Passed:27/56
Total Shortcomings:11

All Known Shortcomings

Sorted by occurrence count (most frequent first)

# Concept AL Concept Count Affected Tasks
1 interface-definition-syntax interface-definition 2 CG-AL-E008, CG-AL-H023

Description: The model failed to generate any valid AL code for the interface definition. The compilation errors (AL0107: identifier expected, AL0104: syntax errors) indicate the model produced malformed or empty code instead of a proper AL interface definition. The task asked for an interface with ID 70000, but AL interfaces don't use numeric IDs - they use quoted names. The model appears to have either generated nothing meaningful or used incorrect syntax for defining an AL interface.

Correct Pattern:
interface "Payment Processor"
{
    Access = Public;
    
    /// <summary>
    /// Processes a payment with the specified amount and method.
    /// </summary>
    procedure ProcessPayment(Amount: Decimal; PaymentMethod: Text): Boolean;
    
    /// <summary>
    /// Validates the payment data.
    /// </summary>
    procedure ValidatePayment(PaymentData: Text): Boolean;
    
    /// <summary>
    /// Gets the transaction fee for the specified amount.
    /// </summary>
    procedure GetTransactionFee(Amount: Decimal): Decimal;
}
Incorrect Pattern:
// Generated code not found (or malformed content at line 4)

Error Codes: AL0107, AL0185

2 json-value-extraction-methods json-handling 2 CG-AL-H014, CG-AL-M020

Description: The model generated incorrect AL code for JSON value extraction. The errors show: 1) Using JsonObject as second argument to Get() instead of 'var JsonToken' - the Get method requires a JsonToken variable to receive the value. 2) Passing Text to a method expecting Boolean - likely misusing an overload of a JSON getter method. The model doesn't understand that AL's JsonObject.Get() method requires passing a JsonToken variable by reference, then converting that token to the appropriate type (AsObject(), AsValue().AsText(), etc.).

Correct Pattern:
var Token: JsonToken; DataObj: JsonObject; begin if RootJson.Get('data', Token) then DataObj := Token.AsObject(); // For SafeGetText with default: if not Obj.Get(Key, Token) then exit(DefaultValue); exit(Token.AsValue().AsText());
Incorrect Pattern:
Based on errors: likely used syntax like 'RootJson.Get('data', Level1Json)' where Level1Json is JsonObject instead of JsonToken, and misused GetText overload parameters

Error Codes: AL0133

3 event-subscriber-xrec-parameter event-subscriber-definition 1 CG-AL-E010

Description: The model generated an event subscriber for OnAfterInsertEvent on the Item table but incorrectly included an 'xRec' parameter. The OnAfterInsertEvent does not have an xRec parameter because there is no 'before' record state during an insert operation - the record didn't exist before. The xRec parameter is only available for events like OnAfterModifyEvent or OnBeforeModifyEvent where there's a previous state to compare against.

Correct Pattern:
[EventSubscriber(ObjectType::Table, Database::Item, 'OnAfterInsertEvent', '', false, false)]
local procedure OnAfterInsertItem(var Rec: Record Item; RunTrigger: Boolean)
begin
    Message('New item created: %1', Rec."No.");
end;
Incorrect Pattern:
[EventSubscriber(ObjectType::Table, Database::Item, 'OnAfterInsertEvent', '', false, false)]
local procedure OnAfterInsertItem(var Rec: Record Item; var xRec: Record Item; RunTrigger: Boolean)

Error Codes: AL0282

4 table-definition-syntax table-definition 1 CG-AL-H003

Description: The model failed to generate valid AL code. The compilation errors indicate the model produced malformed table definition syntax - it appears to have started a table object but immediately had syntax errors at line 4 where 'fields' keyword was expected. The model likely generated incomplete or incorrectly structured AL code for the table definition, missing the proper 'fields { }' block structure required in AL table objects.

Correct Pattern:
table 70203 "CG Discount Result"
{
    DataClassification = ToBeClassified;
    
    fields
    {
        field(1; "Line No."; Integer)
        {
            Caption = 'Line No.';
        }
        field(2; "Item No."; Code[20])
        {
            Caption = 'Item No.';
        }
        field(3; "Original Price"; Decimal)
        {
            Caption = 'Original Price';
        }
        field(4; "Discount Percent"; Decimal)
        {
            Caption = 'Discount Percent';
        }
        field(5; "Final Price"; Decimal)
        {
            Caption = 'Final Price';
        }
    }
    
    keys
    {
        key(PK; "Line No.")
        {
            Clustered = true;
        }
    }
}
Incorrect Pattern:
// Generated code not found - but errors indicate malformed table structure at line 4

Error Codes: AL0104

5 query-object-syntax query-definition 1 CG-AL-H011

Description: The model generated syntactically invalid AL code for a query object. The compilation errors indicate syntax problems at line 11 (identifier expected, '=' expected) and line 19 ('}' expected), suggesting the model doesn't understand the proper structure of AL query objects including dataitem definitions, column definitions with aggregation methods, and filter elements. The 'Generated code not found' message in the output suggests the code was malformed enough that it couldn't be properly captured, but the compilation errors confirm code was generated with fundamental syntax issues.

Correct Pattern:
query 70011 "CG Sales Summary"
{
    QueryType = Normal;

    elements
    {
        dataitem(SalesLine; "Sales Line")
        {
            column(Document_No; "Document No.") { }
            column(Sell_to_Customer_No; "Sell-to Customer No.") { }
            column(Line_Amount_Sum; "Line Amount") { Method = Sum; }
            column(Line_Count; "Line No.") { Method = Count; }
            filter(Document_Type; "Document Type") { ColumnFilter = const(Order); }
        }
    }

    trigger OnBeforeOpen()
    begin
        SetAscending(Sell_to_Customer_No, true);
    end;
}
Incorrect Pattern:
// Generated code not found - but compilation errors indicate malformed query syntax at lines 11 and 19

Error Codes: AL0107

6 query-crossjoin-dataitem-hierarchy query-dataitem-structure 1 CG-AL-H017

Description: The model failed to understand AL Query CrossJoin structure. In AL queries, CrossJoin requires proper dataitem nesting - you cannot have multiple dataitems at the same level (AL0342 error). The model placed dataitems at the same level instead of nesting them properly. Additionally, the model incorrectly set DataItemLink on a top-level DataItem (AL0331) and had column source issues (AL0345, AL0353). AL queries with CrossJoin still require hierarchical dataitem structure where child dataitems are nested inside parent dataitems.

Correct Pattern:
query 70017 "CG Dimension Matrix"
{
    QueryType = Normal;
    
    elements
    {
        dataitem(DeptDimValue; "Dimension Value")
        {
            DataItemTableFilter = "Dimension Code" = const('DEPARTMENT');
            column(DepartmentCode; Code) { }
            column(DepartmentName; Name) { }
            
            dataitem(ProjDimValue; "Dimension Value")
            {
                SqlJoinType = CrossJoin;
                DataItemTableFilter = "Dimension Code" = const('PROJECT');
                column(ProjectCode; Code) { }
                column(ProjectName; Name) { }
                
                dataitem(DimSetEntry; "Dimension Set Entry")
                {
                    SqlJoinType = LeftOuterJoin;
                    DataItemLink = "Dimension Code" = DeptDimValue."Dimension Code";
                    column(MatchCount; "Dimension Set ID") { Method = Count; }
                }
            }
        }
    }
}
Incorrect Pattern:
The model created dataitems at the same level attempting CrossJoin, resulting in: 'You cannot combine two DataItems at the same level because unions are not supported'

Error Codes: AL0342

7 code-generation-failure codeunit-structure 1 CG-AL-H022

Description: The model failed to generate any valid AL code for the codeunit. The compilation errors show 'Semicolon expected' at lines 88 and 91, and 'App generation failed', indicating the model produced malformed or incomplete AL code. The 'Generated code not found' message suggests the model either didn't output proper code or the code was so malformed it couldn't be captured. This is a fundamental failure to produce the required codeunit with RecordRef/FieldRef procedures.

Correct Pattern:
codeunit 70224 "CG Dynamic Record Handler"
{
    Access = Public;

    procedure GetTableName(TableId: Integer): Text
    var
        RecRef: RecordRef;
    begin
        if not RecRef.Open(TableId, false) then
            exit('');
        exit(RecRef.Name);
    end;

    // ... other procedures
}
Incorrect Pattern:
// Generated code not found - model failed to produce valid AL codeunit structure

Error Codes: AL0111

8 complex-report-structure report-definition 1 CG-AL-M007

Description: The model failed to generate valid AL code for a complex report. The compilation errors indicate a syntax error at line 70 with a missing closing brace '}', suggesting the model produced incomplete or malformed report structure. The task required creating a 'Sales Performance Analysis' report with multiple data items, complex calculations, triggers, and a request page. The model's generated code was either incomplete or had structural issues that prevented compilation.

Correct Pattern:
report 70001 "Sales Performance Analysis"
{
    ApplicationArea = All;
    UsageCategory = ReportsAndAnalysis;
    Caption = 'Sales Performance Analysis';
    
    dataset
    {
        dataitem(Customer; Customer)
        {
            RequestFilterFields = "No.", "Customer Posting Group";
            
            column(CustomerNo; "No.") { }
            column(CustomerName; Name) { }
            
            dataitem(SalesHeader; "Sales Header")
            {
                DataItemLink = "Sell-to Customer No." = field("No.");
                
                dataitem(SalesLine; "Sales Line")
                {
                    DataItemLink = "Document No." = field("No.");
                    
                    trigger OnAfterGetRecord()
                    begin
                        // Calculation logic
                    end;
                }
            }
            
            trigger OnPreDataItem()
            begin
                // Initialization
            end;
        }
    }
    
    requestpage
    {
        layout
        {
            area(Content)
            {
                group(Options)
                {
                    // Filter fields
                }
            }
        }
    }
}
Incorrect Pattern:
// Generated code not found - but compilation errors indicate incomplete/malformed report structure at line 70

Error Codes: AL0104

9 logmessage-overload-signature telemetry-logging 1 CG-AL-M008

Description: The model generated code that calls LogMessage with 5 arguments, but the AL built-in LogMessage method requires either 8-10 arguments (with individual dimension parameters) or 6 arguments (with a Dictionary). The model doesn't understand the correct signature for the LogMessage telemetry function in AL. Additionally, the model made type conversion errors trying to assign Text values to Boolean variables.

Correct Pattern:
LogMessage should be called with signature: LogMessage(Text, Text, Verbosity, DataClassification, TelemetryScope, Text, Text, [Text], [Text]) or LogMessage(Text, Text, Verbosity, DataClassification, TelemetryScope, Dictionary of [Text, Text]). Boolean assignments should use proper Boolean expressions, not Text values.
Incorrect Pattern:
LogMessage(...) with 5 arguments; also assigning Text to Boolean variables

Error Codes: AL0126

10 activity-log-field-names standard-table-fields 1 CG-AL-M009

Description: The model attempted to use the 'Activity Log' table for logging/audit trail functionality but used incorrect field names. The model referenced 'Activity Log Context' and 'Activity Description' fields which do not exist in the standard 'Activity Log' table in Business Central. The actual Activity Log table has different field names - it uses 'Context' (not 'Activity Log Context') and 'Description' (not 'Activity Description'). The model hallucinated field names that don't exist in the BC standard table schema.

Correct Pattern:
The Activity Log table in BC uses fields like 'Context', 'Description', 'Activity Date', 'Activity Message', 'Table No Filter', 'Record ID', etc. For logging, the model should either use the correct field names from the actual Activity Log table, use the LogActivity procedure from the Activity Log codeunit, or implement a custom logging mechanism without relying on non-existent fields.
Incorrect Pattern:
ActivityLog."Activity Log Context" := ActivityLog."Activity Log Context"::Shipping;
ActivityLog."Activity Description" := ...

Error Codes: AL0132

11 foreach-json-key-iteration json-object-iteration 1 CG-AL-M021

Description: The model attempted to use 'Key' as a variable or keyword in a foreach loop context when iterating over JsonObject keys. In AL, you cannot use 'Key' directly as a loop variable name without proper declaration, and the syntax for iterating over JsonObject keys requires using the Keys() method which returns a List of Text, then iterating with 'foreach KeyName in KeyList do'. The model likely wrote something like 'foreach Key in JsonObj.Keys do' or used 'Key' incorrectly in the iteration context.

Correct Pattern:
var KeyList: List of [Text]; KeyName: Text; ... JsonObj.Keys(KeyList); foreach KeyName in KeyList do begin ... end;
Incorrect Pattern:
Key (at line 56, 75, 108 - used incorrectly in foreach or iteration context)

Error Codes: AL0519