← Back to Benchmark Results

anthropic/claude-haiku-4-5-20251001

Pass Rate:37.5%
Tasks Passed:21/56
Total Shortcomings:18

All Known Shortcomings

Sorted by occurrence count (most frequent first)

# Concept AL Concept Count Affected Tasks
1 event-subscriber-codeunit-subtype event-subscriber-binding 1 CG-AL-E010

Description: The model incorrectly added 'Subtype = Test;' to the event subscriber codeunit. The task asked for a simple event subscriber codeunit, not a test codeunit. When a codeunit has Subtype = Test, event subscribers within it must use manual binding (EventSubscriberInstance = Manual). The model confused the purpose of the codeunit - it should be a regular codeunit that subscribes to events, not a test codeunit. Additionally, the model included test procedures in what should be a simple event subscriber codeunit.

Correct Pattern:
codeunit 70001 "Item Event Subscriber"
{
    Access = Internal;

    [EventSubscriber(ObjectType::Table, Database::Item, 'OnAfterInsertEvent', '', false, false)]
    local procedure OnAfterInsertItem(var Rec: Record Item; RunTrigger: Boolean)
    begin
        Message('New item created: %1', Rec."No.");
    end;
}
Incorrect Pattern:
codeunit 70001 "Item Event Subscriber"
{
    Access = Internal;
    Subtype = Test;

    [EventSubscriber(ObjectType::Table, Database::Item, 'OnAfterInsertEvent', '', false, false)]
    local procedure OnAfterInsertItem(var Rec: Record Item; RunTrigger: Boolean)

Error Codes: AL0501

2 evaluate-function-syntax built-in-functions 1 CG-AL-E051

Description: The model incorrectly used the Evaluate function. In AL, Evaluate() is a function that parses a string into a variable and requires a variable as the first parameter (passed by reference), not a direct assignment. The model wrote 'NumericPart := Evaluate(CopyStr(Value, StartPos))' which is syntactically incorrect. Evaluate returns a Boolean indicating success/failure, and the parsed value is stored in the first parameter passed by reference.

Correct Pattern:
Evaluate(NumericPart, CopyStr(Value, StartPos));
Incorrect Pattern:
NumericPart := Evaluate(CopyStr(Value, StartPos))

Error Codes: AL0135

3 test-attribute-codeunit-subtype codeunit-subtype-test 1 CG-AL-E052

Description: The model incorrectly added [Test] procedures with test attributes to a regular codeunit (not a Test subtype codeunit). In AL, the [Test] attribute can only be used within a codeunit that has 'Subtype = Test'. The task asked for a utility codeunit with type conversion procedures, but the model added test procedures with [Test] attributes directly in the main codeunit, which is invalid.

Correct Pattern:
codeunit 70052 "CG Type Converter"
{
    Access = Public;

    procedure IntegerToText(Value: Integer): Text
    begin
        exit(Format(Value));
    end;

    procedure DecimalToText(Value: Decimal): Text
    begin
        exit(Format(Value));
    end;

    procedure BooleanToText(Value: Boolean): Text
    begin
        exit(Format(Value));
    end;

    procedure DateToText(Value: Date): Text
    begin
        exit(Format(Value));
    end;

    procedure FormatOrderSummary(OrderNo: Integer; Amount: Decimal; IsShipped: Boolean; OrderDate: Date): Text
    begin
        exit(StrSubstNo('Order: %1, Amount: %2, Shipped: %3, Date: %4', OrderNo, Amount, IsShipped, OrderDate));
    end;
}
Incorrect Pattern:
codeunit 70052 "CG Type Converter"
{
    Access = Public;
    ...
    [Test]
    procedure TestDateToText_ValidDate()
    begin
        // Test with a valid date
        Assert.AreEqual('01/15/2024', DateToText(DMY2Date(15, 1, 2024)), 'DateToText should convert valid date correctly');
    end;

    [Test]
    procedure TestDateToText_FirstDayOfYear()
    begin
        // Test with first day of year
        Assert.AreEqual('01/01/2024', DateToText(DMY2Date(1, 1, 2024)), 'DateToText should convert first day of year correctly');
    end;
}

Error Codes: AL0243

4 pageextension-property-syntax page-extension-properties 1 CG-AL-E053

Description: The model incorrectly wrapped the CardPageId property in a 'properties' block. In AL page extensions, properties like CardPageId should be declared directly at the top level of the pageextension body, not inside a 'properties { }' wrapper. The 'properties' keyword is not valid syntax in this context. The model also attempted to use a feature (overriding CardPageId on page extensions) that may not be available or has different syntax requirements in BC 2025 Wave 1.

Correct Pattern:
pageextension 70053 "CG Item List Extension" extends "Item List"
{
    CardPageId = "Item Card";

    actions
    {
        addlast(processing)
        {
            action("Show Item Details")
            {
                ApplicationArea = All;
                Image = Card;

                trigger OnAction()
                begin
                    Page.Run(Page::"Item Card", Rec);
                end;
            }
        }
    }
}
Incorrect Pattern:
pageextension 70053 "CG Item List Extension" extends "Item List"
{
    properties
    {
        CardPageId = "Item Card";
    }
    ...

Error Codes: AL0124

5 codeunit-enum-structure al-object-definition 1 CG-AL-H007

Description: The model failed to generate proper AL object definitions. Instead of creating a complete codeunit and enum with proper structure, the model only output a single line 'ErrorInfo := ErrorInfo.Create();' which is not a valid AL file. AL files must start with an application object keyword (table, codeunit, enum, etc.) and contain the full object definition. The task required creating both a codeunit 'CG Validation Engine' (ID 70210) and an enum 'CG Validation Error' (ID 70211) with specific procedures and values.

Correct Pattern:
enum 70211 "CG Validation Error"
{
    Extensible = true;
    value(0; None) { }
    value(1; EmptyField) { }
    value(2; InvalidFormat) { }
    value(3; OutOfRange) { }
    value(4; DuplicateValue) { }
}

codeunit 70210 "CG Validation Engine"
{
    Access = Public;

    procedure CreateValidationError(ErrorCode: Enum "CG Validation Error"; FieldName: Text; ErrorMessage: Text): ErrorInfo
    var
        ErrInfo: ErrorInfo;
    begin
        ErrInfo := ErrorInfo.Create();
        ErrInfo.Message := ErrorMessage;
        ErrInfo.CustomDimensions.Add('ErrorCode', Format(ErrorCode.AsInteger()));
        ErrInfo.CustomDimensions.Add('FieldName', FieldName);
        ErrInfo.Collectible := IsCollectingErrors();
        exit(ErrInfo);
    end;

    procedure GetErrorCode(ErrInfo: ErrorInfo): Enum "CG Validation Error"
    var
        ErrorCodeInt: Integer;
    begin
        Evaluate(ErrorCodeInt, ErrInfo.CustomDimensions.Get('ErrorCode'));
        exit(Enum::"CG Validation Error".FromInteger(ErrorCodeInt));
    end;

    procedure ValidateNotEmpty(Value: Text; FieldName: Text)
    begin
        if Value = '' then
            Error(CreateValidationError("CG Validation Error"::EmptyField, FieldName, 'Field cannot be empty'));
    end;

    procedure ValidateInRange(Value: Decimal; MinValue: Decimal; MaxValue: Decimal; FieldName: Text)
    begin
        if (Value < MinValue) or (Value > MaxValue) then
            Error(CreateValidationError("CG Validation Error"::OutOfRange, FieldName, StrSubstNo('Value must be between %1 and %2', MinValue, MaxValue)));
    end;
}
Incorrect Pattern:
ErrorInfo := ErrorInfo.Create();

Error Codes: AL0198

6 query-groupby-syntax query-definition 1 CG-AL-H011

Description: The model incorrectly used 'GroupBy;' as a standalone property inside column definitions. In AL queries, GroupBy is not a standalone property but should be used as 'Method = Group' or the column should be listed in a separate 'keys' section. Additionally, the model used incorrect syntax for 'orderby' - it should be a separate element outside the dataitem with proper syntax like 'orderby(Sell_to_Customer_No)' without the 'Ascending' parameter inside parentheses.

Correct Pattern:
column(Document_No; "Document No.")
{
}
column(Sell_to_Customer_No; "Sell-to Customer No.")
{
}
column(Line_Amount_Sum; "Line Amount")
{
    Method = Sum;
}
column(Line_Count; "Document No.")
{
    Method = Count;
}
filter(DocumentType; "Document Type")
{
    ColumnFilter = const(Order);
}

keys
{
    key(Key1; Document_No, Sell_to_Customer_No)
    {
    }
}
Incorrect Pattern:
column(Document_No; "Document No.")
{
    GroupBy;
}
column(Sell_to_Customer_No; "Sell-to Customer No.")
{
    GroupBy;
}
orderby(Sell_to_Customer_No; Ascending)

Error Codes: AL0124

7 jsonobject-method-signatures json-handling 1 CG-AL-H014

Description: The model incorrectly assumed that JsonObject has direct typed getter methods like GetText(), GetInteger(), GetBoolean(), GetArray(), and GetDecimal() that return values directly. In AL, JsonObject uses Get() method which requires a var JsonToken parameter (out parameter pattern), and JsonArray.Get() also requires a var JsonToken parameter. The model also incorrectly used GetText(Key, DefaultValue) overload which doesn't exist - the actual pattern requires checking if the key exists first or using TryGetValue patterns.

Correct Pattern:
var
    JToken: JsonToken;
    JValue: JsonValue;
begin
    if CustomerJson.Get('name', JToken) then begin
        JValue := JToken.AsValue();
        Name := JValue.AsText();
    end;
    // For arrays:
    if OrderJson.Get('items', JToken) then
        ItemsArray := JToken.AsArray();
    // For array elements:
    if ItemsArray.Get(Index, JToken) then
        ItemObject := JToken.AsObject();
Incorrect Pattern:
Name := CustomerJson.GetText('name');
ItemsArray := OrderJson.GetArray('items');
ItemObject := ItemsArray.Get(Index).AsObject();
exit(Obj.GetText(KeyName, DefaultValue));

Error Codes: AL0135

8 incomplete-multi-object-generation interface-implementation-pattern 1 CG-AL-H015

Description: The model only generated the interface definition but failed to generate the three required codeunits: 'PayPal Provider' (ID 70215), 'Credit Card Provider' (ID 70216), and 'Payment Service' (ID 70217). The task required implementing a complete interface pattern with multiple objects, but the model stopped after creating just the interface.

Correct Pattern:
interface "Payment Gateway"
{
    procedure Authorize(Amount: Decimal): Boolean;
    procedure GetGatewayName(): Text;
}

codeunit 70215 "PayPal Provider" implements "Payment Gateway"
{
    procedure Authorize(Amount: Decimal): Boolean
    begin
        exit(Amount < 1000);
    end;

    procedure GetGatewayName(): Text
    begin
        exit('PayPal');
    end;
}

codeunit 70216 "Credit Card Provider" implements "Payment Gateway"
{
    procedure Authorize(Amount: Decimal): Boolean
    begin
        exit(true);
    end;

    procedure GetGatewayName(): Text
    begin
        exit('Credit Card');
    end;
}

codeunit 70217 "Payment Service"
{
    procedure Process(Gateway: Interface "Payment Gateway"; Amount: Decimal): Text
    begin
        if Gateway.Authorize(Amount) then
            exit('Authorized by ' + Gateway.GetGatewayName())
        else
            exit('Declined by ' + Gateway.GetGatewayName());
    end;
}
Incorrect Pattern:
interface "Payment Gateway"
{
    procedure Authorize(Amount: Decimal): Boolean;
    procedure GetGatewayName(): Text;
}

Error Codes: AL0185

9 query-count-column-syntax query-aggregation-columns 1 CG-AL-H017

Description: The model incorrectly defined a Count column in an AL query. When using Method = Count, the column should not reference a specific field as its data source. The model wrote 'column(MatchCount; "Dimension Set ID")' with Method = Count, but for Count aggregation, the column should either have no data source field or use a valid pattern. The error AL0353 indicates that a Column must have a valid data source OR have Method = Count (not both in this invalid combination). For CrossJoin queries with Count aggregation, the column definition needs to follow proper AL query aggregation syntax.

Correct Pattern:
For a Count column in AL queries, when using Method = Count, you typically need to either:
1. Use a valid field reference that can be counted, or
2. Define the column without a field reference when counting rows:

column(MatchCount; "Entry No.")
{
    Method = Count;
}

Or ensure proper DataItemLink is established for the LeftOuterJoin to have valid data source context.
Incorrect Pattern:
column(MatchCount; "Dimension Set ID")
{
    Method = Count;
}

Error Codes: AL0353

10 codeunit-self-reference codeunit-fluent-api 1 CG-AL-H018

Description: The model attempted to use 'Rec' to reference the current codeunit instance, but 'Rec' is only valid in table contexts (for record variables), not in codeunits. In AL, codeunits don't have a 'Rec' or 'this' equivalent. For fluent API patterns in codeunits, the model should simply return the codeunit variable directly using 'exit(Self)' where Self is declared as the codeunit type, or more commonly in AL, use a SingleInstance codeunit pattern or return 'this' codeunit by declaring a local variable and assigning it properly.

Correct Pattern:
procedure SetUrl(Url: Text): Codeunit "CG Request Builder"
begin
    RequestUrl := Url;
    exit(this);
end;

// Or using a module-level self-reference pattern with SingleInstance:
// The codeunit should be marked as SingleInstance = true;
// and methods should simply exit(CurrCodeunit) or use a stored self-reference
Incorrect Pattern:
procedure SetUrl(Url: Text): Codeunit "CG Request Builder"
var
    Instance: Codeunit "CG Request Builder";
begin
    RequestUrl := Url;
    Instance := Rec;
    exit(Instance);
end;

Error Codes: AL0118

11 foreach-dictionary-keys-syntax dictionary-iteration 1 CG-AL-H020

Description: The model incorrectly used 'foreach Key in Dict1 do' and 'foreach Key in Dict do' syntax to iterate over dictionary keys. In AL, when iterating over a Dictionary using foreach, you must use the Keys() method to get the list of keys first. The correct syntax is 'foreach Key in Dict.Keys() do' or alternatively iterate using 'Dict.Keys.Get()' in a for loop. The model's syntax 'foreach Key in Dict1 do' is not valid AL syntax for dictionary iteration.

Correct Pattern:
foreach Key in Dict1.Keys() do
    MergedDict.Add(Key, Dict1.Get(Key));

foreach Key in Dict.Keys() do
    KeysList.Add(Key);
Incorrect Pattern:
foreach Key in Dict1 do
    MergedDict.Add(Key, Dict1.Get(Key));

foreach Key in Dict do
    KeysList.Add(Key);

Error Codes: AL0519

12 recordref-open-return-type recordref-operations 1 CG-AL-H022

Description: The model incorrectly assumed that RecordRef.Open() returns a Boolean value that can be compared with '= false'. In AL, RecordRef.Open() is a procedure that returns nothing (void). It throws a runtime error if the table cannot be opened. To handle invalid table IDs, the code should use a TryFunction or check if the table exists before opening. The model wrote 'if RecRef.Open(TableId) = false then' which causes compilation error AL0175 because Open() returns 'None' (void), not Boolean.

Correct Pattern:
// Option 1: Use TryFunction
[TryFunction]
local procedure TryOpenTable(var RecRef: RecordRef; TableId: Integer)
begin
    RecRef.Open(TableId);
end;

// Then call:
if not TryOpenTable(RecRef, TableId) then
    exit('');

// Option 2: Just open and handle error with try pattern
procedure GetTableName(TableId: Integer): Text
var
    RecRef: RecordRef;
begin
    RecRef.Open(TableId);  // Will error if invalid
    exit(RecRef.Name);
end;
Incorrect Pattern:
if RecRef.Open(TableId) = false then
    exit('');

Error Codes: AL0175

13 interface-codeunit-parameter interface-usage 1 CG-AL-H023

Description: The model incorrectly declared a procedure parameter using 'Codeunit "IFieldTransformer"' syntax. In AL, when a procedure accepts an interface implementation, the parameter type should be the interface name directly (without the 'Codeunit' keyword), not 'Codeunit "InterfaceName"'. The compiler error AL0185 indicates it's looking for a codeunit named 'IFieldTransformer' which doesn't exist - the model defined an interface, not a codeunit.

Correct Pattern:
procedure TransformRecord(var RecRef: RecordRef; Transformer: Interface "IFieldTransformer"): Integer

OR more commonly in AL:

procedure TransformRecord(var RecRef: RecordRef; Transformer: "IFieldTransformer"): Integer

Where IFieldTransformer is the interface type directly, not wrapped in 'Codeunit'
Incorrect Pattern:
procedure TransformRecord(var RecRef: RecordRef; Transformer: Codeunit "IFieldTransformer"): Integer

Error Codes: AL0185

14 dataitem-link-syntax report-dataitem-definition 1 CG-AL-M007

Description: The model used incorrect syntax for DataItemLink in report dataitems. In AL reports, DataItemLink must use the FIELD keyword to reference parent dataitem fields, not direct field references. The model wrote 'DataItemLink = "Bill-to Customer No." = Customer."No."' but should have written 'DataItemLink = "Bill-to Customer No." = FIELD("No.")'. Similarly for SalesLine linking to SalesHeader.

Correct Pattern:
dataitem(SalesHeader; "Sales Header")
{
    DataItemLink = "Bill-to Customer No." = FIELD("No.");
    ...
}

dataitem(SalesLine; "Sales Line")
{
    DataItemLink = "Document Type" = FIELD("Document Type"), "Document No." = FIELD("No.");
Incorrect Pattern:
dataitem(SalesHeader; "Sales Header")
{
    DataItemLink = "Bill-to Customer No." = Customer."No.";
    ...
}

dataitem(SalesLine; "Sales Line")
{
    DataItemLink = "Document Type" = SalesHeader."Document Type", "Document No." = SalesHeader."No.";

Error Codes: AL0104

15 approval-entry-document-type-enum enum-values 1 CG-AL-M008

Description: The model incorrectly used 'ApprovalEntry."Document Type"::Purchase' but the Approval Entry table's Document Type field uses a different enum with values like 'Purchase Order', 'Purchase Invoice', etc., not simply 'Purchase'. Additionally, the model referenced a non-existent field 'Date-Time Approved' on the Approval Entry record - the correct field name is likely 'Last Date-Time Modified' or similar. The model lacks knowledge of the specific enum values and field names in the standard BC Approval Entry table.

Correct Pattern:
ApprovalEntry."Document Type" := ApprovalEntry."Document Type"::"Purchase Order";
// Use existing fields like 'Last Date-Time Modified' or 'Date-Time Sent for Approval' instead of non-existent 'Date-Time Approved'
Incorrect Pattern:
ApprovalEntry."Document Type" := ApprovalEntry."Document Type"::Purchase;
...
ApprovalEntry."Date-Time Approved" := CurrentDateTime;

Error Codes: AL0132

16 jsonobject-keys-enumeration json-object-iteration 1 CG-AL-M021

Description: The model attempted to enumerate JsonObject keys using an integer index with JsonObject.Get(Index, JsonToken), which is not valid AL syntax. In AL, JsonObject.Get() requires a key name (Text) as the first parameter, not an integer index. The model also incorrectly tried to use JsonToken.Path.Substring(2) to extract key names. The correct approach to iterate over JsonObject keys in AL is to use the Keys() method which returns a List of [Text], or to use SelectToken/Contains patterns.

Correct Pattern:
local procedure GetJsonObjectKeys(JsonObject: JsonObject): List of [Text]
var
    Keys: List of [Text];
begin
    Keys := JsonObject.Keys();
    exit(Keys);
end;
Incorrect Pattern:
local procedure GetJsonObjectKeys(JsonObject: JsonObject): List of [Text]
var
    Keys: List of [Text];
    JsonToken: JsonToken;
    Index: Integer;
begin
    Index := 0;
    if JsonObject.Get(Index, JsonToken) then
        repeat
            Keys.Add(JsonToken.Path.Substring(2));
            Index += 1;
        until not JsonObject.Get(Index, JsonToken);
    exit(Keys);
end;

Error Codes: AL0519

17 temporary-table-deleteall-scope temporary-table 1 CG-AL-H003

Description: The model used TempResult.DeleteAll() to clear the temporary table, but this only deletes records that match the current filters on the record variable. Since TempResult is passed by reference from the test with pre-inserted data, and the test inserts a record with 'Line No.' = 99999 and 'Item No.' = 'DUMMY', the DeleteAll() should work. However, the real issue is that the test is looking for items with Inventory >= 100 and Unit Price > 0 in the actual Item table, and if no such items exist in the test database, the test exits early. But when items DO exist, the test fails because the model's code processes items correctly but the TempResult.SetRange and FindFirst may not find the expected item. The actual problem is that the model's code works correctly, but the test database may have items with high inventory that get the 15% discount, yet the test's SetRange on 'Item No.' after ProcessItemsWithDiscount doesn't find them because the Item record variable used to find the item initially is a different instance than what's in TempResult. The model correctly implements the logic, but there may be a data-dependent issue where the specific item found by the test query isn't being added to TempResult for some reason - possibly the Item.Inventory field returns a different value than expected (it's a FlowField that needs CalcFields).

Correct Pattern:
Item.CalcFields(Inventory);
CalculatedDiscount := CalculateDiscount(Item.Inventory);
Incorrect Pattern:
if Item.FindSet() then
    repeat
        if Item."Unit Price" > 0 then begin
            CalculatedDiscount := CalculateDiscount(Item.Inventory);
            ...
18 volume-discount-return-value business-logic-implementation 1 CG-AL-M002

Description: The model implemented ApplyVolumeDiscount to return the discount amount instead of the discounted total. The task clearly states 'apply tiered discounts' and the tests expect the function to return the order total AFTER the discount is applied (e.g., 500 returns 500, 2000 returns 1900, 10000 returns 9000). The model incorrectly returns just the discount amount (e.g., for 500 it returns 0, for 2000 it returns 100, for 10000 it returns 1000).

Correct Pattern:
procedure ApplyVolumeDiscount(OrderTotal: Decimal): Decimal
var
    DiscountPercent: Decimal;
begin
    if OrderTotal > 5000 then
        DiscountPercent := 10
    else if OrderTotal >= 1000 then
        DiscountPercent := 5
    else
        DiscountPercent := 0;

    exit(OrderTotal - (OrderTotal * (DiscountPercent / 100)));
end;
Incorrect Pattern:
procedure ApplyVolumeDiscount(OrderTotal: Decimal): Decimal
var
    DiscountPercent: Decimal;
begin
    DiscountPercent := 0;

    if OrderTotal >= 5000 then
        DiscountPercent := 10
    else if OrderTotal >= 1000 then
        DiscountPercent := 5
    else
        DiscountPercent := 0;

    exit(OrderTotal * (DiscountPercent / 100));
end;