openai/gpt-5.3-codex
Known Shortcomings (14)
Sorted by occurrence count (most frequent first)
| # | Concept | AL Concept | Count | Affected Tasks |
|---|---|---|---|---|
| 1 | query-object-syntax | query-definition | 2 | CG-AL-H011, CG-AL-H017 |
|
Description: The model failed to generate any valid AL code for the query object. The compilation errors at line 12 (identifier expected, '=' expected) and the note 'Generated code not found' suggest the model either produced no output or produced syntactically invalid AL for a Query object. The model likely does not know the correct syntax for defining an AL Query object with dataitems, columns, aggregation methods (Sum, Count), column filters, and ordering. AL Query objects have a specific structure with 'query', 'elements', 'dataitem', 'column', and 'filter' keywords that differs significantly from other AL object types.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0107, AL0353 |
||||
| 2 | empty-or-missing-code-generation | interface-definition | 2 | CG-AL-H021, CG-AL-M009 |
|
Description: The model failed to generate any valid AL code. The generated code appears to be empty or contains no recognizable AL object declarations. The task required creating an interface 'INotificationChannel', three implementing codeunits (70221, 70222, 70223), and a manager codeunit (70220) using List of [Interface] and Dictionary of [Text, Interface] collections. The compilation errors (AL0107 'identifier expected', AL0198 'Expected one of the application object keywords') at position 1:11 indicate the very first line of the output file is not a valid AL object declaration. The model either produced no code, produced a non-AL response, or failed to structure the output as proper AL objects.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0107, AL0104 |
||||
| 3 | parse-failure | unknown | 2 | CG-AL-M001, CG-AL-M112 |
|
Description: Failed to parse LLM analysis response: Looking at this failure, I need to analyze the compilation errors carefully: 1. **AL0118/AL0132 errors about `"No."`**: The test references `Product."No."` and `Product.SetRange("No.", ...)`, but the
Correct Pattern:
Incorrect Pattern:
|
||||
| 4 | dictionary-keys-method-signature | collection-types | 1 | CG-AL-H020 |
|
Description: The model incorrectly called Dictionary.Keys() with an argument (e.g., Dict.Keys(KeyList)), but in AL the Keys() method takes no arguments and returns a List directly. The correct pattern is to assign the result: KeyList := Dict.Keys(). This error occurs in multiple places: MergeDictionaries (iterating Dict1 and Dict2 keys) and GetKeys (returning dictionary keys).
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0126 |
||||
| 5 | al-syntax-basics | codeunit-definition | 1 | CG-AL-M005 |
|
Description: The model generated AL code with a syntax error at line 255, column 60. The generated code was not captured/found in the output ('Generated code not found'), but the compilation errors (AL0104: Syntax error, ')' expected and '}' expected) indicate the model produced malformed AL code - likely incorrect procedure signatures, malformed expressions, or improperly structured code blocks. The task and test definitions are valid; the model simply failed to produce syntactically correct AL code for the External Payment Service codeunit.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0104 |
||||
| 6 | json-typed-getter-methods | json-api-usage | 1 | CG-AL-H014 |
|
Description: The model failed to generate valid AL code for JSON parsing. The compilation error AL0133 at line 47:33 indicates the model tried to pass a Text value where a Boolean was expected, likely misusing JsonObject methods. In AL, JsonObject doesn't have typed getter methods like GetText(), GetInteger(), GetBoolean() directly. Instead, you must use Get() to retrieve a JsonToken, then convert it to a JsonValue, and call AsText(), AsInteger(), AsBoolean(), AsDecimal() etc. The model either didn't generate code at all (the generated code section says 'not found' but the error suggests code was generated) or generated code using incorrect JSON API patterns. The task description itself uses fictional method names like 'GetText()', 'GetInteger()', 'GetBoolean()', 'GetArray()' which don't exist on JsonObject in AL - the correct approach is JsonObject.Get(Key, JsonToken) then JsonToken.AsValue().AsText() etc. However, this is still a model knowledge gap since a knowledgeable model should know the correct AL JSON API and translate the task intent accordingly.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0133 |
||||
| 7 | secrettext-isolated-storage-api | secrettext-handling | 1 | CG-AL-H016 |
|
Description: The model generated code that attempts to pass SecretText values to IsolatedStorage.Set and IsolatedStorage.Get methods incorrectly. The AL0133 errors ('cannot convert from SecretText to Joker') at lines 23 and 39 indicate the model did not correctly use the IsolatedStorage overloads that accept SecretText. In AL, IsolatedStorage has specific overloads for SecretText: IsolatedStorage.Set(Text, SecretText [, DataScope]) for storing and IsolatedStorage.Get(Text, DataScope, var SecretText) for retrieving. The model likely tried to use the wrong overload or passed arguments in the wrong order/type, causing the type conversion errors. The generated code was not captured but the compilation errors clearly show the model failed to correctly call IsolatedStorage APIs with SecretText parameters.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0133 |
||||
| 8 | table-trigger-names | table-triggers | 1 | CG-AL-M010 |
|
Description: The model used invalid trigger names 'OnAfterInsert', 'OnAfterModify', and 'OnAfterDelete' on a table object. In AL, the valid table triggers are 'OnInsert', 'OnModify', 'OnDelete', and 'OnRename'. The 'OnAfter*' variants are integration event names (e.g., OnAfterInsertEvent) that are published by the system but are not valid as trigger declarations within a table definition. The model confused integration/subscription event names with actual table trigger names.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0162 |
||||
| 9 | table-extension-structure | table-extension-definition | 1 | CG-AL-M006 |
|
Description: The model failed to generate valid AL code for the table extension. The compilation errors (AL0104 'Syntax error, } expected' and AL0198 'Expected one of the application object keywords') at line 64 indicate the generated code has fundamental structural/syntax issues - likely malformed table extension definition, misplaced procedures, or incorrect block nesting. The generated code was either empty, truncated, or structurally invalid. The model needed to produce a complete tableextension object with fields, procedures (UpdateRiskLevel, CalculatePaymentHistoryRating, GetCreditLimit, ValidateNewOrder, TriggerRiskAssessment), and proper validation triggers, but failed to generate syntactically valid AL code.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0104 |
||||
| 10 | table-field-property-context | table-definition | 1 | CG-AL-H005 |
|
Description: The model generated AL code for the two tables but incorrectly used the 'Caption' property in a context where it is not allowed (error AL0124). The compilation error at line 121 indicates the model placed a Caption property somewhere invalid—likely on a field or object in a way that AL does not permit. The model failed to produce syntactically correct table definitions. Additionally, the generated code was not captured ('Generated code not found'), suggesting the model may have produced malformed output that couldn't be properly extracted, but the compilation was still attempted and failed with the Caption property error. This is a model knowledge gap about proper AL table/field property usage.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0124 |
||||
| 11 | complex-report-with-helper-codeunit | report-definition-and-codeunit-generation | 1 | CG-AL-M007 |
|
Description: The model failed to generate any valid AL code at all. The task required creating both a Report 70001 'Sales Performance Analysis' and a helper codeunit 'CG-AL-M007 Mock Calculator' that the test file references. The generated code appears to be empty or syntactically invalid (the compilation error AL0104 'Syntax error, } expected' at line 11 and 'App generation failed' indicate the model produced malformed or incomplete AL code). The model needed to understand that the test file depends on two objects: (1) a Report object with Customer, Sales Header, and Sales Line data items, and (2) a Codeunit named 'CG-AL-M007 Mock Calculator' with methods like Initialize(), AddSalesLine(), GetRunningTotalByCustomer(), GetRunningTotalByRegion(), CalculateAverageOrderValue(), GetCustomerRank(), GetTopProduct(), GetProductSalesQuantity(), CalculateYoYComparison(), CalculateOrderFrequency(), GetTotalSales(), and GetCustomerCount(). The model either produced no code or fundamentally broken code.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0104 |
||||
| 12 | yaml-parsing-string-manipulation | text-parsing-and-json-handling | 1 | CG-AL-M021 |
|
Description: The model failed to generate valid AL code for the YAML Handler codeunit. The compilation error at line 41 (syntax error, ')' expected) indicates the model produced syntactically invalid AL code, likely misusing string manipulation functions or incorrectly structuring procedure parameters/expressions. The task requires implementing custom YAML parsing (splitting text by line feeds, extracting key-value pairs) and converting between YAML and JSON using AL's JsonObject type. The model apparently struggled with the AL syntax for text manipulation operations needed to parse YAML content, producing code with syntax errors. Since the generated code was not captured ('Generated code not found'), but the compilation errors clearly point to model-generated code issues (line 41 of the generated .al file), this is a model knowledge gap in writing correct AL syntax for string parsing and JsonObject manipulation.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0104 |
||||
| 13 | al-syntax-complex-codeunit | codeunit-procedure-implementation | 1 | CG-AL-M008 |
|
Description: The model generated AL code with syntax errors at line 122, column 49. The generated code was not captured/saved properly ('Generated code not found'), but the compilation errors (AL0104 syntax errors expecting ')', 'end', ';', and '}') indicate the model produced malformed AL code for the Purchase Approval Workflow codeunit. This is a complex codeunit requiring multiple procedures with various parameter types and return values, dictionary/list usage for in-memory tracking, and event subscribers. The model likely made a syntax error in a procedure body - possibly incorrect use of a complex expression, mismatched parentheses, or incorrect variable declaration syntax around line 122 of its output.
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0104 |
||||
| 14 | no-series-auto-number-generation | table-definition-with-triggers | 1 | CG-AL-M003 |
|
Description: The model failed to generate any valid AL code for the Sales Contract table. The compilation errors (AL0185: Page '0' is missing) suggest the model produced a malformed or nearly empty table object that references non-existent pages (likely from incorrect LookupPageId/DrillDownPageId properties set to 0). The 'Generated code not found' note and the trivial compilation errors indicate the model either failed to produce output or produced a stub/skeleton that doesn't properly define the table with all required fields, triggers, and validation logic. The task requires knowledge of: AL table definition syntax, NoSeriesManagement for auto-generating Contract No., OnInsert/OnDelete trigger patterns, field validation triggers with Error() calls, Option fields, TableRelation properties, and proper page ID references (or omitting them when no page exists).
Correct Pattern:
Incorrect Pattern:
Error Codes: AL0185 |
||||