Skip to content

Conversation

@DecimalTurn
Copy link
Contributor

@DecimalTurn DecimalTurn commented Oct 18, 2025

This pull request adds direct unit tests for the ANTLR VBA preprocessor parser, allowing us to catch syntax errors and undesired implicit tokens without relying on the VS Code diagnostics layer. It introduces a new test script in package.json, a dedicated test file for parser validation; and new test cases to verify correct parsing of function calls.

I originally discovered an issue with the preprocessor when dealing with this line:

Err.Raise 10101, "XMLConverter", xml_ParseErrorMessage(xml_String, xml_Index, "Expecting '<'")

Sometimes, depending on if there was an extra space somewhere on this line or not, the precompilation would break and cause issues with conditional compilation. I've managed to simplify the problem to be about matching the last closing parenthesis on the line without it being caught by the implicit T__1 or T__2 token. See ParsingParenthesis.bas for a minimal reproducible example with the following line:

y = Format( "Test '<'")

Which leads to a syntax error that can be observed using Debug ANTLR4 Pre grammar:

image

Or by using the new test script this PR introduces:

    📝 Input:
       1: y = Format( "Test '<'")
       2: 
       3: 
    🔤 Tokens:
       0: ANYCHARS     = "y"
       1: WS           = " "
       2: EQ           = "="
       3: WS           = " "
       4: ANYCHARS     = "Format("
       5: WS           = " "
       6: LITSTRING    = ""Test '<'""
       7: T__1         = ")"
       8: NEWLINE      = "\r\n\r\n\r\n"
    📊 Syntax errors: 1
    ❌ Found 1 implicit token(s): T__1
    1) should parse function call with string literal and parentheses

I have the fix below ready, but I'd like to see the CI run without the fix first to demonstrate the issue.
Basically, simply having an explicit definition for LPAREN and RPAREN and including them as a reservedWord solves the above issue:

 server/src/antlr/vbapre.g4 | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/server/src/antlr/vbapre.g4 b/server/src/antlr/vbapre.g4
index face58f..bf0be80 100644
--- a/server/src/antlr/vbapre.g4
+++ b/server/src/antlr/vbapre.g4
@@ -23,7 +23,7 @@ constDirectiveName
     ;
 
 directiveParenthesizedExpression
-    : '(' WS? directiveExpression WS? ')'
+    : LPAREN WS? directiveExpression WS? RPAREN
     ;
 
 directiveLiteralExpression
@@ -154,6 +154,8 @@ reservedWord
     | PLUS
     | SUBT
     | THEN
+    | LPAREN
+    | RPAREN
     ;
 
 unreservedWord
@@ -292,6 +294,14 @@ fragment UNDERSCORE
     : '_'
     ;
 
+LPAREN
+    : '('
+    ;
+
+RPAREN
+    : ')'
+    ;
+
 COLON
     : ':'
     ;

@DecimalTurn
Copy link
Contributor Author

@SSlinky, I know this PR is still in draft mode, but could you please allow the workflow to run? I need it to illustrate the error I'm trying to solve with this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant