Skip to content

adds formatter option for preserving comments (redo) #187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dzsquared wants to merge 2 commits into
base: main
Choose a base branch
from
Open

adds formatter option for preserving comments (redo) #187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6783258
revised approach
dzsquared Jan 29, 2026
db61757
expanded tests
dzsquared Jan 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/CommentInfo.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
//------------------------------------------------------------------------------
// <copyright file="CommentInfo.cs" company="Microsoft">
// Copyright (c) Microsoft Corporation. All rights reserved.
// </copyright>
//------------------------------------------------------------------------------

namespace Microsoft.SqlServer.TransactSql.ScriptDom.ScriptGenerator
{
/// <summary>
/// Intermediate structure for processing comments during script generation.
/// </summary>
internal sealed class CommentInfo
{
/// <summary>
/// Gets or sets the original comment token from the token stream.
/// </summary>
public TSqlParserToken Token { get; set; }

/// <summary>
/// Gets or sets the position of the comment relative to code.
/// </summary>
public CommentPosition Position { get; set; }

/// <summary>
/// Gets or sets the token index of the associated fragment.
/// </summary>
public int AssociatedFragmentIndex { get; set; }

/// <summary>
/// Gets whether this is a single-line comment (-- style).
/// </summary>
public bool IsSingleLineComment
{
get { return Token != null && Token.TokenType == TSqlTokenType.SingleLineComment; }
}

/// <summary>
/// Gets whether this is a multi-line comment (/* */ style).
/// </summary>
public bool IsMultiLineComment
{
get { return Token != null && Token.TokenType == TSqlTokenType.MultilineComment; }
}

/// <summary>
/// Gets the text content of the comment.
/// </summary>
public string Text
{
get { return Token?.Text; }
}

/// <summary>
/// Creates a new CommentInfo instance.
/// </summary>
/// <param name="token">The comment token.</param>
/// <param name="position">The position relative to code.</param>
/// <param name="fragmentIndex">The associated fragment token index.</param>
public CommentInfo(TSqlParserToken token, CommentPosition position, int fragmentIndex)
{
Token = token;
Position = position;
AssociatedFragmentIndex = fragmentIndex;
}
}
}
29 changes: 29 additions & 0 deletions SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/CommentPosition.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
//------------------------------------------------------------------------------
// <copyright file="CommentPosition.cs" company="Microsoft">
// Copyright (c) Microsoft Corporation. All rights reserved.
// </copyright>
//------------------------------------------------------------------------------

namespace Microsoft.SqlServer.TransactSql.ScriptDom.ScriptGenerator
{
/// <summary>
/// Categorizes comment placement relative to code during script generation.
/// </summary>
internal enum CommentPosition
{
/// <summary>
/// Comment appears before the associated code on previous line(s).
/// </summary>
Leading,

/// <summary>
/// Comment appears after code on the same line.
/// </summary>
Trailing,

/// <summary>
/// Comment not directly associated with code (e.g., end of file, standalone block).
/// </summary>
Standalone
Comment on lines +22 to +27
Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CommentPosition.Standalone enum value is defined but never used in the codebase. All comments are classified as either Leading or Trailing. If Standalone comments are not needed for this implementation, remove this enum value to avoid confusion. If they are needed for future functionality (e.g., orphaned comments at the end of a script), add a comment explaining the intended use case.

Suggested change
Trailing,
/// <summary>
/// Comment not directly associated with code (e.g., end of file, standalone block).
/// </summary>
Standalone
Trailing

Copilot uses AI. Check for mistakes.
}
}
264 changes: 264 additions & 0 deletions SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.Comments.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,264 @@
//------------------------------------------------------------------------------
// <copyright file="SqlScriptGeneratorVisitor.Comments.cs" company="Microsoft">
// Copyright (c) Microsoft Corporation. All rights reserved.
// </copyright>
//------------------------------------------------------------------------------
using System;
using System.Collections.Generic;

namespace Microsoft.SqlServer.TransactSql.ScriptDom.ScriptGenerator
{
internal abstract partial class SqlScriptGeneratorVisitor
{
#region Comment Tracking Fields

/// <summary>
/// Tracks the last token index processed for comment emission.
/// Used to find comments between visited fragments.
/// </summary>
private int _lastProcessedTokenIndex = -1;

/// <summary>
/// The current script's token stream, set when visiting begins.
/// </summary>
private IList<TSqlParserToken> _currentTokenStream;
Comment on lines +13 to +24
Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _lastProcessedTokenIndex and _currentTokenStream fields are instance fields of SqlScriptGeneratorVisitor without any thread synchronization. If the same SqlScriptGenerator instance is used concurrently from multiple threads (which is not a typical pattern but is possible), these fields could be corrupted. Consider documenting that SqlScriptGenerator instances are not thread-safe and should not be shared across threads, or add appropriate thread-safety mechanisms.

Copilot uses AI. Check for mistakes.

#endregion

#region Comment Preservation Methods

/// <summary>
/// Sets the token stream for comment tracking.
/// Call this before visiting the root node when PreserveComments is enabled.
/// </summary>
/// <param name="tokenStream">The token stream from the parsed script.</param>
protected void SetTokenStreamForComments(IList<TSqlParserToken> tokenStream)
{
_currentTokenStream = tokenStream;
_lastProcessedTokenIndex = -1;
}

/// <summary>
/// Gets leading comments that appear between the last processed token and the current fragment.
/// </summary>
/// <param name="fragment">The current fragment being visited.</param>
/// <returns>List of comment information for leading comments.</returns>
protected List<CommentInfo> GetLeadingComments(TSqlFragment fragment)
{
var comments = new List<CommentInfo>();

if (_currentTokenStream == null || fragment == null || !_options.PreserveComments)
{
return comments;
}

int startIndex = _lastProcessedTokenIndex + 1;
int endIndex = fragment.FirstTokenIndex;

// Scan for comments between last processed and current fragment
for (int i = startIndex; i < endIndex && i < _currentTokenStream.Count; i++)
{
var token = _currentTokenStream[i];
if (IsCommentToken(token))
{
comments.Add(new CommentInfo(token, CommentPosition.Leading, fragment.FirstTokenIndex));
Comment on lines +59 to +64
Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CommentInfo constructor receives 'fragmentIndex' parameter but the comment gathering methods (GetLeadingComments, GetTrailingComments) don't pass the actual token index of the comment itself. The fragmentIndex parameter should be renamed to clarify it represents the associated fragment's index, not the comment's own token index. Better yet, add a TokenIndex property to CommentInfo and populate it with the actual index (i) when creating CommentInfo objects in GetLeadingComments and GetTrailingComments. This would enable efficient token tracking without the nested loop in AfterVisitFragment.

Copilot uses AI. Check for mistakes.
}
}

return comments;
}

/// <summary>
/// Gets trailing comments that appear on the same line after the fragment.
/// </summary>
/// <param name="fragment">The current fragment being visited.</param>
/// <returns>List of comment information for trailing comments.</returns>
protected List<CommentInfo> GetTrailingComments(TSqlFragment fragment)
{
var comments = new List<CommentInfo>();

if (_currentTokenStream == null || fragment == null || !_options.PreserveComments)
{
return comments;
}

int lastTokenIndex = fragment.LastTokenIndex;
if (lastTokenIndex < 0 || lastTokenIndex >= _currentTokenStream.Count)
{
return comments;
}

var lastToken = _currentTokenStream[lastTokenIndex];
int lastTokenLine = lastToken.Line;

// Scan for comments after the last token on the same line
for (int i = lastTokenIndex + 1; i < _currentTokenStream.Count; i++)
{
var token = _currentTokenStream[i];

// Stop if we've gone past the same line (unless it's whitespace with no newline)
if (token.Line > lastTokenLine)
{
break;
}
Comment on lines +99 to +103
Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The GetTrailingComments method only captures comments on the same line as the fragment's last token (checking token.Line > lastTokenLine). This means comments that appear on subsequent lines after a statement, but before the next statement, will be missed and not emitted. These "orphaned" comments would typically be captured as leading comments of the next fragment, which is acceptable. However, comments at the very end of the script (after the last statement) would not be captured at all unless TSqlScript.LastTokenIndex extends beyond the last statement to include them. Consider documenting this behavior or adding a test case to verify end-of-script comments are handled correctly.

Copilot uses AI. Check for mistakes.

// Found a comment on the same line
if (IsCommentToken(token))
{
comments.Add(new CommentInfo(token, CommentPosition.Trailing, lastTokenIndex));

// For single-line comments, there can't be anything else after on this line
if (token.TokenType == TSqlTokenType.SingleLineComment)
{
break;
}
}
}

return comments;
}

/// <summary>
/// Emits a comment using the script writer with current indentation.
/// </summary>
/// <param name="commentInfo">The comment to emit.</param>
protected void EmitComment(CommentInfo commentInfo)
{
if (commentInfo == null || commentInfo.Token == null)
{
return;
}

var text = commentInfo.Text;
if (string.IsNullOrEmpty(text))
{
return;
}

if (commentInfo.IsSingleLineComment)
{
EmitSingleLineComment(text, commentInfo.Position);
}
else if (commentInfo.IsMultiLineComment)
{
EmitMultiLineComment(text, commentInfo.Position);
}
}

/// <summary>
/// Emits a single-line comment.
/// </summary>
private void EmitSingleLineComment(string text, CommentPosition position)
{
if (position == CommentPosition.Trailing)
{
// Trailing: add space before comment, keep on same line
_writer.AddToken(ScriptGeneratorSupporter.CreateWhitespaceToken(1));
}
else
{
// Leading: comment goes on its own line
// Indentation is already applied by current context
}

// Write the comment as-is (preserving the -- prefix)
_writer.AddToken(new TSqlParserToken(TSqlTokenType.SingleLineComment, text));

if (position == CommentPosition.Leading)
{
// After a leading comment, we need a newline
_writer.NewLine();
}
}

/// <summary>
/// Emits a multi-line comment, preserving internal structure.
/// </summary>
private void EmitMultiLineComment(string text, CommentPosition position)
{
if (position == CommentPosition.Trailing)
{
// Trailing: add space before comment
_writer.AddToken(ScriptGeneratorSupporter.CreateWhitespaceToken(1));
}

// For multi-line comments, we preserve the content as-is
// The comment includes /* and */ delimiters
_writer.AddToken(new TSqlParserToken(TSqlTokenType.MultilineComment, text));

if (position == CommentPosition.Leading)
{
// After a leading multi-line comment, add newline
_writer.NewLine();
}
}

/// <summary>
/// Called before visiting a fragment to emit any leading comments.
/// </summary>
/// <param name="fragment">The fragment about to be visited.</param>
protected void BeforeVisitFragment(TSqlFragment fragment)
{
if (!_options.PreserveComments || _currentTokenStream == null || fragment == null)
{
return;
}

var leadingComments = GetLeadingComments(fragment);
foreach (var comment in leadingComments)
{
EmitComment(comment);
}
}

/// <summary>
/// Called after visiting a fragment to emit any trailing comments and update tracking.
/// </summary>
/// <param name="fragment">The fragment that was just visited.</param>
protected void AfterVisitFragment(TSqlFragment fragment)
{
if (!_options.PreserveComments || _currentTokenStream == null || fragment == null)
{
return;
}

var trailingComments = GetTrailingComments(fragment);
foreach (var comment in trailingComments)
{
EmitComment(comment);
}

// Update the last processed token index
if (fragment.LastTokenIndex >= 0)
{
// Account for any trailing comments we just emitted
int newLastIndex = fragment.LastTokenIndex;
foreach (var comment in trailingComments)
{
// Find the index of this comment token
for (int i = newLastIndex + 1; i < _currentTokenStream.Count; i++)
{
if (_currentTokenStream[i] == comment.Token)
{
newLastIndex = i;
break;
}
}
Comment on lines +236 to +246
Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The token lookup logic uses reference equality comparison (_currentTokenStream[i] == comment.Token) which is inefficient and potentially fragile. Instead of linear search through tokens on each comment, consider tracking token indices directly in CommentInfo. Store the actual token index from the stream when creating CommentInfo, then update _lastProcessedTokenIndex directly from that stored value. This would eliminate the nested loop and improve performance when there are many trailing comments.

Copilot uses AI. Check for mistakes.
}
_lastProcessedTokenIndex = newLastIndex;
}
}

/// <summary>
/// Checks if a token is a comment token.
/// </summary>
private static bool IsCommentToken(TSqlParserToken token)
{
return token != null &&
(token.TokenType == TSqlTokenType.SingleLineComment ||
token.TokenType == TSqlTokenType.MultilineComment);
}

#endregion
}
}
12 changes: 12 additions & 0 deletions SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.TSqlBatch.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,30 @@ partial class SqlScriptGeneratorVisitor
{
public override void ExplicitVisit(TSqlBatch node)
{
// Emit leading comments before the batch
BeforeVisitFragment(node);

foreach (TSqlStatement statement in node.Statements)
{
// Emit leading comments before each statement
BeforeVisitFragment(statement);

GenerateFragmentIfNotNull(statement);

GenerateSemiColonWhenNecessary(statement);

// Emit trailing comments after each statement
AfterVisitFragment(statement);

if (statement is TSqlStatementSnippet == false)
{
NewLine();
NewLine();
}
}

// Emit trailing comments after the batch
AfterVisitFragment(node);
}
Comment on lines 12 to 38
Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment preservation logic only calls BeforeVisitFragment/AfterVisitFragment on TSqlScript, TSqlBatch, and TSqlStatement nodes. Comments embedded within sub-expressions (e.g., inside CASE expressions, JOIN clauses, column definitions, or WHERE predicates) will not be preserved because these fragment types don't have the Before/AfterVisitFragment calls. Either document this limitation clearly in the PreserveComments property documentation, or expand the implementation to call these methods for all fragment types (which would require modifying the visitor pattern significantly).

Copilot uses AI. Check for mistakes.
}
}
Loading
Loading