feat: Add --lint support for validating definition files#27
feat: Add --lint support for validating definition files#27
Conversation
Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/a75bab3c-44b6-4dcf-8da7-eb85c41f6ec3
…ge assertions, corrupted YAML tests Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/245f177b-039d-4058-a8eb-3fce0818cd47
| // Assert — non-zero exit code and log contains an error about invalid content | ||
| var logContent = File.ReadAllText(logFile); | ||
| Assert.AreEqual(1, context.ExitCode); | ||
| Assert.Contains("Error:", logContent); |
There was a problem hiding this comment.
I'm going to need to see more information and more tests than this - does it say WHAT file is bad. Just saying it contains "Error:" doesn't tell me it actually provides useful corrupted-file information.
I'm also going to want a test where a parent requirements.yaml file is valid, but it includes a child file which is corrupt, and I want to see that included file is reported as being the source of the problem.
Ideally I'd like to see that both the file and line number is reported. This is a "--lint" test, and linting is supposed to provide enough information for the user to actually fix the problem.
There was a problem hiding this comment.
Addressed in commit 72a7c70:
Error messages now include filename and line number. Load uses a new ReviewMarkConfigurationHelpers.DeserializeRaw(yaml, filePath) helper that formats YAML parse errors as "Failed to parse 'filename' at line X, column Y: ...". Semantic errors (missing evidence-source, etc.) are formatted as "Invalid configuration in 'filename': ...".
Includes support added. The .reviewmark.yaml format now accepts an includes: list of relative paths to child fragment files. Their reviews and needs-review content is merged into the parent. If a child file is missing or corrupted, the error message names the child file (not the parent) with line/column.
Tests strengthened:
CorruptedYamlandIncludedFileCorruptedtests now assert"at line"(not just"line") so a numeric position is verified- New
Program_Run_WithLintFlag_IncludedFileCorrupted_ReportsChildFilename: valid parent, corrupted child; asserts child filename and"at line"in error - 5 new
ReviewMarkConfigurationTests: filename-in-error for invalid YAML and missing evidence-source; includes merging; missing/corrupted included file errors naming the child file and line
…pport, and stronger test assertions Co-authored-by: Malcolmnixon <1863707+Malcolmnixon@users.noreply.github.com> Agent-Logs-Url: https://github.com/demaconsulting/ReviewMark/sessions/7318deb4-3700-47ba-b4b8-b7c3e37c4b4e
Adds a
--lintcommand that reads the.reviewmark.yamlconfiguration file and reports structural and semantic issues, with non-zero exit code on failure. Lint errors always identify the problematic file and, for YAML syntax errors, the line and column number.Description
Context.csLintboolean property and--lintflag parsing (mirrors--validatepattern)Program.cs--lintto help textRun()(between self-validation and main tool logic)RunLintLogicthat:--definitionif specified, else.reviewmark.yamlin working dir)context.WriteError, always naming the problematic file and line numberevidence-sourcetypes (must beurlorfileshare)reviews[1] has duplicate ID 'Core-Logic' (first defined at reviews[0])ReviewMarkConfiguration.csReviewMarkConfigurationHelpersclass containingDeserializeRawandBuildConfigurationmethods, allowing file-local YAML types to appear in method signaturesLoadnow embeds the filename in all exceptions:"Failed to parse 'filename' at line X, column Y: <message>""Invalid configuration in 'filename': <message>""Failed to read configuration file 'filename': <message>"includes:support: a parent.reviewmark.yamlcan list relative paths to child fragment files; theirreviewsandneeds-reviewpatterns are merged into the parent. Errors from included files name the child file and line number.Validation.csReviewMark_Lintself-validation testTestDirectory.cs(new)TestDirectory— anIDisposablehelper class for the test project that creates and auto-deletes a temporary directory, replacing the repetitivetry/finallypattern across testsTests
ContextTests:Context_Create_LintFlag_SetsLintTrue,Context_Create_NoArguments_LintIsFalseProgramTests: refactored elaborate and lint tests to useTestDirectory; updated all error tests to capture output via--logand assert specific error message content (filename,"at line", error type); added:Program_Run_WithLintFlag_CorruptedYaml_ReportsError— verifies filename and"at line"appear in the errorProgram_Run_WithLintFlag_MissingEvidenceSource_ReportsError— verifies filename and"evidence-source"appear in the errorProgram_Run_WithLintFlag_IncludedFileCorrupted_ReportsChildFilename— valid parent includes a corrupted child; verifies child filename and"at line"appear in the errorReviewMarkConfigurationTests: added five new tests coveringLoaderror message quality and theincludesfeature:ReviewMarkConfiguration_Load_InvalidYaml_ErrorIncludesFilenameAndLineReviewMarkConfiguration_Load_MissingEvidenceSource_ErrorIncludesFilenameReviewMarkConfiguration_Load_WithIncludes_MergesReviewsReviewMarkConfiguration_Load_MissingIncludedFile_ErrorNamesIncludedFileReviewMarkConfiguration_Load_CorruptedIncludedFile_ErrorNamesIncludedFileAndLinerequirements.yamlReviewMark-Cmd-Lintrequirement linked to all 13 test casesReviewMark_Lintto all platform requirements (Windows/Linux/macOS, .NET 8/9/10)Type of Change
Related Issues
Pre-Submission Checklist
Before submitting this pull request, ensure you have completed the following:
Build and Test
dotnet build --configuration Releasedotnet test --configuration Releasedotnet run --project src/DemaConsulting.ReviewMark --configuration Release --framework net10.0--no-build -- --validateCode Quality
dotnet format --verify-no-changesQuality Checks
Please run the following checks before submitting:
cspell "**/*.{md,cs}"markdownlint "**/*.md"yamllint .Testing
Documentation
Additional Notes
The lint check intentionally reports all detectable issues rather than stopping at the first error, so users can fix multiple problems in one pass. All error messages identify the problematic file; YAML syntax errors also include the line and column number. Error tests capture output via
--logand assert specific content (filename,"at line", field name) so assertions are reliable regardless of--silentmode.Original prompt
🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.