Skip to content

Add modern CI reporting, Phase 7.7 perf regression tests, and CLI --regression command#71

Merged
SureshKViswanathan merged 7 commits intomainfrom
copilot/generate-modern-report-ci
Feb 18, 2026
Merged

Add modern CI reporting, Phase 7.7 perf regression tests, and CLI --regression command#71
SureshKViswanathan merged 7 commits intomainfrom
copilot/generate-modern-report-ci

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 17, 2026

Description

Replaces the minimal one-line CI coverage output with rich GitHub Job Summary reports. Completes Phase 7.7 performance testing milestone with regression baseline tests and exposes the capability via the CLI.

Related Issues

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance improvement
  • Documentation update
  • Code refactoring
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Security fix

Changes Made

CI Reporting (ci.yml):

  • Build & Test job: per-module test result parsing (pass/fail/skip), per-module LCOV coverage with visual █░ progress bars, build/test duration tracking, 90% threshold checks, job outputs for downstream consumption
  • Lint job: dual SwiftLint runs (inline annotations + JSON summary), collapsible per-file violation details
  • Documentation job: per-module doc generation status table
  • Fix: grep -c returns exit code 1 on zero matches — || echo "0" produced "0\n0" breaking arithmetic; changed to || true

Phase 7.7 Performance Regression Tests (PerformanceRegressionTests.swift):

  • 10 tests mapping 1:1 to Phase 7.7 milestone items: v2/v3/FHIR throughput baselines, memory profiling (1,000+ msgs), concurrent scalability (2/4/8 tasks), MLLP framing, object pool reuse, streaming bundles, p50/p95/p99 latency, BenchmarkRunner regression comparison with PASS/WARN/FAIL

CLI --regression flag:

  • hl7 benchmark --regression runs baseline checks against defined thresholds, reports PASS/WARN/FAIL per metric
  • Supports --format json for programmatic consumption
  • Returns exit code 1 on baseline failure
hl7 benchmark --regression
hl7 benchmark --regression --format json -n 500

Docs: CHANGELOG.md [Unreleased], milestone.md Phase 7.7 ✓, README.md CI Reports section.

Testing

Test Coverage

  • Unit tests added/updated
  • Integration tests added/updated
  • Performance tests added/updated
  • Code coverage ≥ 90%

Manual Testing

python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml'))"
# YAML valid, 3 jobs

# grep -c fix verification
RESULT=$(grep -c "nonexistent" /dev/null 2>/dev/null || true)
echo "$RESULT"  # "0" — no arithmetic error

6 new CLI tests (4 parser + 2 execution) for --regression. 10 new performance regression baseline tests.

Documentation

  • API documentation added/updated (DocC comments)
  • README.md updated (if applicable)
  • Examples updated (if applicable)
  • CHANGELOG.md updated under "Unreleased" section
  • Migration guide updated (if breaking change)

Standards Compliance

N/A — CI infrastructure, performance testing, and CLI tooling only.

Checklist

  • My code follows the project's style guidelines (CODING_STANDARDS.md)
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published
  • I have checked my code for security vulnerabilities
  • I have removed all PHI/sensitive data from tests and examples

Performance Impact

No runtime library impact. CI workflow adds ~30s for report generation. CLI --regression executes benchmarks on demand only.

Breaking Changes

None.

Screenshots (if applicable)

Reports render on the GitHub Actions Summary tab after CI runs.

Additional Notes

  • Python snippets in CI use specific exception types (FileNotFoundError, json.JSONDecodeError) instead of bare except:
  • Progress bar handles edge cases where filled or empty count is 0
  • Coverage AWK parser uses split($0, a, ":") for colon-delimited LCOV format
  • CodeQL returned 0 alerts

For Reviewers:

  • Code quality and style
  • Test coverage adequate
  • Documentation complete
  • No security concerns
  • Performance acceptable
  • Standards compliance verified

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits February 17, 2026 16:19
…ts, coverage breakdown, lint report, and documentation status

Co-authored-by: SureshKViswanathan <257696045+SureshKViswanathan@users.noreply.github.com>
…pecific exception handling in Python snippets

Co-authored-by: SureshKViswanathan <257696045+SureshKViswanathan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add modern report generation for CI processes Add modern CI reporting with GitHub Job Summary Feb 17, 2026
@SureshKViswanathan
Copy link
Copy Markdown
Contributor

fix the build issue @copilot

…d build failure

Co-authored-by: SureshKViswanathan <257696045+SureshKViswanathan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Feb 17, 2026

fix the build issue @copilot

Fixed in 49c5f1c. The issue was that grep -c outputs 0 to stdout (exit code 1) when no matches are found, and || echo "0" appended a second 0, producing "0\n0" which caused an arithmetic syntax error in TOTAL=$((PASSED + FAILED + SKIPPED)). Changed to || true which suppresses the exit code without adding extra output.

Copilot AI and others added 2 commits February 18, 2026 09:07
Add PerformanceRegressionTests.swift with 10 test methods covering:
- V2/V3/FHIR parsing throughput baselines
- Memory profiling for large message volumes
- Concurrent parsing scalability (2/4/8 tasks)
- MLLP framing/deframing throughput
- Object pool reuse rate regression checks
- Streaming bundle performance
- Latency percentile baselines (p50/p95/p99)
- Comprehensive regression comparison with BenchmarkRunner

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e milestones

Co-authored-by: SureshKViswanathan <257696045+SureshKViswanathan@users.noreply.github.com>
Copilot AI changed the title Add modern CI reporting with GitHub Job Summary Add modern CI reporting and Phase 7.7 performance regression tests Feb 18, 2026
Co-authored-by: SureshKViswanathan <257696045+SureshKViswanathan@users.noreply.github.com>
Copilot AI changed the title Add modern CI reporting and Phase 7.7 performance regression tests Add modern CI reporting, Phase 7.7 perf regression tests, and CLI --regression command Feb 18, 2026
@SureshKViswanathan SureshKViswanathan marked this pull request as ready for review February 18, 2026 14:22
@SureshKViswanathan SureshKViswanathan merged commit e67f456 into main Feb 18, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants