Skip to content

test(cli): expand offline e2e coverage for service integrity and lifecycle#47

Open
Ethan-Dunia wants to merge 2 commits intomainfrom
test/expand-cli-e2e-coverage
Open

test(cli): expand offline e2e coverage for service integrity and lifecycle#47
Ethan-Dunia wants to merge 2 commits intomainfrom
test/expand-cli-e2e-coverage

Conversation

@Ethan-Dunia
Copy link
Copy Markdown

Summary

  • expand the default CLI e2e suite with offline coverage for service validate/sign/verify
  • add tamper-detection coverage for lockfile-backed services
  • add command-service lifecycle coverage, including disabled/enabled transitions and status guidance
  • add command-level regression tests for the same service integrity flows

Why

The existing cmd/kimbap/e2e_test.go suite covered a few golden-path CLI flows, but it did not exercise the newer v0.2.x service integrity surface or stateful service lifecycle transitions.

This PR keeps the default e2e suite focused on deterministic, offline CLI behavior and closes the most obvious coverage gaps without introducing GUI/TCC-dependent instability.

Scope

Included:

  • default CLI e2e coverage in cmd/kimbap/e2e_test.go
  • command-layer regression coverage in cmd/kimbap/service_lifecycle_test.go

Out of scope:

  • darwin && integration AppleScript GUI tests
  • CI workflow changes for opt-in macOS integration runs

I intentionally left the GUI AppleScript integration suite out of this PR. That path has different ergonomics and reliability constraints (TCC prompts, app launch timing, local app state) and should be proposed separately as an opt-in integration-testing change.

Validation

  • go test ./cmd/kimbap -run '^TestE2E' -count=1 (2 fresh rounds after the final e2e additions)
  • go test ./cmd/kimbap -count=1
  • go test ./...

Notes

I first attempted to push via the local work SSH key (~/.ssh/id_ed25520, the kompas.ai key), but both github.com:22 and ssh.github.com:443 were closed by the remote/network path on this machine. The branch and PR were therefore created with the active GitHub CLI account Ethan-Dunia, which is the company account.

WangYixiao and others added 2 commits April 13, 2026 11:59
Replace brittle text substring checks ("1 enabled", "0 stored",
"Run 'kimbap link github' to connect.") with a structured assertion
against the status --format json contract, and keep only the CLI
command fragment "kimbap link github" as the footer check. The JSON
schema is stable across copy changes; the command fragment is CLI
syntax rather than prose.
@Ethan-Dunia
Copy link
Copy Markdown
Author

Guarded commits / contracts

Mapping each added test to the concrete recent fix or contract it locks in, so future regressions in these paths surface as CI failures rather than user bug reports.

Added test What it guards
TestE2EServiceValidateSignVerifyFlow service validateinstallverifysignverify round-trip; the verified / locked / signed / signature_valid fields on service verify --format json
TestE2EServiceVerifyDetectsTampering Manifest-tamper detection via installed-file modification; stderr signal and the verified=false / locked=true JSON payload
TestE2ECommandServiceCall End-to-end command-adapter call path (indirect regression guard for 94f0961 fix(command): refresh allowed executables from installed services)
TestE2EServiceEnableDisableLifecycle install --no-activateenabledisable flow; enabled count reported by service list / status; disabled-service gating on call
TestE2EStatusShowsLinkHintForDisconnectedService 58852f2 fix(status): show actionable link commands for disconnected services
TestServiceValidateCommandPrintsInstallHintInTextMode Text-mode install hint after service validate
TestServiceSignAndVerifyCommandsWithPinnedKey Sign + pinned-key verify round-trip at unit level
TestServiceVerifyCommandReturnsJSONFailureForTamperedManifest Unit-level tamper detection, including the differing expected_digest / actual_digest fields

Follow-up (intentionally out of scope)

  • Approval-flow e2e — natural follow-up to fa16989 fix(api): return resumed execution status from approval resolution
  • callaudit.jsonl persistence e2e — better paired with future audit-surface work
  • Exit-code contract table e2e — orthogonal, should be its own PR
  • serve + HTTP API smoke e2e — larger surface, warrants its own PR

Follow-up commit on this branch

  • 987239d test(cli): assert status counts via json in link hint e2e — reduces English-copy coupling in TestE2EStatusShowsLinkHintForDisconnectedService by asserting services / credentials counts via the status --format json schema and keeping only the CLI command fragment kimbap link github as the footer substring check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant