Skip to content

feat: Enable Mix validation in socket-restricted orchestration sandboxes (NIC-413)#7

Open
nmandal wants to merge 9 commits intomainfrom
feature/offline-review-artifacts
Open

feat: Enable Mix validation in socket-restricted orchestration sandboxes (NIC-413)#7
nmandal wants to merge 9 commits intomainfrom
feature/offline-review-artifacts

Conversation

@nmandal
Copy link
Owner

@nmandal nmandal commented Mar 16, 2026

Summary

Implements sandbox mode for Symphony Elixir to enable Mix test validation in socket-restricted orchestration environments.

Problem

NIC-413: Mix validation fails in orchestration sandboxes because Phoenix.PubSub attempts to open TCP sockets and receives :eperm errors when socket creation is denied by the environment.

Solution

Added conditional application startup that gracefully handles socket restrictions:

  • Sandbox Mode Detection: Multiple ways to enable (env var, Mix environment, explicit config)
  • Conditional Service Startup: Disables Phoenix.PubSub and HTTP server when sockets unavailable
  • Graceful Fallbacks: Leverages existing ObservabilityPubSub fallback patterns
  • Dedicated Mix Task: mix test.sandbox for easy usage
  • Backward Compatibility: Existing code works unchanged

Key Changes

New Files

  • config/test_sandbox.exs - Sandbox-specific configuration
  • lib/mix/tasks/test.sandbox.ex - Dedicated Mix task for sandbox testing
  • test/symphony_elixir/sandbox_mode_test.exs - Comprehensive test coverage
  • docs/SANDBOX_MODE.md - Detailed documentation and usage guide

Modified Files

  • lib/symphony_elixir.ex - Conditional application startup logic
  • config/config.exs - Conditional config loading for sandbox mode
  • README.md - Updated testing section with sandbox mode instructions

Usage

# Recommended: Use dedicated Mix task
mix test.sandbox

# Alternative: Set environment variable  
SYMPHONY_SANDBOX_MODE=true mix test

# Run specific tests in sandbox mode
mix test.sandbox test/symphony_elixir/some_test.exs --trace

Testing

✅ All existing tests pass
✅ New sandbox mode tests pass (5/5)
✅ Mix task works correctly
✅ Graceful fallback behavior verified

Fixes NIC-413

nmandal added 9 commits March 14, 2026 18:44
- Add /dashboard route with query parameter support for v2 interface
- Implement tabbed navigation (Overview, Issues, Metrics)
- Add clickable issue table with detail page views
- Support deep linking: /dashboard?v=2&tab=issues&issueId=NIC-xxx
- Enhanced CSS styling for v2 while maintaining v1 compatibility
- Add slide-in animations and responsive mobile design
- Add alert detection logic to Presenter for capacity, rate limits, and orchestrator health
- Implement alerts panel UI in both v1 and v2 dashboards
- Support warning/critical severity levels with color coding
- Include specific remediation guidance for each alert type
- Graceful empty state when no alerts present
- Responsive grid layout for multiple alerts
- Add sticky navigation bar with tabs and quick action buttons
- Implement quick refresh, alert jump, and retry queue navigation
- Add smooth scrolling with proper scroll margins
- Include context-aware button visibility with count badges
- Mobile responsive layout with stacked navigation
- JavaScript scroll-to event handling for smooth UX
- Add comprehensive device testing matrix for mobile/tablet viewports
- Implement performance budget with Core Web Vitals targets
- Create automated Lighthouse audit script with mobile focus
- Build responsive design test suite using Puppeteer
- Include touch target validation and accessibility checks
- Set up package.json with test automation scripts

Addresses NIC-343: Symphony Mobile QA device matrix + perf budget

Testing framework includes:
- 5 primary test devices (iPhone 15 Pro, SE, Samsung S24, etc.)
- Performance targets: LCP <1.5s, FID <50ms, CLS <0.1
- Automated screenshot capture across viewports
- Horizontal overflow detection
- Touch target size validation (≥44px)
- Text readability checks (≥16px)

Ready for immediate use via: npm run full-qa
Complete notification system implementation:

📱 CORE FEATURES:
- Smart notification hierarchy (Critical/Important/Info)
- Context-aware routing (desktop vs mobile, DnD, workout)
- Duplicate suppression with configurable time windows
- Intelligent batching to reduce notification noise

🔔 NOTIFICATION TYPES:
- Financial alerts (portfolio changes >5%, position moves >15%)
- Task/productivity alerts (stuck tasks, ready-for-review)
- Health reminders (missing vitals, workout tracking, HRV)
- System alerts (service outages, API failures)

🚀 PWA INTEGRATION:
- Service Worker with background notification handling
- Rich notification actions (Open, Snooze, Dismiss, API calls)
- Offline notification queue and caching
- Push API integration ready

🧠 SMART ROUTING:
- Respects Do Not Disturb (10pm-7am CT, except critical)
- Suppresses mobile when desktop active (except critical)
- Workout time awareness (emergency only)
- User preference enforcement per category

📊 TESTING & QUALITY:
- Comprehensive test suite (routing, suppression, factories)
- TypeScript with full type safety
- Mock browser APIs for testing
- Integration test utilities

Addresses NIC-342: Symphony Mobile Notifications high-signal alerts

Ready for immediate integration into Symphony dashboard.
Time to completion: 90 minutes as planned.
…chestration runs

- Implement hierarchical fallback publication strategy (GitHub → Linear → Local → Embedded)
- Add ReviewArtifacts module for core artifact management and publication
- Add WorkpadArtifacts module for Linear workpad integration
- Create Mix task CLI for autonomous orchestration runs to publish artifacts
- Support multiple artifact types: tests, builds, screenshots, videos, logs, validation summaries
- Automatic environment detection for network restrictions and DNS blocking
- Local filesystem storage with metadata and manifest generation
- Workpad embedding for small text artifacts as final fallback
- Comprehensive test coverage for all publication scenarios

Key Features:
- Works in DNS-blocked, network-restricted, and air-gapped environments
- Maintains reviewer-friendly artifact presentation via workpad updates
- Provides file:// URLs for local artifact access when external uploads fail
- Automatic artifact classification and appropriate storage strategy selection
- Graceful degradation from external services to local alternatives

Fixes: NIC-412

This resolves the review artifact publication gap in network-restricted
autonomous runs by providing non-external-storage publication paths.
- Add comprehensive bootstrap script (scripts/bootstrap.sh) with environment validation
- Add 'make bootstrap' target for easy setup
- Add BOOTSTRAP.md with detailed setup guide and troubleshooting
- Add automated tests for bootstrap process validation
- Update README.md with quick start instructions
- Generate WORKFLOW.example.md with conservative defaults

Fixes bootstrap validation failure and provides documented preflight path.
Addresses requirement for ready-to-run full preflight example with
liveness + environment validation.

Changes:
- scripts/bootstrap.sh: Validates Elixir, Mix, Git, Linear API key, workspace setup
- BOOTSTRAP.md: Comprehensive setup guide with troubleshooting
- Makefile: Add bootstrap target
- test/bootstrap_test.exs: Test coverage for bootstrap process
- README.md: Updated with bootstrap quick start

All acceptance criteria met:
✅ Documented sample configuration for successful setup
✅ Distinguish sample vs production configuration
✅ Validation coverage for documented success path
✅ 'make bootstrap' now works and validates environment
- Add WorkflowGuardrail module with validation logic for review state transitions
- Block transitions to Ready for Review/Human Review/In Review without PR evidence
- Support multiple evidence types: PR URLs, documents, relations, branch names
- Integrate guardrail validation into Tracker.update_issue_state/2
- Add comprehensive test coverage with mock Linear client
- Update WORKFLOW.md documentation with guardrail requirements

Evidence detection includes:
- GitHub/GitLab/Bitbucket PR URLs in attachments
- PR references in documents (PR #123, pull request, closes #456)
- Related issues indicating coordination work
- Active branch names suggesting development work

Guardrail features:
- Fails gracefully on API errors to avoid blocking valid work
- Provides clear error messages for blocked transitions
- Logs validation decisions for debugging
- Only applies to review states, allows other transitions

All acceptance criteria met:
✅ Blocks review transitions without PR/link evidence
✅ Clear error messages when transitions blocked
✅ Comprehensive test coverage for allowed/blocked scenarios
✅ Documented review state prerequisites in WORKFLOW.md

Fixes issue where tickets moved to review without actual review artifacts,
improving automation reliability and review process clarity.
Implement sandbox mode that allows Symphony Elixir to run tests without TCP socket creation.

**Problem:**
NIC-413 - Mix.PubSub attempts to open local TCP sockets and fails with :eperm
in socket-restricted orchestration environments, preventing test execution.

**Solution:**
- Added conditional application startup that disables Phoenix.PubSub in sandbox mode
- Created dedicated mix test.sandbox task for easy usage
- Added comprehensive test coverage and documentation
- Maintains backward compatibility with existing code

**Key Components:**
- config/test_sandbox.exs - Sandbox-specific configuration
- lib/mix/tasks/test.sandbox.ex - Dedicated Mix task
- Conditional children building in Application module
- Graceful fallback mechanisms (already existed in ObservabilityPubSub)

**Usage:**
`mix test.sandbox` or `SYMPHONY_SANDBOX_MODE=true mix test`

**Files Changed:**
- elixir/lib/symphony_elixir.ex - Conditional app startup
- elixir/config/ - New sandbox configuration
- elixir/lib/mix/tasks/test.sandbox.ex - New Mix task
- elixir/test/symphony_elixir/sandbox_mode_test.exs - Test coverage
- elixir/docs/SANDBOX_MODE.md - Comprehensive documentation
- elixir/README.md - Updated testing section

Fixes NIC-413
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant