Greygor

Pre-collapse detection for file systems

Greygor is a production-focused pre-collapse detection service for file system anomalies (ransomware, corruption, wiping, etc.). The repo includes a pure-stdlib simulator, detector, tests, and benchmarks to exercise the detector end-to-end. main

Greygor is a Python library for detecting file system anomalies including ransomware attacks, storage degradation, and data corruption. It uses signal-based detection to identify unusual patterns before catastrophic data loss occurs.

Features

=======

Production readiness snapshot

Greygor now ships with production guardrails: alerting hooks (Prometheus, syslog, email, webhook), health monitoring, tuned profiles (high-security, balanced, storage, database), runtime-safe config reloads, and preflight validation with observability counters. Detection/false-positive targets are documented in the benchmarks and exercised by the test suite. Pair deployments with the runbooks below to keep the service SLA-ready.

=======

Quick start

main

• Collapse-agnostic detection: Works across ransomware, storage failures, and database corruption
• Multiple signal analysis: Entropy, compression, mutation rate, timing patterns
• Production-ready: Comprehensive error handling, logging, and configuration
• High accuracy: >95% detection rate with <2% false-positive rate
• Performant: Handles 100+ events/sec with low latency
• Extensively tested: 90%+ code coverage with comprehensive test suite

Installation

pip install greygor

Quick Start

from greygor.detector import GreygorDetector, DetectorConfig, FileEvent

# Initialize detector with baseline files
detector = GreygorDetector(["/path/to/monitored/files"])

# Process file events
event = FileEvent(
    path="/path/to/file.txt",
    timestamp=time.time(),
    before_bytes=b"original content",
    after_bytes=b"modified content",
    ext_before=".txt",
    ext_after=".txt",
)

detector.update(event)

# Check for alerts
if detector.should_alert():
    print("Anomaly detected!")
    print(f"Snapshot: {detector.last_snapshot}")

Documentation

• EXAMPLES.md: Usage examples and scenarios
• TESTING.md: Comprehensive testing and validation guide
• Greygor Theory Paper: Theoretical foundation

Testing and Validation

Greygor includes a comprehensive test suite to ensure reliability:

Quick Validation

# Run basic functionality test
python test_basic_functionality.py

# Run unit tests
python scripts/run_all_tests.py --suite unit

# Run all tests
python scripts/run_all_tests.py

Comprehensive Testing

# Phase 1: Unit Tests (90%+ coverage)
python -m unittest tests.test_signals_comprehensive
python -m unittest tests.test_detector_comprehensive

# Phase 2: Integration Tests
python -m unittest tests.test_integration

# Phase 3: Stress Tests
python -m unittest tests.test_stress

# Phase 4: Benchmarks
python scripts/run_benchmarks.py

Coverage Reports

# Generate coverage report
python scripts/generate_coverage.py

# Generate HTML coverage report
python scripts/generate_coverage.py --html

Performance Benchmarks

# Run all scenario benchmarks
python scripts/run_benchmarks.py

# Quick benchmark (3 runs)
python scripts/run_benchmarks.py --quick

# Comprehensive (20 runs)
python scripts/run_benchmarks.py --runs 20

test/detector-validation-stress-suite Benchmark Scenarios:

• Ransomware simulation (burst mode, high mutation)
• Storage degradation (slow mode, data corruption)
• Database corruption (transaction log bloat)
• Benign workload (false-positive testing)

Expected Results:

• Detection accuracy: >95% on collapse scenarios
• False-positive rate: <2% on benign workloads
• Throughput: 200-500 events/sec
• Latency: <10ms average, <50ms P99

Configuration

from greygor.detector import DetectorConfig

# High-sensitivity configuration
config = DetectorConfig(
    window_size=10,
    min_events=10,
    min_signal_count=2,
    entropy_drift_max=0.2,
    mutation_rate_max=0.4,
    compression_delta_min=-0.3,
)

detector = GreygorDetector(initial_paths, config=config)

See configs/ directory for scenario-specific configurations:

• ransomware_detector.json: High-sensitivity ransomware detection
• storage_detector.json: Storage degradation monitoring
• database_detector.json: Database corruption detection

Examples

See examples/ directory for complete examples:

• error_handling_example.py: Production error handling patterns
• monitor_example.py: Filesystem monitoring integration

Development

Project Structure

======= Run the filesystem monitor with production-style defaults (configurable via JSON/YAML):

python -m greygor.monitor --paths /data,/logs --config-file configs/ransomware_detector.json --monitor-config configs/monitor_production.yaml --log-config configs/logging_production.json --health-file reports/monitor_health.json

codex/gather-feedback-on-project-c7r7zw By default the CLI preflights all monitored paths for readability/existence before starting; use --skip-preflight only for controlled environments where the check is handled elsewhere.

======= main Write JSON reports: main

greygor/
├── detector.py       # Core detection logic
├── signals.py        # Signal computation functions
├── simulate.py       # Simulation helpers
├── monitor.py        # Filesystem monitoring
├── prevention.py     # Prevention mechanisms
└── config.py         # Configuration management

tests/
├── test_signals_comprehensive.py    # Signal unit tests
├── test_detector_comprehensive.py   # Detector unit tests
├── test_integration.py               # Integration tests
└── test_stress.py                    # Stress/performance tests

benchmarks/
└── benchmark_scenarios.py            # Scenario benchmarks

scripts/
├── run_all_tests.py                  # Test runner
├── run_benchmarks.py                 # Benchmark runner
└── generate_coverage.py              # Coverage report

Running Tests

# All tests
python scripts/run_all_tests.py

# Specific suite
python scripts/run_all_tests.py --suite unit
python scripts/run_all_tests.py --suite integration
python scripts/run_all_tests.py --suite stress

# With coverage
python scripts/generate_coverage.py

Contributing

When contributing:

1. Write tests for new features
2. Ensure 90%+ code coverage
3. Run full test suite
4. Update benchmarks if needed
5. Update documentation

License

MIT License

Citation

If you use Greygor in research, please cite:

Greygor: A Theory of Pre-Collapse Detection in Symbolic File Systems

Support

• GitHub Issues: [Report bugs or request features]
• Documentation: See TESTING.md and EXAMPLES.md
• Theory: See Greygor theory paper

Acknowledgments

Greygor implements collapse-agnostic detection theory for file systems, providing early warning before catastrophic data loss.