Test: Database File Corruption and Recovery Scenarios

[nickna]

Description

Test database behavior when persistent storage is corrupted, partially written, or becomes unavailable during operations.

Phase

Phase 1: Chaos Engineering and Failure Injection

Epic

Related to #202

Acceptance Criteria

• Test recovery from corrupted database files
• Verify behavior with partial write scenarios (power loss simulation)
• Test file locking conflicts and concurrent access issues
• Validate backup/restore mechanisms
• Ensure graceful degradation when storage is unavailable

Corruption Scenarios

1. Header corruption - Invalid file version or metadata
2. Partial vector data - Truncated vector serialization
3. Index corruption - Corrupted KD-Tree or HNSW data structures
4. File locking conflicts - Multiple processes accessing same file
5. Permission denied - Read-only file system scenarios

Test Structure

[Test]
[Category("Chaos")]
public async Task Database_LoadFromCorruptedFile_HandlesGracefully()
{
    // Arrange: Create valid database file
    var validDb = new VectorDatabase();
    PopulateDatabase(validDb, vectorCount: 1000);
    var tempPath = CreateTempDatabasePath();
    await validDb.SaveAsync(tempPath);
    
    // Act: Corrupt the file
    CorruptDatabaseFile(tempPath, CorruptionType.HeaderDamage);
    
    // Assert: Loading should either recover or fail gracefully
    var corruptedDb = new VectorDatabase();
    
    if (await TryLoadAsync(corruptedDb, tempPath))
    {
        // If loaded, verify data integrity
        Assert.That(corruptedDb.Count, Is.GreaterThan(0));
        VerifyBasicOperations(corruptedDb);
    }
    else
    {
        // If failed, ensure clean error handling
        Assert.That(corruptedDb.Count, Is.EqualTo(0));
    }
}

File System Simulation

• Create IFileSystemSimulator for controlled I/O failures
• Implement partial write scenarios
• Mock permission and locking issues
• Test with different corruption patterns