Add comprehensive unit tests with Swift Testing framework

Package.swift

@@ -29,6 +29,7 @@ let package = Package(
             ]),
         .testTarget(
             name: "gtfs-importerTests",
-            dependencies: ["gtfs-importer", "GTFSModel"]),
+            dependencies: ["gtfs-importer", "GTFSModel"],
+            resources: [.copy("testData")]),
     ]
 )

TESTING_PLAN_AND_PROGRESS.md

@@ -0,0 +1,385 @@
+# Comprehensive Unit Testing Plan & Progress for GTFSImporter & GTFSModel
+
+**Created:** 2025-12-21
+**Last Updated:** 2025-12-22
+**Status:** Phase 1 COMPLETE ✅ | Phase 2 IN PROGRESS (Utility Tests Complete)
+
+---
+
+## Table of Contents
+1. [Summary](#summary)
+2. [Phase 1: GTFSModel Testing - COMPLETE ✅](#phase-1-gtfsmodel-testing---complete-)
+3. [Phase 2: GTFSImporter Testing - IN PROGRESS](#phase-2-gtfsimporter-testing---in-progress)
+4. [Detailed Implementation Plan](#detailed-implementation-plan)
+5. [Next Steps](#next-steps)
+
+---
+
+## Summary
+
+### Overall Progress: 50% Complete
+
+**Phase 1 (GTFSModel):** ✅ **COMPLETE**
+- 61 tests across 14 suites - ALL PASSING
+- 91.44% code coverage (exceeds 90% target)
+- PR created: https://github.com/jogi/GTFSModel/pull/3
+
+**Phase 2 (GTFSImporter):** 🔄 **IN PROGRESS**
+- 12 utility tests - ALL PASSING
+- Test infrastructure complete
+- ~80 importing/integration tests remaining
+
+---
+
+## Phase 1: GTFSModel Testing - COMPLETE ✅
+
+### Final Results
+
+**Pull Request:** https://github.com/jogi/GTFSModel/pull/3
+**Branch:** `add-swift-testing-framework`
+**Target:** `main` branch
+
+**Test Stats:**
+- ✅ **61 tests** in **14 test suites** - ALL PASSING
+- ✅ **91.44% code coverage** (exceeds 90% target)
+- ✅ **100% coverage** on DateFormatter.swift, Database.swift, Agency.swift
+
+### Test Suites Created
+
+```
+Tests/GTFSModelTests/
+├── TestUtilities/
+│   ├── DatabaseTestHelper.swift      # 5 utility methods
+│   └── TestFixtures.swift            # 11 factory methods
+├── ModelTests/
+│   ├── AgencyTests.swift             # 9 tests
+│   ├── CalendarTests.swift           # 2 tests
+│   ├── CalendarDateTests.swift       # 3 tests
+│   ├── DirectionTests.swift          # 2 tests
+│   ├── FareAttributeTests.swift      # 2 tests
+│   ├── FareRuleTests.swift           # 2 tests
+│   ├── RouteTests.swift              # 5 tests
+│   ├── ShapeTests.swift              # 2 tests
+│   ├── StopTests.swift               # 5 tests
+│   ├── StopTimeTests.swift           # 3 tests
+│   └── TripTests.swift               # 4 tests
+├── IntegrationTests/
+│   └── DatabaseIntegrationTests.swift # 3 tests
+├── DateFormatterTests.swift          # 14 tests
+└── DatabaseHelperTests.swift         # 3 tests
+```
+
+### Coverage Report
+
+```
+Filename                          Lines Coverage
+----------------------------------------------------
+Models/Agency.swift              32    100.00%
+Models/Calendar.swift            42     92.86%
+Models/CalendarDate.swift        29     89.66%
+Models/Direction.swift           28     85.71%
+Models/FareAttribute.swift       31     90.32%
+Models/FareRule.swift            27     85.19%
+Models/Route.swift               44     88.64%
+Models/Shape.swift               26     88.46%
+Models/Stop.swift                53     92.45%
+Models/StopTime.swift            48     93.75%
+Models/Trip.swift                38     86.84%
+Utilities/Database.swift         10    100.00%
+Utilities/DateFormatter.swift    24    100.00%
+----------------------------------------------------
+TOTAL                           432     91.44%
+```
+
+### Key Achievements
+
+- ✅ Migrated from XCTest to Swift Testing framework
+- ✅ Zero mocking - all tests use real database operations
+- ✅ Proper foreign key constraint handling
+- ✅ Swift 6 concurrency compatible (removed async/await)
+- ✅ Parameterized tests where applicable
+- ✅ Comprehensive test coverage of all 11 GTFS models
+
+---
+
+## Phase 2: GTFSImporter Testing - IN PROGRESS
+
+### Current Status: Utility Tests Complete ✅
+
+**Tests Completed:** 12 tests in 3 suites - ALL PASSING
+**Remaining:** ~80 importing and integration tests
+
+### Test Infrastructure Created ✅
+
+```
+Tests/gtfs-importerTests/
+├── TestUtilities/
+│   ├── DatabaseTestHelper.swift      # Database test utilities
+│   ├── TestDataHelper.swift          # CSV and GTFS dataset helpers
+│   └── TemporaryFileHelper.swift     # Temp file/directory management
+└── UtilityTests/
+    ├── StringExtensionTests.swift    # 5 tests ✅
+    ├── ConsoleTests.swift            # 3 tests ✅
+    └── StopTimeInterpolatorTests.swift # 4 tests ✅
+```
+
+### Completed Tests (12 total)
+
+#### 1. StringExtensionTests (5 tests) ✅
+Tests for time string sanitization (`sanitizedTimeString` extension):
+- ✅ Times < 24:00:00 preserve hour (with leading zero removed)
+  - "00:00:00" → "0:00:00"
+  - "08:15:30" → "8:15:30"
+  - "23:59:59" → "23:59:59"
+- ✅ Times >= 24:00:00 subtract 24 from hour
+  - "24:00:00" → "0:00:00"
+  - "25:30:15" → "1:30:15"
+  - "27:15:30" → "3:15:30"
+- ✅ Edge case: "48:00:00" → "24:00:00"
+- ✅ Minutes and seconds preserved
+
+#### 2. ConsoleTests (3 tests) ✅
+Tests for ANSI color console utilities:
+- ✅ ANSI color codes are correct
+- ✅ Color wrapping produces expected format
+- ✅ String extension properties match Console methods
+
+#### 3. StopTimeInterpolatorTests (4 tests) ✅
+Tests for time interpolation algorithm:
+- ✅ Interpolation fills missing times based on distance (Haversine formula)
+- ✅ Interpolation preserves existing times
+- ✅ Interpolation handles overnight times (>= 24:00:00)
+- ✅ Interpolation handles empty trips gracefully
+
+### Deleted XCTest Artifacts ✅
+
+- ❌ `Tests/gtfs-importerTests/gtfs_importerTests.swift` (removed)
+- ❌ `Tests/gtfs-importerTests/XCTestManifests.swift` (removed)
+- ❌ `Tests/LinuxMain.swift` (removed)
+
+---
+
+## Detailed Implementation Plan
+
+### Remaining Work: Phase 2 (GTFSImporter)
+
+#### Step 1: Importing Tests (~66 tests estimated)
+
+**A. Import Orchestrator Tests** (`ImportingTests/ImporterTests.swift`)
+- [ ] Test `importAllFiles()` orchestration (~5 tests)
+- [ ] Test import order (dependencies first)
+- [ ] Test error in one file doesn't stop others
+- [ ] Test missing optional files skipped gracefully
+- [ ] Test missing required files throw error
+
+**B. Entity Import Tests** (11 files, ~6 tests each = ~66 tests)
+
+Create directory: `Tests/gtfs-importerTests/ImportingTests/`
+
+Simple importers (use default behavior):
+- [ ] `AgencyImportingTests.swift`
+- [ ] `StopImportingTests.swift`
+- [ ] `ShapeImportingTests.swift`
+- [ ] `FareAttributeImportingTests.swift`
+- [ ] `FareRuleImportingTests.swift`
+- [ ] `DirectionImportingTests.swift`
+
+Complex importers (with custom logic):
+- [ ] `RouteImportingTests.swift` - Test default color/sortOrder/continuousPickup values
+- [ ] `TripImportingTests.swift` - Test default wheelchair/bikes values
+- [ ] `CalendarImportingTests.swift` - Test date parsing (yyyyMMdd format)
+- [ ] `CalendarDatesImportingTests.swift` - Test date parsing and exception types
+- [ ] `StopTimeImportingTests.swift` - Test time parsing, overnight times, updateLastStop()
+
+**Pattern for each test:**
+```swift
+@Suite("EntityName Importing Tests")
+struct EntityNameImportingTests {
+    @Test("fileName returns correct CSV file name")
+    func testFileName() { ... }
+
+    @Test("Import applies default values")
+    func testDefaultValues() throws { ... }
+
+    @Test("Import handles required fields")
+    func testRequiredFields() throws { ... }
+
+    @Test("Import handles optional fields")
+    func testOptionalFields() throws { ... }
+
+    @Test("Import handles invalid data gracefully")
+    func testInvalidData() throws { ... }
+}
+```
+
+#### Step 2: Integration Tests (~20 tests estimated)
+
+Create directory: `Tests/gtfs-importerTests/IntegrationTests/`
+
+**A. StopRouteTests.swift** (~4 tests)
+- [ ] Test `addStopRoutes()` populates routes field
+- [ ] Test multiple routes per stop (comma-separated)
+- [ ] Test stops with no routes (NULL)
+- [ ] Test route deduplication
+
+**B. EndToEndImportTests.swift** (~10 tests)
+- [ ] Test complete import workflow with minimal dataset
+- [ ] Test complete import workflow with full VTA data
+- [ ] Verify record counts match CSV line counts
+- [ ] Verify database schema matches expected structure
+- [ ] Verify all indexes created
+- [ ] Verify foreign key relationships work
+- [ ] Test vacuum functionality
+- [ ] Test reindex functionality
+- [ ] Test interpolation with full dataset
+- [ ] Test stop-routes with full dataset
+
+**C. PerformanceTests.swift** (~4 tests)
+- [ ] Test stop_times.txt import (428K records) completes in reasonable time
+- [ ] Test full import completes within expected time
+- [ ] Test vacuum completes within 2 seconds
+- [ ] Test interpolation completes within 5 seconds
+
+**Test data locations:**
+- Minimal: Generated via `TestDataHelper.createMinimalGTFSDataset()`
+- Full VTA: `/Users/jogi/Developer/San Jose Transit/GTFSImporter/Tests/testData/`
+
+#### Step 3: Final Steps
+- [ ] Run all GTFSImporter tests and fix failures
+- [ ] Check code coverage (target: ≥90%)
+- [ ] Create branch `add-comprehensive-unit-tests`
+- [ ] Commit with detailed message
+- [ ] Push to GitHub
+- [ ] Create PR against `develop` branch
+
+---
+
+## Next Steps
+
+### Immediate Next Session
+
+1. **Create Importing Tests Directory**
+   ```bash
+   mkdir -p Tests/gtfs-importerTests/ImportingTests
+   mkdir -p Tests/gtfs-importerTests/IntegrationTests
+   ```
+
+2. **Start with Simple Entity Importing Tests**
+   - Begin with `AgencyImportingTests.swift` as template
+   - Test pattern: fileName, default values, required/optional fields
+   - Apply pattern to other simple entities
+
+3. **Implement Complex Entity Importing Tests**
+   - Focus on custom logic in Route, Trip, Calendar, StopTime importers
+   - Test time sanitization integration
+   - Test default value injection
+
+4. **Integration Tests**
+   - StopRouteTests with minimal data
+   - EndToEndImportTests with both minimal and full VTA data
+   - PerformanceTests with full VTA dataset
+
+5. **Finalize**
+   - Run full test suite
+   - Check coverage
+   - Create PR
+
+### Estimated Remaining Work
+
+- **Importing Tests:** ~3-4 hours (66 tests)
+- **Integration Tests:** ~2-3 hours (20 tests)
+- **Testing & Coverage:** ~1 hour
+- **Total:** ~6-8 hours
+
+### Files to Reference
+
+**Key Source Files:**
+- `/Users/jogi/Developer/San Jose Transit/GTFSImporter/Sources/gtfs-importer/Importing/Importer.swift`
+- `/Users/jogi/Developer/San Jose Transit/GTFSImporter/Sources/gtfs-importer/Importing/*.swift` (11 entity importers)
+- `/Users/jogi/Developer/San Jose Transit/GTFSImporter/Sources/gtfs-importer/Importing/StopRoute.swift`
+
+**Test Data:**
+- Full VTA: `/Users/jogi/Developer/San Jose Transit/GTFSImporter/Tests/testData/`
+- Helper: `TestDataHelper.createMinimalGTFSDataset()` for minimal fixtures
+
+**Documentation:**
+- Original plan: `/Users/jogi/.claude/plans/sleepy-finding-sphinx.md`
+- Project docs: `/Users/jogi/Developer/San Jose Transit/GTFSImporter/CLAUDE.md`
+
+---
+
+## Key Technical Details
+
+### Swift Testing Patterns Used
+
+**Parameterized Tests:**
+```swift
+@Test("Test description", arguments: [
+    ("input1", "expected1"),
+    ("input2", "expected2"),
+])
+func testParameterized(input: String, expected: String) {
+    #expect(input.someMethod() == expected)
+}
+```
+
+**Database Test Pattern:**
+```swift
+@Test("Test description")
+func testDatabaseOperation() throws {
+    let db = try DatabaseTestHelper.createTemporaryDatabase()
+    defer { try? DatabaseTestHelper.cleanup(database: db) }
+
+    try db.write { db in
+        // Create tables
+        // Insert test data
+        // Perform operation
+    }
+
+    // Verify results
+    let result = try db.read { db in
+        // Query database
+    }
+    #expect(result == expected)
+}
+```
+
+### Critical Patterns to Follow
+
+1. **No async/await** - Use synchronous GRDB operations (Swift 6 concurrency)
+2. **Use defer for cleanup** - No setUp/tearDown methods
+3. **Use #expect()** - Not XCTAssert*
+4. **Foreign key dependencies** - Insert parent records before children
+5. **Required NOT NULL fields** - Provide all required values in INSERT statements
+
+### Common Pitfalls Encountered
+
+1. **Swift 6 Concurrency:** Avoid `async throws` on test methods
+2. **Calendar Namespace:** Use `GTFSModel.Calendar` to avoid Foundation conflict
+3. **Parameterized Enums:** Can't use enums in parameterized tests (not Sendable)
+4. **Foreign Keys:** Must insert dependent records (routes, calendars) before trips/stop_times
+5. **NOT NULL Constraints:** Some fields like `location_type`, `wheelchair_boarding` required
+
+---
+
+## Success Criteria
+
+### Phase 1 (GTFSModel) - COMPLETE ✅
+- ✅ All tests pass: `swift test`
+- ✅ Code coverage ≥ 90%
+- ✅ No XCTest dependencies
+- ✅ PR created against `main` branch
+
+### Phase 2 (GTFSImporter) - IN PROGRESS
+- [ ] All tests pass: `swift test`
+- [ ] Code coverage ≥ 90%
+- [ ] No XCTest dependencies
+- [ ] XCTest artifacts deleted (✅ Done)
+- [ ] Integration tests pass with full VTA data
+- [ ] Performance benchmarks documented
+- [ ] PR created against `develop` branch
+
+---
+
+**Last Updated:** 2025-12-22
+**Next Session:** Continue with importing tests (Step 1B - Entity Import Tests)