refactor: extract withTransaction helper for GTFS bulk imports#702
refactor: extract withTransaction helper for GTFS bulk imports#702tejasva-vardhan wants to merge 3 commits intoOneBusAway:mainfrom
Conversation
|
Hi @aaronbrethorst , I noticed the OpenAPI conformance failures. It seems the schema in openapi.yml is out of sync with the current models. I'll wait for #700 to be merged, then I'll merge main into this branch to fix the test failures. My transaction helper changes are ready to go otherwise. |
aaronbrethorst
left a comment
There was a problem hiding this comment.
Hey Tejasva, the withTransaction helper is a well-crafted abstraction — it correctly handles both the caller-owned and self-managed transaction cases, and the mechanical refactoring across all 7 functions is faithful. Nice work eliminating that boilerplate.
A few things to address before this can merge.
Critical Issues
1. The Stops field addition needs its own PR
This PR bundles two unrelated changes: the transaction helper refactor (mechanical, low-risk) and the addition of a Stops []string field to ScheduleForRouteEntry + handler.
These should be separate PRs for a couple of reasons:
- They have different risk profiles and review concerns
- The
Stopsfield work deserves its own discussion — PR #700 just removedstopsandtripsfrom the OpenAPI spec to align it with the current code. This PR is now re-addingstops(as[]stringinstead of[]Stop), which puts the response back out of alignment with the spec. If the intent is to add stops support to the endpoint, that should go through as a deliberate feature PR with the spec updated to match.
Recommendation: Remove the Stops changes from this PR (revert the model and handler changes) and submit the transaction refactor on its own. If you want to add stops to schedule-for-route, open a follow-up PR that also updates the OpenAPI spec.
Important Issues
2. db.Begin() → db.BeginTx(ctx, nil) is a behavioral change
Six of the seven refactored functions previously used db.Begin() (context-unaware). The new withTransaction uses c.DB.BeginTx(ctx, nil), which will cancel the transaction if the context is cancelled. For long-running bulk imports (shapes can be 10k+ rows), a context cancellation could now abort work that previously would have completed.
This is likely an improvement, and your PR description mentions it, but it should be called out explicitly as a deliberate behavioral change, not just a standardization note.
3. Wrap errors in withTransaction for debuggability
Since you already have the label parameter, wrap the BeginTx and Commit errors:
newTx, err := c.DB.BeginTx(ctx, nil)
if err != nil {
return fmt.Errorf("failed to begin transaction for %s: %w", label, err)
}
// ...
if err := newTx.Commit(); err != nil {
return fmt.Errorf("failed to commit transaction for %s: %w", label, err)
}
return nilSmall change, big payoff when debugging import failures across 7 callers.
4. bulkInsertCalendarDates lost its logging
Every other refactored function retains LogOperation calls for "inserting_X" and "X_inserted". bulkInsertCalendarDates had its logger removed entirely — it's now the only bulk insert function with zero operational logging. Add the start/end log messages for consistency.
Suggestions
5. Document the tx != nil contract
The godoc says "does not commit" for non-nil tx, but doesn't mention rollback responsibility. Add: "When tx is non-nil, the caller is responsible for committing or rolling back the transaction on error."
Strengths
- The
withTransactionhelper is well-designed — the nil/non-nil tx branching is clean and correct defer SafeRollbackWithLoggingcorrectly handles the post-commit case (it checks for "already committed" and silently returns)- The mechanical refactoring is faithful — insert logic within each function is preserved exactly
- Good PR description with clear impact summary
Recommended Action: Request Changes
- Remove the
Stopsfield changes — submit as a separate PR with OpenAPI spec alignment - Wrap errors in
withTransaction(BeginTx + Commit) using thelabelparam - Restore logging to
bulkInsertCalendarDates - Document the
tx != nilrollback contract in the godoc - Rebase onto main (to pick up #700) and confirm
make test+make lintpass
Summary
This PR extracts a centralized
withTransactionhelper for GTFS bulk operations to eliminate boilerplate and ensure consistent transaction handling across the import process. This is a follow-up to the review suggestions in #653.Changes
Client.withTransactioninhelpers.go. This helper handles transaction checking, context-aware creation, deferred rollbacks, and commits.bulkInsertStopsbulkInsertTripsbulkInsertStopTimesbulkInsertShapesbulkInsertFrequenciesbulkInsertCalendarDatesbuildBlockTripIndexImpact
c.DB.BeginTx(ctx, nil), ensuring context cancellation is properly respected across all bulk inserts.Checklist
withTransactionhelperBeginTx(ctx, nil)make test)make lint)