refactor(gtfs): clean up dead cache code and add singleflight optimiz…#647
refactor(gtfs): clean up dead cache code and add singleflight optimiz…#6473rabiii wants to merge 1 commit intoOneBusAway:mainfrom
Conversation
…ation This addresses the code review feedback from the previous performance PR: - Removed dead code: `global_cache.go`, `global_cache_test.go`, and `SetContextCache`. - Updated stale test comments that incorrectly referenced warmed caches. - Retained `SetShapeCache` with updated documentation explaining its use by `DirectionPrecomputer` during startup. - Documented the lifecycle bounds of the `directionResults` `sync.Map`. - Implemented `singleflight` in `CalculateStopDirection` to prevent duplicate database hits during concurrent initial cache misses.
aaronbrethorst
left a comment
There was a problem hiding this comment.
Hey Adel, the direction of this PR is exactly right — removing the dead global_cache.go code, cleaning up stale comments, and adding singleflight to deduplicate concurrent DB hits is a meaningful improvement. There are two items to address before we can merge.
Critical
1. Singleflight captures the HTTP request's ctx, which can poison the permanent cache — advanced_direction_calculator.go:98
The singleflight callback captures ctx from the calling goroutine's HTTP request. If that request is cancelled (client disconnect, timeout), the DB query inside computeFromShapes fails and returns "". That empty string is then permanently stored in directionResults (line 108) and served to every future request for that stop — for the lifetime of the application.
The scenario:
- Request A enters the singleflight for stopID "123" with
ctx_A - Requests B, C coalesce behind it
ctx_Ais cancelled (client disconnects)computeFromShapes(ctx_A, ...)fails → returns""""is cached permanently indirectionResults- All future requests for "123" return
""from cache
Fix: detach the context so the shared computation isn't tied to a single request's lifecycle:
v, _, _ := adc.requestGroup.Do(stopID, func() (interface{}, error) {
if cached, ok := adc.directionResults.Load(stopID); ok {
return cached.(string), nil
}
computedDir := adc.computeFromShapes(context.WithoutCancel(ctx), stopID)
adc.directionResults.Store(stopID, computedDir)
return computedDir, nil
})context.WithoutCancel (Go 1.21+, and you're on 1.24) preserves any values in the context while removing the cancellation signal. Alternatively, context.Background() works if there are no context values needed downstream.
Important
1. contextCache field is now dead code — advanced_direction_calculator.go:29, 167-169
The PR removed SetContextCache and deleted InitializeGlobalCache (the only code that ever set contextCache), but the contextCache field still exists on the struct and is still read in computeFromShapes:
hasCache := adc.contextCache != nil // always false
if hasCache {
stopTrips = adc.contextCache[stopID] // unreachable
}Since NewAdvancedDirectionCalculator doesn't initialize it and no remaining code sets it, contextCache is always nil. Please remove:
- The
contextCachefield from the struct (line 29) - The dead cache-check block in
computeFromShapes(lines 166-171)
This is exactly the kind of cleanup this PR is meant to accomplish — leaving it behind would be ironic!
Thanks again, and I look forward to merging this change.
This follow-up PR addresses the outstanding review comments from the direction calculator performance PR #633
Changes:
global_cache.goandSetContextCache. KeptSetShapeCachebut updated the comments to explicitly state it's retained exclusively forDirectionPrecomputerstartup optimization.// Optimization: Reuse shared DB and Cachecomments in tests to reflect the new reality.directionResults(sync.Map) explaining why unbounded growth is acceptable (bounded by the finite number of real-world stops).golang.org/x/sync/singleflighttoCalculateStopDirection. Now, if 100 concurrent requests hit a cache miss for the exact same stop simultaneously, only one goroutine will query the DB and compute the shape, while the other 99 wait and share the result.@aaronbrethorst