-
Notifications
You must be signed in to change notification settings - Fork 9
Optimize lobby session data retrieval by caching at initialization #92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
Copilot
wants to merge
20
commits into
main
Choose a base branch
from
copilot/fix-89
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… for improved dependency management
* feat: Add file-level options and swagger definitions to proto files * Update evr.proto Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update evr.proto Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update evr_http.proto Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
#25) * Initial plan * Add comprehensive characterization tests for LoginHistory struct Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Implement focused improvements to LoginHistory struct methods Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update protobuf field names --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…oles command (#50) * Initial plan * Implement interactive role configuration interface for /set-roles command Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Add unit tests for role configuration functionality Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…hoTools Nakama (#65) * Initial plan * Complete README.md with comprehensive EchoTools Nakama deployment guide Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Update README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…treamModeLobbySessionTelemetry API (#67) * Initial plan * Implement Redis Streams Event Journaling, MongoDB Summarization, and StreamModeLobbySessionTelemetry API Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Add telemetry integration utilities and documentation Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Update server/evr_telemetry_manager.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_telemetry_integration.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_telemetry_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
… match data in MongoDB (#42) * Initial plan * Implement EventMatchDataJournal integration with Redis queue and MongoDB persistence Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Apply gofmt formatting to new files * Add demo script and finalize MatchDataJournal integration Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Update GitHub Actions workflow to reduce unnecessary builds * Enforcement Suspension System Refactor (#47) * Remove obsolete ServiceConfig * Refactor enforcement records and suspension handling - Introduced AllModes and PrivateModes constants for better mode management. - Updated GuildEnforcementRecord to use Expiry instead of SuspensionExpiry. - Enhanced CheckEnforcementSuspensions to return active suspensions by game mode. - Adjusted lobby authorization to handle game mode-specific suspensions. - Removed LimitedAccess role checks and related logic from GuildGroup. - Updated tests to verify correct behavior of group ID and user ID assignments in enforcement records. * Refactor GitHub Actions workflows to streamline build process (#48) * Refactor GitHub Actions workflows to streamline build process * Enforcement Suspension System Refactor (#47) * Remove obsolete ServiceConfig * Refactor enforcement records and suspension handling - Introduced AllModes and PrivateModes constants for better mode management. - Updated GuildEnforcementRecord to use Expiry instead of SuspensionExpiry. - Enhanced CheckEnforcementSuspensions to return active suspensions by game mode. - Adjusted lobby authorization to handle game mode-specific suspensions. - Removed LimitedAccess role checks and related logic from GuildGroup. - Updated tests to verify correct behavior of group ID and user ID assignments in enforcement records. * Update .github/workflows/build.yaml Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update build workflow: add Go module download step and change build command to 'make nakama' * Update Docker and Makefile configurations for improved build process and caching * Refactor build workflow: remove Go module download step and integrate 'go mod vendor' before building binary * Disable pull request trigger for tests workflow and update event configuration * Enforcement Suspension System Refactor (#47) * Remove obsolete ServiceConfig * Refactor enforcement records and suspension handling - Introduced AllModes and PrivateModes constants for better mode management. - Updated GuildEnforcementRecord to use Expiry instead of SuspensionExpiry. - Enhanced CheckEnforcementSuspensions to return active suspensions by game mode. - Adjusted lobby authorization to handle game mode-specific suspensions. - Removed LimitedAccess role checks and related logic from GuildGroup. - Updated tests to verify correct behavior of group ID and user ID assignments in enforcement records. --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Refactor storage system to use simplified StorableAdapter pattern (#46) * Initial plan * Create StorableAdapter pattern and convert LatencyHistory Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Convert GuildGroupState and LoginHistory to StorableAdapter pattern Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Convert DisplayNameHistory and EarlyQuitConfig to StorableAdapter pattern Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Convert remaining types to StorableAdapter pattern Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Complete Storable interface refactoring with legacy compatibility Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Replace Storable with initial proposed implementation * Remove deprecated legacy storage interfaces and functions in favor of StorableAdapter * Refactor latency history handling to remove StorableAdapter usage * Refactor storage index registration to use StorableIndexer pattern * Refactor most structs to use simplified StorableAdapter pattern Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Enforcement Suspension System Refactor (#47) * Remove obsolete ServiceConfig * Refactor enforcement records and suspension handling - Introduced AllModes and PrivateModes constants for better mode management. - Updated GuildEnforcementRecord to use Expiry instead of SuspensionExpiry. - Enhanced CheckEnforcementSuspensions to return active suspensions by game mode. - Adjusted lobby authorization to handle game mode-specific suspensions. - Removed LimitedAccess role checks and related logic from GuildGroup. - Updated tests to verify correct behavior of group ID and user ID assignments in enforcement records. * Complete StorableAdapter refactoring to simplified pattern Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Enforcement Suspension System Refactor (#47) * Remove obsolete ServiceConfig * Refactor enforcement records and suspension handling - Introduced AllModes and PrivateModes constants for better mode management. - Updated GuildEnforcementRecord to use Expiry instead of SuspensionExpiry. - Enhanced CheckEnforcementSuspensions to return active suspensions by game mode. - Adjusted lobby authorization to handle game mode-specific suspensions. - Removed LimitedAccess role checks and related logic from GuildGroup. - Updated tests to verify correct behavior of group ID and user ID assignments in enforcement records. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> * Refactor profile and display name handling (#49) * WIP: Refactor profile * Refactor profile handling to improve display name management and streamline related structures * Fix display name history caching by limiting to 15 recent names per group and filtering by age * Add kick-player full audit log message * Update interaction signature formatting and log message structure for improved clarity * Refactor storage index registration to include DisplayNameHistory and DeveloperApplications * Update server/evr_account_settings.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_account_displayname_history.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Refactor display name assignment in DisplayNameHistory and clean up unused code in ProfileOptions and LatencyHistory --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Potential crash fix for evr_discord_appbot (#54) * Add check for `GamePauseSettings` being nil. --------- Signed-off-by: Andrew Bates <a@sprock.io> * Update .gitignore to include 'nakama-debug' and remove unnecessary comments * Optimize Dockerfile by adding 'go mod tidy' and 'go mod vendor' steps for improved dependency management * Refactor EVRProfile structure and remove ProfileOptions * Update Go version to 1.25 * Remove unused pprof server setup from EvrPipeline * Update server/evr_runtime_events.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_runtime_events.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_runtime_event_matchdatajournal.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_match_data_indexing.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_runtime_event_matchdatajournal.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Avery ❤️ <avery442@proton.me>
…ng (#62) * Initial plan * Implement Discord sessions channel with automated lobby session tracking Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Complete Discord sessions channel implementation with cleanup and service guild support Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_discord_sessions_channel.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Update server/evr_discord_sessions_channel.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Initial plan * Implement Discord Linked Roles with custom requirement options Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Complete Discord Linked Roles implementation with CORS support and validation guide Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Address review feedback: move guide to docs, fix User-Agent, remove gitignore entry Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Replace redundant findUserIDByDiscordID with existing GetUserIDByDiscordID function Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io>
* Initial plan * Add comprehensive Discord bot guild setup documentation Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Andrew Bates <a@sprock.io> --------- Signed-off-by: Andrew Bates <a@sprock.io> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> Co-authored-by: Andrew Bates <a@sprock.io> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
… similar calculation corruption (#69) * Initial plan * Initial investigation of profile statistics issue Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Fix profile statistics showing large negative numbers Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> * Add comprehensive validation to prevent corrupted statistics in all calculations Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com>
Bumps the go_modules group with 1 update in the / directory: [golang.org/x/net](https://github.com/golang/net). Updates `golang.org/x/net` from 0.37.0 to 0.38.0 - [Commits](golang/net@v0.37.0...v0.38.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.38.0 dependency-type: indirect dependency-group: go_modules ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com>
…use cached data Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com>
Co-authored-by: thesprockee <962164+thesprockee@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Ensure all lobby session data is retrieved at initialization (reduce runtime DB/storage calls)
Optimize lobby session data retrieval by caching at initialization
Aug 21, 2025
3046b52 to
4a7d4cc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements comprehensive caching of lobby session data at session initialization to eliminate redundant database and storage calls during lobby operations.
Problem
The
lobbySessionRequestcode path was making multiple database/storage calls on every lobby request (find/join/create/spectate):nk.FriendsList()- loading friends list to extract blocked players (1-N paginated calls)MatchmakingRatingLoad()- loading player ratings for skill-based matchmakingCalculateSmoothedPlayerRankPercentile()- calculating rank percentiles with database queriesMatchLabelByID()- validating next match informationStreamUserList()- getting presence data for next match hostsFor users with many friends, this could result in 5-10+ database calls per lobby request, creating unnecessary load and latency.
Solution
Session-Level Caching
Extended
SessionParameterswith new cached data fields:friendsList- complete friends list loaded once at session startblockedPlayerIDs- extracted blocked player IDs for matchmaking queriesmatchmakingRatingsByGroup- ratings cached by group and game moderankPercentilesByGroup- rank percentiles cached by group and game modenextMatchInfo- resolved next match information including Discord host lookupsInitialization-Time Loading
Added three new functions called during
initializeSession():loadAndCacheFriendsData()- loads complete friends list and extracts blocked playersloadAndCacheMatchmakingData()- pre-loads ratings/percentiles for all user groupsloadAndCacheNextMatchInfo()- resolves and caches next match metadataLobby Parameter Optimization
Updated
NewLobbyParametersFromRequest()to use cached data instead of fresh database calls:Performance Impact
Data Freshness
Cached until session restart:
Still retrieved fresh on every request:
The caching approach balances performance with data consistency - friend changes and rating updates from other sessions require a session restart to be reflected, which is an acceptable trade-off for the significant performance improvement.
Fixes #89.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.