Tracker Service v0.4: scene config dynamic loading & update#1009
Tracker Service v0.4: scene config dynamic loading & update#1009
Conversation
dc67fee to
6d7d2f7
Compare
There was a problem hiding this comment.
Pull request overview
Adds dynamic scene loading to the tracker by fetching scenes from the Manager REST API (auth + fetch + transform + schema validate) and wiring this into config/env, build, and automated tests.
Changes:
- Introduces
ManagerRestClient+ApiSceneLoaderand updatescreate_scene_loader()/mainto supportscenes.source=api. - Extends config schema/config loader with
infrastructure.manager+ env overrides for Manager settings. - Adds unit/service tests plus compose-based mock Manager for end-to-end API scene loading.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| tracker/test/unit/utils/mock_manager_rest_client.hpp | Adds gmockable Manager REST client interface for unit tests |
| tracker/test/unit/manager_rest_client_test.cpp | Adds end-to-end tests for ManagerRestClient using local httplib::Server |
| tracker/test/unit/config_loader_test.cpp | Expands config loader tests for API scenes source + Manager config/env overrides |
| tracker/test/unit/api_scene_loader_test.cpp | Adds unit tests for API scene loader pipeline and API->schema transforms |
| tracker/test/unit/CMakeLists.txt | Links new API/REST client sources and adds OpenSSL to unit test target |
| tracker/test/service/test_scene_loading.py | Adds service tests validating API scene loading and message flow |
| tracker/test/service/test_mqtt.py | Forces file-based scenes in TLS MQTT service test fixture |
| tracker/test/service/mocks/mock_manager.py | Adds mock Manager REST API server used by service tests |
| tracker/test/service/fixtures/test-scenes-api.json | Adds fixture payload emulating Manager /api/v1/scenes response |
| tracker/test/service/fixtures/test-auth.json | Adds fixture auth file for API mode tests |
| tracker/test/service/docker-compose.yaml | Adds mock-manager service/profile and API-related env/secrets wiring |
| tracker/test/service/conftest.py | Adds tracker_service_api fixture to run tracker against mock Manager |
| tracker/src/scene_loader.cpp | Refactors file loader parsing to shared parser; routes API mode to create_api_scene_loader |
| tracker/src/manager_rest_client.cpp | Implements auth + fetch scenes against Manager REST API via cpp-httplib |
| tracker/src/main.cpp | Passes Manager config + schema dir into scene loader factory |
| tracker/src/config_loader.cpp | Defaults scenes source to API; parses/overrides Manager config and validates API requirements |
| tracker/src/api_scene_loader.cpp | Implements API scene loading (auth file -> REST -> transform -> schema validate -> parse) |
| tracker/schema/config.schema.json | Updates schema for Manager auth file + API scenes default/requirements |
| tracker/inc/scenes_config.hpp | Introduces SceneSource/ScenesConfig in a dedicated header |
| tracker/inc/scene_registry.hpp | Adjusts includes to align with refactor |
| tracker/inc/scene_parser.hpp | Adds shared JSON->Scene parser used by both file and API loaders |
| tracker/inc/scene_loader.hpp | Extends factory API; adds Manager client factory + API helper declarations |
| tracker/inc/manager_rest_client.hpp | Adds IManagerRestClient + concrete ManagerRestClient interface |
| tracker/inc/env_vars.hpp | Adds Manager-related env var constants |
| tracker/inc/config_loader.hpp | Adds ManagerConfig and InfrastructureConfig::manager |
| tracker/config/tracker.json | Switches default config to API scenes and adds Manager section |
| tracker/conanfile.txt | Enables OpenSSL in cpp-httplib via Conan option |
| tracker/Makefile | Includes service-test mock scripts in autopep8 format/lint targets |
| tracker/CMakeLists.txt | Adds OpenSSL dependency and builds new API/REST client sources |
| sample_data/docker-compose-dl-streamer-example.yml | Configures tracker container for API scenes + Manager secrets and startup ordering |
- config_loader: validate url/auth_path emptiness after env overrides - manager_rest_client: configurable chrono timeouts (default 10s/30s) - manager_rest_client: always enable TLS server cert verification - manager_rest_client_test: ASSERT_GT on bind_to_any_port - manager_rest_client_test: use ephemeral port + 50ms timeout for connection-refused test (62ms vs 30s) - manager_rest_client_test: remove no-op FetchScenesInvalidUrlThrows - api_scene_loader: guard validate_scenes with IsArray() check - config_loader_test: add missing MANAGER_AUTH_PATH env in API test - scene_loader: remove dead ApiSceneLoader stub class
7ed7183 to
5b7d618
Compare
|
I've done development on this one. I'll keep it in draft until #968 is merged. Then I'll rebase and open for general review against main branch. |
1b10dfe to
142ca6e
Compare
T_EDITOR="true" git rebase --continue t status
142ca6e to
8f5b1f7
Compare
tdorauintc
left a comment
There was a problem hiding this comment.
There is one issue I would suggest to fix in this PR. Comments starting with [future] are not blocking the PR but added as a reminder to fix it in another future PR.
| void MessageHandler::handleDatabaseUpdateMessage(const std::string& topic, | ||
| const std::string& payload) { | ||
| LOG_INFO_ENTRY(LogEntry("Database update received, triggering restart") | ||
| .component("message_handler") | ||
| .mqtt({.topic = topic, .direction = "subscribe"})); | ||
|
|
||
| if (shutdown_callback_) { | ||
| shutdown_callback_(); | ||
| } else { | ||
| LOG_WARN("Database update received but no shutdown callback registered"); | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
[future] This will cause tracker restart even in case of adding tripwire or region, which impacts analytics but not the tracker itself. To be fixed later.
There was a problem hiding this comment.
ACK. This is done this way because Manager do not send update events for deleted scenes, only generic "database update". As an improvement either we can change the Manager to send delete updates on specific scene topic (just like it sends edit updates), or query manager API for current scenes state and compute locally if this update requires us to reload scenes.
Allows MANAGER_URL, MANAGER_AUTH_PATH, and MANAGER_CA_CERT_PATH environment variables to populate infrastructure.manager config before validation, instead of failing early when only env vars are set.
Co-authored-by: Lukasz Talarczyk <lukasz.talarczyk@intel.com>
Co-authored-by: Lukasz Talarczyk <lukasz.talarczyk@intel.com>
📝 Description
Introduce dynamic scene loading capabilities by allowing the application to fetch scenes from a Manager REST API instead of relying solely on local files.
Testing
✨ Type of Change
Select the type of change your PR introduces:
🧪 Testing Scenarios
Describe how the changes were tested and how reviewers can test them too:
✅ Checklist
Before submitting the PR, ensure the following: