Add performance logging#268
Open
cmatKhan wants to merge 8 commits intoBrentLab:devfrom
Open
Conversation
Member
Author
|
This is deployed and the cloudwatch logging is nominally working 
but this is mostly AI and needs refinement, especially in what is performance monitored |
Member
Author
|
Closes #267 |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds structured performance/session logging throughout the Shiny app and provisions CloudWatch metrics/alarms/dashboard to operationalize that telemetry.
Changes:
- Introduces a
profilerutility that emits single-line JSON timing spans (PROFILE) and session lifecycle records (SESSION). - Instruments VirtualDB initialization, query paths, and plot HTML generation across modules with
profile_span(...), and threadsprofile_logger/session_idthrough module servers. - Adds production/ops wiring: CLI/env toggles for profiling, CloudWatch log groups + metric filters + alarms + dashboard, and Docker log routing updates.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
tfbpshiny/utils/vdb_init.py |
Adds init-time profiling spans around expensive DB setup/lookup queries. |
tfbpshiny/utils/profiler.py |
New JSON structured timing/session logging helpers. |
tfbpshiny/modules/select_datasets/server/workspace.py |
Profiles key vdb.query calls used to build the matrix. |
tfbpshiny/modules/perturbation/server/workspace.py |
Profiles correlation queries and Plotly HTML generation. |
tfbpshiny/modules/comparison/server/workspace.py |
Profiles top-N query execution, dataframe concat, and plot build. |
tfbpshiny/modules/binding/server/workspace.py |
Profiles correlation queries and Plotly HTML generation. |
tfbpshiny/app.py |
Configures profiler logger, logs session START/END, passes profiler context into modules and init. |
tfbpshiny/__main__.py |
Adds CLI flags to select profile handler and disable profiling. |
configure_logger.py |
Adds configure_profile_logger(...) for a dedicated profiler logger format/handler. |
terraform/main.tf |
Pins EC2 instance replacement behavior for AMI/user-data changes via ignore_changes. |
terraform/cloudwatch.tf |
Creates log groups, metric filters, alarms, and a dashboard based on JSON profiler logs. |
production.yml |
Adds profiler env + routes Traefik logs to its own log group/region. |
docs/development.md |
Documents existence of profiling log (but examples need updating to JSON format). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+216
to
+218
| _pl = profile_logger or logging.getLogger("profiler") | ||
| with profile_span(_pl, "init.regulator_table"): | ||
| vdb._conn.execute(sql) |
Comment on lines
+237
to
+243
| _pl = profile_logger or logging.getLogger("profiler") | ||
| with profile_span( | ||
| _pl, | ||
| "vdb.execute", | ||
| dataset=_REGULATOR_DISPLAY_NAME_TABLE, | ||
| context="get_regulator_display_name", | ||
| ): |
| :param logger: The ``"profiler"`` logger. | ||
| :param op: Operation label, e.g. ``"vdb.query"`` or ``"plot.build"``. | ||
| :param module: Shiny module name, e.g. ``"binding"``. Empty for init-time spans. | ||
| :param dataset: Dataset(s) involved, e.g. ``"harbison"`` or ``"harbisонxrossi"``. |
Comment on lines
32
to
36
| vdb: VirtualDB, | ||
| logger: Logger, | ||
| profile_logger: Logger, | ||
| session_id: str = "", | ||
| ) -> None: |
Comment on lines
+50
to
+60
| There is both a general log and profiling log | ||
|
|
||
| Both loggers write to stdout/stderr, Docker's awslogs driver captures everything into the shinyapp stream under /tfbpshiny/production. To later separate PROFILE lines from main log lines when parsing: | ||
|
|
||
| ```{python} | ||
| import pandas as pd | ||
| df = pd.read_csv("exported.log", sep="|", header=None, skipinitialspace=True, | ||
| names=["marker","timestamp","elapsed_s","op","module","dataset","context"]) | ||
| profile = df[df["marker"].str.strip() == "PROFILE"] | ||
| ``` |
Comment on lines
32
to
36
| vdb: VirtualDB, | ||
| logger: Logger, | ||
| profile_logger: Logger, | ||
| session_id: str = "", | ||
| ) -> None: |
Comment on lines
+74
to
77
| profile_logger: Logger, | ||
| session_id: str = "", | ||
| ) -> None: | ||
| """Render the Top-N by Binding workspace plot.""" |
Comment on lines
+42
to
46
| profile_logger: Logger, | ||
| session_id: str = "", | ||
| ) -> None: | ||
| """Render the sample-count matrix for all active datasets.""" | ||
|
|
Comment on lines
+66
to
+68
| fields @timestamp, @message | ||
| | filter @message like /^PROFILE/ | ||
| | parse @message "PROFILE | * | * | * | * | * | *" as timestamp, elapsed_s, op, module, dataset, context |
Comment on lines
58
to
65
| if level not in [logging.DEBUG, logging.INFO, logging.WARNING, logging.ERROR]: | ||
| raise ValueError("Invalid logging level") | ||
| if not isinstance(format, str): | ||
| raise ValueError("format must be a string") | ||
| if handler_type not in ["console", "file"]: | ||
| if handler_type not in ("console", "file"): | ||
| raise ValueError("handler_type must be 'console' or 'file'") | ||
| if handler_type == "file" and not log_file: | ||
| raise ValueError("log_file must be specified for file handler") |
Collaborator
|
LGTM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.