Skip to content

Write structured logs; send uncaught logs to portal log#740

Merged
matthew-li merged 2 commits intoupgrade/rocky8-python3.13-developfrom
upgrade/rocky8-python3.13-structured-logs
Feb 9, 2026
Merged

Write structured logs; send uncaught logs to portal log#740
matthew-li merged 2 commits intoupgrade/rocky8-python3.13-developfrom
upgrade/rocky8-python3.13-structured-logs

Conversation

@matthew-li
Copy link
Collaborator

@matthew-li matthew-li commented Feb 8, 2026

Description

**** Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change. ****

  • Added structlog and django-structlog as dependencies.
  • Configured structlog to intercept all existing log messages and write them as structured logs.
  • Added two log formatters:
    • Added a console_dev formatter for pretty printing logs.
    • Added a json formatter for writing logs as JSON.
  • Updated log handlers:
    • Made the console handler use console_dev if DEBUG else json.
    • Made the file and api_file handlers use json.
      • If STREAM_LOGS_TO_STDOUT, made them use console_dev if DEBUG else json.
  • Added a root logger to catch all uncaught logs and send them to file.
    • Some were previously written to the Apache log in mod_wsgi-based deployments.
    • django-q2 were previously written to files in /tmp.
  • Added request ID and user ID to each log message (via django_structlog.middlewares.RequestMiddleware).

Type of change

**** Please delete options that are not relevant. ****

  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • There are no log parsers in place. However, some logs that were previously sent to the Apache log or to /tmp will now be sent to the portal log file.

How Has This Been Tested?

**** Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration. ****

  • Ensure that test suite passes on GitHub Actions.
  • Ensure that logs are pretty printed to stdout in the Docker development environment.
  • Ensure that new logs are written to stdout in the Docker development environment.

PR Self Evaluation

Strikethrough things that don’t make sense for your PR.

  • My code follows the agreed upon best practices
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (if needed)
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in the appropriate modules
  • I have performed a self-review of my own code

* Wrap all existing unstructured logs as structured
* Pretty print when DEBUG = true, else log JSON
* Add root logger to send uncaught logs to portal log, not Apache log
@matthew-li matthew-li merged commit a1626fc into upgrade/rocky8-python3.13-develop Feb 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant