As a user, X-TASK-DIAGNOSTICS / settings.TASK_DIAGNOSTICS supports capturing task-specific logs and presenting them via the profile artifacts mechanism

[dralley]

Is your feature request related to a problem? Please describe.

It can be challenging to gain access to Pulp logs in some circumstances, and in addition it can sometimes be difficult to parse the logs apart if there are many actions being simultaneously logged without external log processing tools like splunk + correlation IDs.

Describe the solution you'd like

It would be useful if a developer/user could use the X-TASK-DIAGNOSTICS mechanism to collect task-specific logs and make them available via the /profile_artifacts/ mechanism. This would combine nicely with #7214, as developers could then debug long-running tasks with verbose debug logging without needing to filter out as much unrelated information.

[mdellweg]

I think this is a nice idea. Can we make sure to only capture the logs from the actual tasks execution context (keep in mind that tasks may or may not run in a separate process)?
Also in interfering with another related issue, I think we should never present debug logs to users (accessible via the api or downloadable as the task diagnostic). This exposes the risk to leak security related information.

[dralley]

Also in interfering with another related issue, I think we should never present debug logs to users (accessible via the api or downloadable as the task diagnostic). This exposes the risk to leak security related information.

Yes, that's a bit of an open question, Brian thinks it's OK, but probably it should at least not be the default necessarily?

The implementation is also kinda hacky, because you need to change the level of the root logger to DEBUG level during the period where you want to capture those logs, or else start at DEBUG level and have them filtered out by the console log handler (but then pay more overhead because they are filtered later in the chain)