From 1453627d700adfbea5a5ec73462d81a789ce1a05 Mon Sep 17 00:00:00 2001 From: runllm Date: Fri, 18 Apr 2025 13:01:36 +0000 Subject: [PATCH 1/2] Update ui-ingestion.md --- docs/ui-ingestion.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/ui-ingestion.md b/docs/ui-ingestion.md index 521d8db26011cf..14fb6a42549ddd 100644 --- a/docs/ui-ingestion.md +++ b/docs/ui-ingestion.md @@ -309,6 +309,26 @@ on the corresponding ingestion run.

+## Ingestion Logs Location and Management + +### Log Location +Ingestion logs for the actions pod are stored in the `/tmp/datahub/logs` directory within the pod. Logs are dynamically created under `/tmp/datahub/ingest/{ingestion_exec_id}/` and are deleted after the ingestion process completes. + +### Log Lifecycle +Logs are temporary and are deleted after the ingestion process completes, so timely access is crucial for debugging purposes. + +### Accessing Logs +To access the logs, use the `kubectl exec` command to enter the pod and view the logs. Example: +```bash +kubectl exec -n -- ls -la /tmp/datahub/logs +``` + +### Saving Logs +To save logs to your local filesystem for further analysis, use `kubectl cp` or redirect the output to a file. Example: +```bash +kubectl cp /:/tmp/datahub/logs ./local-logs +``` + ## FAQ From 0a3bce4b69d6a20ea55d1feb7a11fdc0d7052d08 Mon Sep 17 00:00:00 2001 From: runllm Date: Fri, 18 Apr 2025 13:01:37 +0000 Subject: [PATCH 2/2] Update extract-container-logs.md --- docs/how/extract-container-logs.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/how/extract-container-logs.md b/docs/how/extract-container-logs.md index d702a0acc91230..73070bbb825eba 100644 --- a/docs/how/extract-container-logs.md +++ b/docs/how/extract-container-logs.md @@ -126,6 +126,22 @@ docker exec --privileged 122a2488ab63 cat /tmp/datahub/logs/gms/gms.debug.log > Now you should be able to view the logs locally. +### Ingestion Logs Location and Management + +- **Log Location**: Ingestion logs for the actions pod are stored in the `/tmp/datahub/logs` directory within the pod. Logs are dynamically created under `/tmp/datahub/ingest/{ingestion_exec_id}/` and are deleted after the ingestion process completes. + +- **Log Lifecycle**: Logs are temporary and are deleted after the ingestion process completes, so timely access is important for debugging purposes. + +- **Accessing Logs**: Use `kubectl exec` to access the pod and view the logs. Example command: + ```bash + kubectl exec -n -- ls -la /tmp/datahub/logs + ``` + +- **Saving Logs**: To save logs to the local filesystem for further analysis, use `kubectl cp` or shell redirection. Example: + ```bash + kubectl cp /:/tmp/datahub/logs/my-log.log ./my-local-log.log + ``` + ### Kubernetes & Helm There are a few ways to get files out of the pod and into a local file. You can either use `kubectl cp` or simply `cat` and pipe the file of interest. We'll show an example using the latter approach: