diff --git a/docs/how/extract-container-logs.md b/docs/how/extract-container-logs.md index d702a0acc9123..73070bbb825eb 100644 --- a/docs/how/extract-container-logs.md +++ b/docs/how/extract-container-logs.md @@ -126,6 +126,22 @@ docker exec --privileged 122a2488ab63 cat /tmp/datahub/logs/gms/gms.debug.log > Now you should be able to view the logs locally. +### Ingestion Logs Location and Management + +- **Log Location**: Ingestion logs for the actions pod are stored in the `/tmp/datahub/logs` directory within the pod. Logs are dynamically created under `/tmp/datahub/ingest/{ingestion_exec_id}/` and are deleted after the ingestion process completes. + +- **Log Lifecycle**: Logs are temporary and are deleted after the ingestion process completes, so timely access is important for debugging purposes. + +- **Accessing Logs**: Use `kubectl exec` to access the pod and view the logs. Example command: + ```bash + kubectl exec -n -- ls -la /tmp/datahub/logs + ``` + +- **Saving Logs**: To save logs to the local filesystem for further analysis, use `kubectl cp` or shell redirection. Example: + ```bash + kubectl cp /:/tmp/datahub/logs/my-log.log ./my-local-log.log + ``` + ### Kubernetes & Helm There are a few ways to get files out of the pod and into a local file. You can either use `kubectl cp` or simply `cat` and pipe the file of interest. We'll show an example using the latter approach: diff --git a/docs/ui-ingestion.md b/docs/ui-ingestion.md index 521d8db26011c..14fb6a42549dd 100644 --- a/docs/ui-ingestion.md +++ b/docs/ui-ingestion.md @@ -309,6 +309,26 @@ on the corresponding ingestion run.

+## Ingestion Logs Location and Management + +### Log Location +Ingestion logs for the actions pod are stored in the `/tmp/datahub/logs` directory within the pod. Logs are dynamically created under `/tmp/datahub/ingest/{ingestion_exec_id}/` and are deleted after the ingestion process completes. + +### Log Lifecycle +Logs are temporary and are deleted after the ingestion process completes, so timely access is crucial for debugging purposes. + +### Accessing Logs +To access the logs, use the `kubectl exec` command to enter the pod and view the logs. Example: +```bash +kubectl exec -n -- ls -la /tmp/datahub/logs +``` + +### Saving Logs +To save logs to your local filesystem for further analysis, use `kubectl cp` or redirect the output to a file. Example: +```bash +kubectl cp /:/tmp/datahub/logs ./local-logs +``` + ## FAQ