Log4j errors in Kafka container with cp-base-new when mapping custom user (FileNotFoundException: Permission denied)

[jovev961]

Hi,

When running the Kafka container (built on Confluent’s common-docker), I encounter Log4j errors related to file permissions when mapping a custom user ID and group ID. The errors prevent log files like server.log, controller.log, and others from being written, and they default to the root / directory.

1. Dockerfile: Add a custom user, e.g., prod_user:

    useradd -u 4327 -g 4327 prod_user

3. docker-compose.yml: Start the Kafka container using this user:

kafka:
  image: ${KAFKA_IMAGE}
  networks:
    - prod_net
  environment:
    KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
    KAFKA_BROKER_ID: 1
    KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    TZ: ${TZ}
  user: "4327:4327"
  command: /usr/local/sbin/kafka-start-compact
  volumes:
    - /opt/prod/kafka-topic-logs:/var/lib/kafka/data

4. Resulting Logs (from container):

log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /server.log (Permission denied)
...
java.io.FileNotFoundException: /controller.log (Permission denied)
...

The Kafka container defaults Log4j file paths to /, which is not writable for the mapped custom user. Even after modifying all log4j.properties files (Kafka, Confluent, cp-base-new), the issue persists. It seems there is a hardcoded or default configuration I cannot override.

Relevant logs: kafkaLog.pdf
Log4j Properties Example: log4jProperties.pdf