ZOOKEEPER: zk: not authenticated

[mikolajmorawski]

Hi,
I am trying to configure Burrow with Zookeeper server started with jaas.conf file. Recently I have solved the problems with KAFKA SSL/SASL (#333), but now I have a problem with this configuration. Zookeeper is started with DigestLoginModule.

jaas.conf:

Server {
       org.apache.zookeeper.server.auth.DigestLoginModule required
       admin="admin_pw";
};

zookeeper env:
SERVER_JVMFLAGS="-Djava.security.auth.login.config=/opt/zookeeper-3.4.9/conf/jaas.conf"

And Burrow with the following configuration:

[general]
access-control-allow-origin="*"

[logging]
level="debug"

[zookeeper]
servers=["183.155.2.197:2181"]

[client-profile.kafka-profile]
kafka-version="1.0.0"
client-id="burrow-client"
sasl="mysasl"
tls="mytls"

[sasl.mysasl]
username="admin"
password="admin-secret"
handshake-first=true

[tls.mytls]
noverify=true

[cluster.my_cluster]
class-name="kafka"
client-profile="kafka-profile"
servers=["183.155.2.197:9094"]
topic-refresh=120
offset-refresh=10

[consumer.consumer_kafka]
class-name="kafka"
cluster="my_cluster"
servers=["183.155.2.197:9094"]
client-profile="kafka-profile"
start-latest=false
offsets-topic="__consumer_offsets"
group-blacklist="^(console-consumer-|python-kafka-consumer-).*$"

[consumer.consumer_zk]
class-name="kafka_zk"
cluster="my_cluster"
servers=["183.155.2.197:2181"]
zookeeper-timeout=30
group-blacklist="^(console-consumer-|python-kafka-consumer-).*$"

[httpserver.default]
address=":8002"

After I have invoked this command on zookeeper:

./zookeeper-security-migration.sh --zookeeper.acl secure --zookeeper.connect 183.155.2.197:2181

i get the following error:

{"level":"info","ts":1517822857.9368994,"msg":"configuring","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1517822857.9369743,"msg":"configuring","type":"coordinator","name":"storage"}
{"level":"info","ts":1517822857.9370685,"msg":"configuring","type":"module","coordinator":"storage","class":"inmemory","name":"default"}
{"level":"info","ts":1517822857.937189,"msg":"configuring","type":"coordinator","name":"evaluator"}
{"level":"info","ts":1517822857.9372375,"msg":"configuring","type":"module","coordinator":"evaluator","class":"caching","name":"default"}
{"level":"info","ts":1517822857.9373174,"msg":"configuring","type":"coordinator","name":"httpserver"}
{"level":"info","ts":1517822857.9373896,"msg":"configuring","type":"coordinator","name":"notifier"}
{"level":"info","ts":1517822857.9374373,"msg":"configuring","type":"coordinator","name":"cluster"}
{"level":"info","ts":1517822857.937574,"msg":"configuring","type":"module","coordinator":"cluster","class":"kafka","name":"local_cluster"}
{"level":"info","ts":1517822857.937773,"msg":"configuring","type":"coordinator","name":"consumer"}
{"level":"info","ts":1517822857.937828,"msg":"configuring","type":"module","coordinator":"consumer","class":"kafka","name":"consumer_kafka"}
{"level":"info","ts":1517822857.9383683,"msg":"configuring","type":"module","coordinator":"consumer","class":"kafka_zk","name":"consumer_zk"}
{"level":"info","ts":1517822857.938516,"msg":"starting","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1517822857.9393747,"msg":"Connected to 183.155.2.197:2181","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1517822857.9441001,"msg":"Authenticated: id=171529193565388812, timeout=6000","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1517822857.944146,"msg":"Re-submitting `0` credentials after reconnect","type":"coordinator","name":"zookeeper"}
{"level":"error","ts":1517822857.9470801,"msg":"cannot create root path","type":"coordinator","name":"zookeeper","error":"zk: not authenticated"}
Burrow failed at February 5, 2018 at 9:27am (UTC)

Is it possible to pass user/password to zookeeper ?

[toddpalino]

At the present time, the zookeeper configurations (both for the general config and for the kafka_zk module) do not currently support authentication.

[mikolajmorawski]

I undestand that you are using this module to communicate with zookeeper:
"github.com/samuel/go-zookeeper/zk"
Do you plan to add zookeeper authentication in next release ?
I saw that this module
https://github.com/samuel/go-zookeeper/blob/master/zk/conn.go
is able to send AuthCreds to zookeeper:
func (c *Conn) AddAuth(scheme string, auth []byte)
Thanks and regards :)

[cluyihunter]

Please refer to #438 for solution. This works with our environment, which allows us to bypass zookeeper authentication if the root path has already been created. However I was not able to add test cases for the conditional statements. Also I have noticed that a lot of conditionals are not tested in Burrow. Is there a plan to fix that issue? Thanks! @toddpalino