Skip to content

docker stop analyticsdb - gets stuck indefinitely #116

@sangupta01

Description

@sangupta01

Using build 3023 but saw with 3018 too.
Seen this issue using docker builds from 10.84.34.155 and for 3018/3023 pvt docker images/registry.

Not able to stop containers. "docker kill" also gets stuck.

  • no errors seen in "docker logs " during install using ansible.
  • no errors seen in "docker logs -f " during "docker stop ..."
    This is seen mostly with analyticsdb and controller containers.
    Sometimes rebooting the node multiple times and trying "docker stop ..." works.
    logging into the container and stopping services (kafka, database-manager) didnt help.

2017-01-13 06:10:11,483 INFO exited: kafka (exit status 1; not expected)
FAILED - RETRYING: TASK: contrail/analyticsdb : Wait till config api server answers (121 retries left).
2017-01-13 06:10:12,486 INFO spawned: 'kafka' with pid 2833
FAILED - RETRYING: TASK: contrail/analyticsdb : Wait till config api server answers (120 retries left).
2017-01-13 06:10:13,657 INFO stopped: kafka (exit status 1)
FAILED - RETRYING: TASK: contrail/analyticsdb : Wait till config api server answers (119 retries left).
...
FAILED - RETRYING: TASK: contrail/analyticsdb : Wait till config api server answers (107 retries left).
2017-01-13 06:10:28,933 INFO stopped: contrail-database-nodemgr (terminated by SIGTERM)

At this point, there is no further dump on the screen even if I do CTRL-C and "docker stop/kill" in another screen.
"docker ps -a" still shows STATUS as UP.

Even waiting till 0 retries left and then failed, "docker stop ..." gets stuck.
Trying to login shows below error.
root@a1s42:~# docker exec -it analyticsdb /bin/bash
rpc error: code = 13 desc = invalid header field value "oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:83: executing setns process caused \"exit status 16\""\n"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions