Skip to content

SeldonDeployment 的pod没有ready #189

@gongyu143

Description

@gongyu143

[root@xxxx ormb]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/tfserving-default-0-mnist-model-6cdfd5d-6vncx 0/2 Running 0 108m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 443/TCP 3d21h
service/tfserving-default ClusterIP 10.102.59.194 8000/TCP,5001/TCP 109m
service/tfserving-default-mnist-model ClusterIP 10.109.93.101 9000/TCP 109m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/tfserving-default-0-mnist-model 0/1 1 0 108m

NAME DESIRED CURRENT READY AGE
replicaset.apps/tfserving-default-0-mnist-model-6cdfd5d 1 1 0 108m

kubectl describe pod/tfserving-default-0-mnist-model-6cdfd5d-6vncx

Events:
Type Reason Age From Message


Warning Unhealthy 40s (x1699 over 4h18m) kubelet Liveness probe errored: strconv.Atoi: parsing "http": invalid syntax

kubectl logs -f tfserving-default-0-mnist-model-6cdfd5d-6vncx mnist-model

2021-09-30 06:10:13.660649: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable mnist-model found under base path /mnt/models. Did you forget to name your leaf directory as a number (eg. '/1/')?
2021-09-30 06:10:14.660861: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable mnist-model found under base path /mnt/models. Did you forget to name your leaf directory as a number (eg. '/1/')?
2021-09-30 06:10:15.662074: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable mnist-model found under base path /mnt/models. Did you forget to name your leaf directory as a number (eg. '/1/')?
2021-09-30 06:10:16.662489: W tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:268] No versions of servable mnist-model found under base path /mnt/models. Did you forget to name your leaf directory as a number (eg. '/1/')?

kubectl logs -f tfserving-default-0-mnist-model-6cdfd5d-6vncx mnist-model-model-initializer

Using / as the root path
Logging to the remote registry demo.goharbor.io
Username: ormbtest
Login insecurely
time="2021-09-30T01:55:43Z" level=debug msg="attempting v2 login to registry endpoint https://demo.goharbor.io/v2/"
Login succeeded
Using / as the root path
v1: Pulling from demo.goharbor.io/tensorflow/fashion_model
ref: demo.goharbor.io/tensorflow/fashion_model:v1
digest: 62c7d273602a9dc6833a1401e0186f50708e1e03b717edc4b0b39781b53c367a
size: 162.1 KiB
format: SavedModel
Status: Downloaded newer model for demo.goharbor.io/tensorflow/fashion_model:v1
ref: demo.goharbor.io/tensorflow/fashion_model:v1
digest: 62c7d273602a9dc6833a1401e0186f50708e1e03b717edc4b0b39781b53c367a
size: 162.1 KiB
format: SavedModel
Moving /mnt/models/model/saved_model.pb to /mnt/models/saved_model.pb
Moving /mnt/models/model/variables to /mnt/models/variables

有没有大神知道怎么解决

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions