Service catalog install failed #182
Description
Hello,
I am install okd 3.11 with this script on a fresh CentOS 7.7 server and unfortunately the service catalog part of the deploy ansible script fails. Below I have pasted the two specific tasks which fail including the debug info.
The only custom settings I have for the installer script is that I have answered yes to use Let's Encrypt (the certbot part of the script went fine) and I am using export DISK="/dev/sdb".
Any hints or help is appreciated. Thanks in advance.
TASK [openshift_service_catalog : Wait for Controller Manager rollout success] *************************************************************************************************************************************************************************************************
FAILED - RETRYING: Wait for Controller Manager rollout success (5 retries left).
FAILED - RETRYING: Wait for Controller Manager rollout success (4 retries left).
FAILED - RETRYING: Wait for Controller Manager rollout success (3 retries left).
FAILED - RETRYING: Wait for Controller Manager rollout success (2 retries left).
FAILED - RETRYING: Wait for Controller Manager rollout success (1 retries left).
fatal: [<IP_REMOVED>]: FAILED! => {"attempts": 5, "changed": false, "cmd": ["oc", "rollout", "status", "--config=/etc/origin/master/admin.kubeconfig", "-n", "kube-service-catalog", "ds/controller-manager"], "delta": "0:00:00.203854", "end": "2020-04-17 23:19:50.143786", "msg": "non-zero return code", "rc": 1, "start": "2020-04-17 23:19:49.939932", "stderr": "error: watch closed before Until timeout", "stderr_lines": ["error: watch closed before Until timeout"], "stdout": "Waiting for daemon set \"controller-manager\" rollout to finish: 0 of 1 updated pods are available...\nWaiting for daemon set \"controller-manager\" rollout to finish: 0 of 1 updated pods are available...", "stdout_lines": ["Waiting for daemon set \"controller-manager\" rollout to finish: 0 of 1 updated pods are available...", "Waiting for daemon set \"controller-manager\" rollout to finish: 0 of 1 updated pods are available..."]}
...ignoring
TASK [openshift_service_catalog : Verify that the Catalog API Server is running] ***********************************************************************************************************************************************************************************************
FAILED - RETRYING: Verify that the Catalog API Server is running (60 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (59 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (58 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (57 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (56 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (55 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (54 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (53 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (52 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (51 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (50 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (49 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (48 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (47 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (46 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (45 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (44 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (43 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (42 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (41 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (40 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (39 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (38 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (37 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (36 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (35 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (34 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (33 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (32 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (31 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (30 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (29 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (28 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (27 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (26 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (25 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (24 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (23 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (22 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (21 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (20 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (19 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (18 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (17 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (16 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (15 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (14 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (13 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (12 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (11 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (10 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (9 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (8 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (7 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (6 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (5 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (4 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (3 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (2 retries left).
FAILED - RETRYING: Verify that the Catalog API Server is running (1 retries left).
fatal: [<IP_REMOVED>]: FAILED! => {"attempts": 60, "changed": false, "cmd": ["curl", "-k", "https://apiserver.kube-service-catalog.svc/healthz"], "delta": "0:00:01.010253", "end": "2020-04-17 23:31:06.368636", "msg": "non-zero return code", "rc": 7, "start": "2020-04-17 23:31:05.358383", "stderr": " % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to apiserver.kube-service-catalog.svc:443; Connection refused", "stderr_lines": [" % Total % Received % Xferd Average Speed Time Time Time Current", " Dload Upload Total Spent Left Speed", "", " 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to apiserver.kube-service-catalog.svc:443; Connection refused"], "stdout": "", "stdout_lines": []}
...ignoring
TASK [openshift_service_catalog : debug] ***************************************************************************************************************************************************************************************************************************************
ok: [<IP_REMOVED>] => {
"msg": [
"In project kube-service-catalog on server https://console-internal.mydomain.com:8443",
"",
"https://apiserver-kube-service-catalog.apps.mydomain.com (passthrough) to pod port secure (svc/apiserver)",
" daemonset/apiserver manages docker.io/openshift/origin-service-catalog:v3.11",
" generation #1 running for 31 minutes - 0/1 pods growing to 1",
" pod/apiserver-hjbmt runs docker.io/openshift/origin-service-catalog:v3.11",
"",
"svc/controller-manager - 172.30.179.105:443 -> 6443",
" daemonset/controller-manager manages docker.io/openshift/origin-service-catalog:v3.11",
" generation #1 running for 31 minutes - 0/1 pods growing to 1",
" pod/controller-manager-5479q runs docker.io/openshift/origin-service-catalog:v3.11",
"",
"Errors:",
" * pod/controller-manager-5479q is crash-looping",
"",
"1 error, 1 warning identified, use 'oc status --suggest' to see details."
]
}
TASK [openshift_service_catalog : Get pods in the kube-service-catalog namespace] **********************************************************************************************************************************************************************************************
changed: [<IP_REMOVED>]
TASK [openshift_service_catalog : debug] ***************************************************************************************************************************************************************************************************************************************
ok: [<IP_REMOVED>] => {
"msg": [
"NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE",
"apiserver-hjbmt 0/1 Running 8 31m 10.128.0.32 myserver.mydomain.com <none>",
"controller-manager-5479q 0/1 CrashLoopBackOff 11 31m 10.128.0.33 myserver.mydomain.com <none>"
]
}
TASK [openshift_service_catalog : Get events in the kube-service-catalog namespace] ********************************************************************************************************************************************************************************************
changed: [<IP_REMOVED>]
TASK [openshift_service_catalog : debug] ***************************************************************************************************************************************************************************************************************************************
ok: [<IP_REMOVED>] => {
"msg": [
"LAST SEEN FIRST SEEN COUNT NAME KIND SUBOBJECT TYPE REASON SOURCE MESSAGE",
"31m 31m 1 apiserver.1606b708b62fc509 DaemonSet Normal SuccessfulCreate daemonset-controller Created pod: apiserver-hjbmt",
"31m 31m 1 apiserver-hjbmt.1606b70943ace0c8 Pod spec.containers{apiserver} Normal Pulling kubelet, myserver.mydomain.com pulling image \"docker.io/openshift/origin-service-catalog:v3.11\"",
"31m 31m 1 controller-manager.1606b709c9ea1b59 DaemonSet Normal SuccessfulCreate daemonset-controller Created pod: controller-manager-5479q",
"31m 31m 2 controller-manager-5479q.1606b709de00d7d6 Pod Warning FailedMount kubelet, myserver.mydomain.com MountVolume.SetUp failed for volume \"service-catalog-ssl\" : secrets \"controllermanager-ssl\" not found",
"31m 31m 1 apiserver-hjbmt.1606b70a9fb9cab4 Pod spec.containers{apiserver} Normal Pulled kubelet, myserver.mydomain.com Successfully pulled image \"docker.io/openshift/origin-service-catalog:v3.11\"",
"31m 31m 1 service-catalog-controller-manager.1606b70b77a69ae0 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"31m 31m 1 service-catalog-controller-manager.1606b70bda237e24 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"31m 31m 1 service-catalog-controller-manager.1606b70f719db525 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"30m 31m 4 controller-manager-5479q.1606b70b6b94652c Pod spec.containers{controller-manager} Normal Created kubelet, myserver.mydomain.com Created container",
"30m 31m 4 controller-manager-5479q.1606b70b769e9857 Pod spec.containers{controller-manager} Normal Started kubelet, myserver.mydomain.com Started container",
"30m 30m 1 service-catalog-controller-manager.1606b7179303ccc6 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"29m 31m 5 controller-manager-5479q.1606b70b462e62a6 Pod spec.containers{controller-manager} Normal Pulled kubelet, myserver.mydomain.com Container image \"docker.io/openshift/origin-service-catalog:v3.11\" already present on machine",
"29m 29m 1 service-catalog-controller-manager.1606b721912d165d ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"28m 31m 4 apiserver-hjbmt.1606b70fe7f2776a Pod spec.containers{apiserver} Normal Pulled kubelet, myserver.mydomain.com Container image \"docker.io/openshift/origin-service-catalog:v3.11\" already present on machine",
"28m 31m 5 apiserver-hjbmt.1606b70acc234e25 Pod spec.containers{apiserver} Normal Created kubelet, myserver.mydomain.com Created container",
"28m 31m 5 apiserver-hjbmt.1606b70ada41daa4 Pod spec.containers{apiserver} Normal Started kubelet, myserver.mydomain.com Started container",
"28m 28m 1 service-catalog-controller-manager.1606b73739e3ccde ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"25m 25m 1 service-catalog-controller-manager.1606b75cbc43ac23 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"20m 20m 1 service-catalog-controller-manager.1606b7a3f3664094 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"16m 30m 61 apiserver-hjbmt.1606b7150294a040 Pod spec.containers{apiserver} Warning BackOff kubelet, myserver.mydomain.com Back-off restarting failed container",
"15m 15m 1 service-catalog-controller-manager.1606b7ebe9dfac99 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"10m 10m 1 service-catalog-controller-manager.1606b832ad46ce74 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"5m 5m 1 service-catalog-controller-manager.1606b879b2aab021 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader",
"1m 11m 121 apiserver-hjbmt.1606b821a488e639 Pod spec.containers{apiserver} Warning Unhealthy kubelet, myserver.mydomain.com Readiness probe failed: HTTP probe failed with statuscode: 500",
"1m 31m 140 controller-manager-5479q.1606b70c2c1048b1 Pod spec.containers{controller-manager} Warning BackOff kubelet, myserver.mydomain.com Back-off restarting failed container",
"15s 15s 1 service-catalog-controller-manager.1606b8c003642347 ConfigMap Normal LeaderElection service-catalog-controller-manager controller-manager-5479q-external-service-catalog-controller became leader"
]
}
TASK [openshift_service_catalog : Get pod logs] ********************************************************************************************************************************************************************************************************************************
changed: [<IP_REMOVED>]
TASK [openshift_service_catalog : debug] ***************************************************************************************************************************************************************************************************************************************
ok: [<IP_REMOVED>] => {
"msg": [
"logging error output: \"[-]etcd failed: reason withheld\\nhealthz check failed\\n\"",
" [[kube-probe/1.11+] 10.128.0.1:45198]",
"I0417 21:30:52.331698 1 run_server.go:134] etcd checker called",
"I0417 21:30:53.267983 1 wrap.go:42] GET /healthz: (79.085µs) 200 [[kube-probe/1.11+] 10.128.0.1:45242]",
"E0417 21:30:53.331836 1 run_server.go:143] etcd failed to reach any server",
"I0417 21:30:53.332025 1 wrap.go:42] GET /healthz/ready: (1.003896742s) 500",
"goroutine 4841 [running]:",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc420439d50, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xd2",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc420439d50, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).WriteHeader(0xc4204814c0, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:192 +0xac",
"net/http.Error(0x7f676e03dbb0, 0xc4200e6d60, 0xc4204a7000, 0x34, 0x1f4)",
"\t/usr/local/go/src/net/http/server.go:1959 +0xda",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz.handleRootHealthz.func1(0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz/healthz.go:130 +0x4c4",
"net/http.HandlerFunc.ServeHTTP(0xc42022df40, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP(0xc42088c9c0, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:241 +0x514",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP(0xc42010f3b0, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:234 +0x8a",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server.director.ServeHTTP(0x1a820d2, 0x19, 0xc42062b290, 0xc42010f3b0, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/handler.go:154 +0x6a8",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go:64 +0x4bd",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb5c0, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/maxinflight.go:160 +0x3fc",
"net/http.HandlerFunc.ServeHTTP(0xc42026fef0, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/impersonation.go:50 +0x1ce4",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb600, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37b00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authentication.go:75 +0x2b3",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb640, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37a00)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithCORS.func1(0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37a00)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/cors.go:75 +0x178",
"net/http.HandlerFunc.ServeHTTP(0xc42008d0e0, 0x7f676e03dbb0, 0xc4200e6d60, 0xc420a37a00)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc4204c61c0, 0x1fa8700, 0xc4200e6d60, 0xc420a37a00, 0xc4201d9980)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:97 +0x8d",
"created by github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:96 +0x1b5",
"",
"logging error output: \"[-]etcd failed: reason withheld\\nhealthz check failed\\n\"",
" [[kube-probe/1.11+] 10.128.0.1:45230]",
"I0417 21:30:57.327797 1 run_server.go:134] etcd checker called",
"E0417 21:30:58.328094 1 run_server.go:143] etcd failed to reach any server",
"I0417 21:30:58.328288 1 wrap.go:42] GET /healthz/ready: (1.000566658s) 500",
"goroutine 4861 [running]:",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc420752000, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xd2",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc420752000, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).WriteHeader(0xc4203ec0e0, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:192 +0xac",
"net/http.Error(0x7f676e03dbb0, 0xc4209f2298, 0xc420b34ec0, 0x34, 0x1f4)",
"\t/usr/local/go/src/net/http/server.go:1959 +0xda",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz.handleRootHealthz.func1(0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz/healthz.go:130 +0x4c4",
"net/http.HandlerFunc.ServeHTTP(0xc42022df40, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP(0xc42088c9c0, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:241 +0x514",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP(0xc42010f3b0, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:234 +0x8a",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server.director.ServeHTTP(0x1a820d2, 0x19, 0xc42062b290, 0xc42010f3b0, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/handler.go:154 +0x6a8",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go:64 +0x4bd",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb5c0, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/maxinflight.go:160 +0x3fc",
"net/http.HandlerFunc.ServeHTTP(0xc42026fef0, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/impersonation.go:50 +0x1ce4",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb600, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4600)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authentication.go:75 +0x2b3",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb640, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4500)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithCORS.func1(0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/cors.go:75 +0x178",
"net/http.HandlerFunc.ServeHTTP(0xc42008d0e0, 0x7f676e03dbb0, 0xc4209f2298, 0xc4200d4500)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc4204c61c0, 0x1fa8700, 0xc4209f2298, 0xc4200d4500, 0xc420bd7380)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:97 +0x8d",
"created by github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:96 +0x1b5",
"",
"logging error output: \"[-]etcd failed: reason withheld\\nhealthz check failed\\n\"",
" [[kube-probe/1.11+] 10.128.0.1:45286]",
"I0417 21:31:02.327443 1 run_server.go:134] etcd checker called",
"I0417 21:31:03.271530 1 wrap.go:42] GET /healthz: (3.603614ms) 200 [[kube-probe/1.11+] 10.128.0.1:45368]",
"E0417 21:31:03.327854 1 run_server.go:143] etcd failed to reach any server",
"I0417 21:31:03.328053 1 wrap.go:42] GET /healthz/ready: (1.000644294s) 500",
"goroutine 4880 [running]:",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc4202f1420, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xd2",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc4202f1420, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).WriteHeader(0xc420481ee0, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:192 +0xac",
"net/http.Error(0x7f676e03dbb0, 0xc4200e6e90, 0xc4204a7a00, 0x34, 0x1f4)",
"\t/usr/local/go/src/net/http/server.go:1959 +0xda",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz.handleRootHealthz.func1(0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz/healthz.go:130 +0x4c4",
"net/http.HandlerFunc.ServeHTTP(0xc42022df40, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP(0xc42088c9c0, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:241 +0x514",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP(0xc42010f3b0, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:234 +0x8a",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server.director.ServeHTTP(0x1a820d2, 0x19, 0xc42062b290, 0xc42010f3b0, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/handler.go:154 +0x6a8",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go:64 +0x4bd",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb5c0, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/maxinflight.go:160 +0x3fc",
"net/http.HandlerFunc.ServeHTTP(0xc42026fef0, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/impersonation.go:50 +0x1ce4",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb600, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f500)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authentication.go:75 +0x2b3",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb640, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f400)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithCORS.func1(0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f400)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/cors.go:75 +0x178",
"net/http.HandlerFunc.ServeHTTP(0xc42008d0e0, 0x7f676e03dbb0, 0xc4200e6e90, 0xc42002f400)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc4204c61c0, 0x1fa8700, 0xc4200e6e90, 0xc42002f400, 0xc4210a89c0)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:97 +0x8d",
"created by github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:96 +0x1b5",
"",
"logging error output: \"[-]etcd failed: reason withheld\\nhealthz check failed\\n\"",
" [[kube-probe/1.11+] 10.128.0.1:45356]",
"I0417 21:31:07.329046 1 run_server.go:134] etcd checker called",
"E0417 21:31:08.329250 1 run_server.go:143] etcd failed to reach any server",
"I0417 21:31:08.329370 1 wrap.go:42] GET /healthz/ready: (1.001460942s) 500",
"goroutine 4948 [running]:",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).recordStatus(0xc4202f1c00, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:207 +0xd2",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog.(*respLogger).WriteHeader(0xc4202f1c00, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/httplog/httplog.go:186 +0x35",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*baseTimeoutWriter).WriteHeader(0xc420510a20, 0x1f4)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:192 +0xac",
"net/http.Error(0x7f676e03dbb0, 0xc4200e6ff8, 0xc420bc0fc0, 0x34, 0x1f4)",
"\t/usr/local/go/src/net/http/server.go:1959 +0xda",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz.handleRootHealthz.func1(0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/healthz/healthz.go:130 +0x4c4",
"net/http.HandlerFunc.ServeHTTP(0xc42022df40, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*pathHandler).ServeHTTP(0xc42088c9c0, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:241 +0x514",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).ServeHTTP(0xc42010f3b0, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:234 +0x8a",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server.director.ServeHTTP(0x1a820d2, 0x19, 0xc42062b290, 0xc42010f3b0, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/handler.go:154 +0x6a8",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthorization.func1(0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authorization.go:64 +0x4bd",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb5c0, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithMaxInFlightLimit.func1(0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/maxinflight.go:160 +0x3fc",
"net/http.HandlerFunc.ServeHTTP(0xc42026fef0, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithImpersonation.func1(0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/impersonation.go:50 +0x1ce4",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb600, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters.WithAuthentication.func1(0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16300)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/endpoints/filters/authentication.go:75 +0x2b3",
"net/http.HandlerFunc.ServeHTTP(0xc4205eb640, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16200)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.WithCORS.func1(0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16200)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/cors.go:75 +0x178",
"net/http.HandlerFunc.ServeHTTP(0xc42008d0e0, 0x7f676e03dbb0, 0xc4200e6ff8, 0xc420d16200)",
"\t/usr/local/go/src/net/http/server.go:1947 +0x44",
"github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP.func1(0xc4204c61c0, 0x1fa8700, 0xc4200e6ff8, 0xc420d16200, 0xc4210a92c0)",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:97 +0x8d",
"created by github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters.(*timeoutHandler).ServeHTTP",
"\t/tmp/openshift/build-rpms/rpm/BUILD/origin-service-catalog-3.11.0/_output/local/go/src/github.com/kubernetes-incubator/service-catalog/vendor/k8s.io/apiserver/pkg/server/filters/timeout.go:96 +0x1b5",
"",
"logging error output: \"[-]etcd failed: reason withheld\\nhealthz check failed\\n\"",
" [[kube-probe/1.11+] 10.128.0.1:45422]"
]
}
TASK [openshift_service_catalog : Report errors] *******************************************************************************************************************************************************************************************************************************
fatal: [<IP_REMOVED>]: FAILED! => {"changed": false, "msg": "Catalog install failed."}
to retry, use: --limit @/home/user/installcentos/openshift-ansible/playbooks/deploy_cluster.retry
The /var/log/messages log file contains these errors repeated every 5 seconds:
Apr 18 11:19:30 myserver origin-node: I0418 11:19:30.480421 2281 prober.go:111] Readiness probe for "apiserver-66bsx_kube-service-catalog(819aff89-8150-11ea-89a7-a0369fa3e320):apiserver" failed (failure): HTTP probe failed with statuscode: 500
Apr 18 11:19:30 myserver origin-node: I0418 11:19:30.995262 2281 prober.go:111] Readiness probe for "controller-manager-jpssz_kube-service-catalog(818589f3-8150-11ea-89a7-a0369fa3e320):controller-manager" failed (failure): HTTP probe failed with statuscode: 500
Apr 18 11:19:35 myserver origin-node: I0418 11:19:35.483845 2281 prober.go:111] Readiness probe for "apiserver-66bsx_kube-service-catalog(819aff89-8150-11ea-89a7-a0369fa3e320):apiserver" failed (failure): HTTP probe failed with statuscode: 500
Apr 18 11:19:35 myserver origin-node: I0418 11:19:35.993962 2281 prober.go:111] Readiness probe for "controller-manager-jpssz_kube-service-catalog(818589f3-8150-11ea-89a7-a0369fa3e320):controller-manager" failed (failure): HTTP probe failed with statuscode: 500
Then the /usr/local/bin/master-logs api api command shows the following errors repeated also quite often:
E0418 11:15:33.399732 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:15:43.417797 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:15:53.436591 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:16:03.455844 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:16:13.474989 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:16:23.493941 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:16:33.512922 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:16:43.531735 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:16:53.551345 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:17:03.569631 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:17:13.596194 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
E0418 11:17:23.615032 1 memcache.go:147] couldn't get resource list for servicecatalog.k8s.io/v1beta1: the server is currently unable to handle the request
I0418 11:17:27.656105 1 controller.go:105] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io
E0418 11:17:27.658974 1 controller.go:111] loading OpenAPI spec for "v1beta1.metrics.k8s.io" failed with: OpenAPI spec does not exists
I0418 11:17:27.658985 1 controller.go:119] OpenAPI AggregationController: action for item v1beta1.metrics.k8s.io: Rate Limited Requeue.
I0418 11:17:28.650920 1 controller.go:105] OpenAPI AggregationController: Processing item v1beta1.servicecatalog.k8s.io
E0418 11:17:28.651017 1 controller.go:111] loading OpenAPI spec for "v1beta1.servicecatalog.k8s.io" failed with: failed to retrieve openAPI spec, http error: ResponseCode: 503, Body: service unavailable
, Header: map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff]]
I0418 11:17:28.651025 1 controller.go:119] OpenAPI AggregationController: action for item v1beta1.servicecatalog.k8s.io: Rate Limited Requeue.