(...snap...) TASK [openshift_control_plane : Check for apiservices/v1beta1.metrics.k8s.io registration] ************************************************* changed: [tmor-master.bc.jonqe.lab.eng.bos.redhat.com] TASK [openshift_control_plane : Wait for /apis/metrics.k8s.io/v1beta1 when registered] ***************************************************** skipping: [tmor-master.bc.jonqe.lab.eng.bos.redhat.com] TASK [openshift_control_plane : Check for apiservices/v1beta1.servicecatalog.k8s.io registration] ****************************************** changed: [tmor-master.bc.jonqe.lab.eng.bos.redhat.com] TASK [openshift_control_plane : Wait for /apis/servicecatalog.k8s.io/v1beta1 when registered] ********************************************** changed: [tmor-master.bc.jonqe.lab.eng.bos.redhat.com] TASK [Upgrade all storage] ***************************************************************************************************************** FAILED - RETRYING: Upgrade all storage (6 retries left). FAILED - RETRYING: Upgrade all storage (5 retries left). FAILED - RETRYING: Upgrade all storage (4 retries left). FAILED - RETRYING: Upgrade all storage (3 retries left). FAILED - RETRYING: Upgrade all storage (2 retries left). FAILED - RETRYING: Upgrade all storage (1 retries left). fatal: [tmor-master.bc.jonqe.lab.eng.bos.redhat.com]: FAILED! => {"attempts": 6, "changed": true, "cmd": ["oc", "adm", "--config=/etc/origin/master/admin.kubeconfig", "migrate", "storage", "--include=*"], "delta": "0:00:09.297087", "end": "2018-10-12 12:56:16.655080", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2018-10-12 12:56:07.357993", "stderr": "", "stderr_lines": [], "stdout": "E1012 12:56:13.751395 error: -n istio-system rules/stdio: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.756659 error: -n istio-system rules/promhttp: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.756711 error: -n istio-system rules/kubeattrgenrulerule: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.756852 error: -n istio-system rules/promtcp: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757038 error: -n istio-system prometheuses/handler: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757128 error: -n istio-system policies/disable-mtls-zipkin: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757246 error: -n istio-system policies/grafana-ports-mtls-disabled: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757337 error: -n istio-system policies/disable-mtls-jaeger-collector: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757557 error: -n istio-system metrics/tcpbytereceived: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757588 error: -n istio-system metrics/tcpbytesent: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757603 error: -n istio-system metrics/responsesize: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757672 error: -n istio-system metrics/requestduration: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757843 error: -n istio-system metrics/requestsize: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757870 error: -n istio-system metrics/requestcount: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.757932 error: -n istio-system logentries/tcpaccesslog: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.758106 error: -n istio-system destinationrules/istio-policy: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.764676 error: -n istio-system rules/tcpkubeattrgenrulerule: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.764766 error: -n istio-system rules/stdiotcp: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.787334 error: -n istio-system destinationrules/disable-mtls-jaeger-collector: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.788639 error: -n istio-system destinationrules/disable-mtls-zipkin: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.788706 error: -n istio-system attributemanifests/istioproxy: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.788835 error: -n istio-system attributemanifests/kubernetes: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.794334 error: -n istio-system logentries/accesslog: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.794368 error: -n istio-system kubernetesenvs/handler: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.794504 error: -n bookinfo gateways/bookinfo-gateway: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.794604 error: -n bookinfo destinationrules/details: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.794722 error: -n bookinfo destinationrules/ratings: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.794834 error: -n istio-system destinationrules/istio-telemetry: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.794960 error: -n bookinfo destinationrules/productpage: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:13.795080 error: -n bookinfo destinationrules/reviews: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:16.644966 error: -n bookinfo virtualservices/bookinfo: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nE1012 12:56:16.645008 error: -n istio-system stdios/handler: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host\nsummary: total=2296 errors=32 ignored=0 unchanged=2261 migrated=3\ninfo: to rerun only failing resources, add --include=attributemanifests,destinationrules,gateways,kubernetesenvs,logentries,metrics,policies,prometheuses,rules,stdios,virtualservices\nerror: 32 resources failed to migrate", "stdout_lines": ["E1012 12:56:13.751395 error: -n istio-system rules/stdio: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.756659 error: -n istio-system rules/promhttp: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.756711 error: -n istio-system rules/kubeattrgenrulerule: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.756852 error: -n istio-system rules/promtcp: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757038 error: -n istio-system prometheuses/handler: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757128 error: -n istio-system policies/disable-mtls-zipkin: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757246 error: -n istio-system policies/grafana-ports-mtls-disabled: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757337 error: -n istio-system policies/disable-mtls-jaeger-collector: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757557 error: -n istio-system metrics/tcpbytereceived: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757588 error: -n istio-system metrics/tcpbytesent: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757603 error: -n istio-system metrics/responsesize: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757672 error: -n istio-system metrics/requestduration: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757843 error: -n istio-system metrics/requestsize: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757870 error: -n istio-system metrics/requestcount: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.757932 error: -n istio-system logentries/tcpaccesslog: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.758106 error: -n istio-system destinationrules/istio-policy: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.764676 error: -n istio-system rules/tcpkubeattrgenrulerule: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.764766 error: -n istio-system rules/stdiotcp: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.787334 error: -n istio-system destinationrules/disable-mtls-jaeger-collector: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.788639 error: -n istio-system destinationrules/disable-mtls-zipkin: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.788706 error: -n istio-system attributemanifests/istioproxy: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.788835 error: -n istio-system attributemanifests/kubernetes: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.794334 error: -n istio-system logentries/accesslog: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.794368 error: -n istio-system kubernetesenvs/handler: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.794504 error: -n bookinfo gateways/bookinfo-gateway: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.794604 error: -n bookinfo destinationrules/details: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.794722 error: -n bookinfo destinationrules/ratings: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.794834 error: -n istio-system destinationrules/istio-telemetry: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.794960 error: -n bookinfo destinationrules/productpage: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:13.795080 error: -n bookinfo destinationrules/reviews: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:16.644966 error: -n bookinfo virtualservices/bookinfo: Internal error occurred: failed calling admission webhook \"pilot.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitpilot?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "E1012 12:56:16.645008 error: -n istio-system stdios/handler: Internal error occurred: failed calling admission webhook \"mixer.validation.istio.io\": Post https://istio-galley.istio-system.svc:443/admitmixer?timeout=30s: dial tcp 172.30.94.32:443: getsockopt: no route to host", "summary: total=2296 errors=32 ignored=0 unchanged=2261 migrated=3", "info: to rerun only failing resources, add --include=attributemanifests,destinationrules,gateways,kubernetesenvs,logentries,metrics,policies,prometheuses,rules,stdios,virtualservices", "error: 32 resources failed to migrate"]} to retry, use: --limit @/usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_11/upgrade.retry PLAY RECAP ********************************************************************************************************************************* localhost : ok=30 changed=0 unreachable=0 failed=0 tmor-master.bc.jonqe.lab.eng.bos.redhat.com : ok=238 changed=35 unreachable=0 failed=1 tmor-node1.bc.jonqe.lab.eng.bos.redhat.com : ok=21 changed=1 unreachable=0 failed=0 tmor-node2.bc.jonqe.lab.eng.bos.redhat.com : ok=21 changed=1 unreachable=0 failed=0 INSTALLER STATUS *************************************************************************************************************************** Initialization : Complete (0:02:39)