2022-03-08T10:40:01.518497279Z W0308 10:40:01.515965       1 etcd_env.go:298] cipher is not supported for use with etcd: "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256"
2022-03-08T10:40:01.518497279Z W0308 10:40:01.516103       1 etcd_env.go:298] cipher is not supported for use with etcd: "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256"
2022-03-08T10:40:01.518497279Z I0308 10:40:01.517413       1 observer_polling.go:52] Starting from specified content for file "/var/run/secrets/serving-cert/tls.crt"
2022-03-08T10:40:01.518497279Z I0308 10:40:01.517462       1 observer_polling.go:52] Starting from specified content for file "/var/run/secrets/serving-cert/tls.key"
2022-03-08T10:40:01.519964385Z I0308 10:40:01.518699       1 cmd.go:200] Using service-serving-cert provided certificates
2022-03-08T10:40:01.519964385Z I0308 10:40:01.518767       1 observer_polling.go:74] Adding reactor for file "/var/run/secrets/serving-cert/tls.crt"
2022-03-08T10:40:01.519964385Z I0308 10:40:01.518832       1 observer_polling.go:74] Adding reactor for file "/var/run/secrets/serving-cert/tls.key"
2022-03-08T10:40:01.519964385Z I0308 10:40:01.518877       1 observer_polling.go:52] Starting from specified content for file "/var/run/configmaps/config/config.yaml"
2022-03-08T10:40:01.526097138Z I0308 10:40:01.523097       1 observer_polling.go:159] Starting file observer
2022-03-08T10:40:01.526097138Z I0308 10:40:01.523180       1 observer_polling.go:159] Starting file observer
2022-03-08T10:40:01.526097138Z I0308 10:40:01.523221       1 observer_polling.go:135] File observer successfully synced
2022-03-08T10:40:01.526097138Z I0308 10:40:01.523350       1 observer_polling.go:135] File observer successfully synced
2022-03-08T10:40:01.580370922Z I0308 10:40:01.580294       1 builder.go:238] openshift-cluster-etcd-operator version v0.0.0-alpha.0-603-g6a3f3b50-6a3f3b506c5e824b40cbca78bcc8f8c0219b5b4a
2022-03-08T10:40:01.584851064Z I0308 10:40:01.583893       1 dynamic_serving_content.go:111] Loaded a new cert/key pair for "serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"
2022-03-08T10:40:01.946287704Z I0308 10:40:01.946214       1 requestheader_controller.go:244] Loaded a new request header values for RequestHeaderAuthRequestController
2022-03-08T10:40:01.951908654Z I0308 10:40:01.951867       1 config.go:638] Not requested to run hook priority-and-fairness-config-consumer
2022-03-08T10:40:01.956412082Z W0308 10:40:01.956377       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.
2022-03-08T10:40:01.961118624Z W0308 10:40:01.961046       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.
2022-03-08T10:40:01.962264253Z I0308 10:40:01.962219       1 leaderelection.go:243] attempting to acquire leader lease  openshift-etcd-operator/openshift-cluster-etcd-operator-lock...
2022-03-08T10:40:01.973451310Z I0308 10:40:01.971998       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-etcd-operator.svc" [serving] validServingFor=[metrics.openshift-etcd-operator.svc,metrics.openshift-etcd-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:21 +0000 UTC to 2024-03-07 10:39:22 +0000 UTC (now=2022-03-08 10:40:01.971975464 +0000 UTC))
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972231       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736001" (2022-03-08 09:40:01 +0000 UTC to 2023-03-08 09:40:01 +0000 UTC (now=2022-03-08 10:40:01.972222161 +0000 UTC))
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972254       1 secure_serving.go:197] Serving securely on [::]:8443
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972285       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972299       1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972324       1 dynamic_serving_content.go:130] Starting serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972346       1 tlsconfig.go:240] Starting DynamicServingCertificateController
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972427       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972433       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972443       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972450       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972506       1 reflector.go:207] Starting reflector *v1.ConfigMap (12h0m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972520       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972765       1 reflector.go:207] Starting reflector *v1.ConfigMap (12h0m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:01.973451310Z I0308 10:40:01.972773       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:01.973451310Z I0308 10:40:01.973024       1 reflector.go:207] Starting reflector *v1.ConfigMap (12h0m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:01.973451310Z I0308 10:40:01.973034       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:01.980725094Z I0308 10:40:01.980604       1 leaderelection.go:253] successfully acquired lease openshift-etcd-operator/openshift-cluster-etcd-operator-lock
2022-03-08T10:40:01.983660509Z I0308 10:40:01.983610       1 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"openshift-etcd-operator", Name:"openshift-cluster-etcd-operator-lock", UID:"09fdc970-c8c0-40be-92ab-e7e5cd5c6a6f", APIVersion:"v1", ResourceVersion:"9879", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' etcd-operator-859798c7f4-kb5lq_02b0f5d9-9c2c-41fc-ad32-5e15a246e928 became leader
2022-03-08T10:40:02.020393154Z I0308 10:40:02.019189       1 base_controller.go:66] Waiting for caches to sync for LoggingSyncer
2022-03-08T10:40:02.020393154Z I0308 10:40:02.019287       1 reflector.go:207] Starting reflector *v1.Etcd (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020393154Z I0308 10:40:02.019296       1 reflector.go:243] Listing and watching *v1.Etcd from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020393154Z I0308 10:40:02.019660       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020393154Z I0308 10:40:02.019677       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020393154Z I0308 10:40:02.019884       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020393154Z I0308 10:40:02.019891       1 reflector.go:243] Listing and watching *v1.Secret from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020393154Z I0308 10:40:02.020031       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020393154Z I0308 10:40:02.020040       1 reflector.go:243] Listing and watching *v1.Secret from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.020544666Z I0308 10:40:02.020519       1 base_controller.go:66] Waiting for caches to sync for ResourceSyncController
2022-03-08T10:40:02.021206551Z I0308 10:40:02.021180       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.021278547Z I0308 10:40:02.021264       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.021565858Z I0308 10:40:02.021544       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.021593349Z I0308 10:40:02.021586       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.021781166Z I0308 10:40:02.021765       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.021804827Z I0308 10:40:02.021798       1 reflector.go:243] Listing and watching *v1.Secret from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022002449Z I0308 10:40:02.021976       1 reflector.go:207] Starting reflector *v1.ClusterRoleBinding (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022002449Z I0308 10:40:02.021990       1 reflector.go:243] Listing and watching *v1.ClusterRoleBinding from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022041726Z I0308 10:40:02.022031       1 base_controller.go:66] Waiting for caches to sync for RemoveStaleConditionsController
2022-03-08T10:40:02.022189877Z I0308 10:40:02.022175       1 base_controller.go:66] Waiting for caches to sync for FSyncController
2022-03-08T10:40:02.022227974Z I0308 10:40:02.022208       1 reflector.go:207] Starting reflector *v1.Node (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022227974Z I0308 10:40:02.022218       1 reflector.go:243] Listing and watching *v1.Node from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022250655Z I0308 10:40:02.022209       1 shared_informer.go:270] caches populated
2022-03-08T10:40:02.022274128Z I0308 10:40:02.022265       1 base_controller.go:72] Caches are synced for FSyncController 
2022-03-08T10:40:02.022295674Z I0308 10:40:02.022288       1 base_controller.go:109] Starting #1 worker of FSyncController controller ...
2022-03-08T10:40:02.022388697Z I0308 10:40:02.022371       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022388697Z I0308 10:40:02.022383       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022599499Z I0308 10:40:02.022555       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022599499Z I0308 10:40:02.022565       1 reflector.go:243] Listing and watching *v1.Secret from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022700066Z I0308 10:40:02.022688       1 base_controller.go:66] Waiting for caches to sync for StaticResourceController
2022-03-08T10:40:02.022721134Z I0308 10:40:02.022709       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022721134Z I0308 10:40:02.022718       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022744405Z I0308 10:40:02.022737       1 base_controller.go:66] Waiting for caches to sync for TargetConfigController
2022-03-08T10:40:02.022774149Z I0308 10:40:02.022767       1 base_controller.go:66] Waiting for caches to sync for EtcdCertSignerController
2022-03-08T10:40:02.022807801Z I0308 10:40:02.022801       1 base_controller.go:66] Waiting for caches to sync for HostEndpoints2Controller
2022-03-08T10:40:02.022835800Z I0308 10:40:02.022829       1 base_controller.go:66] Waiting for caches to sync for EtcdEndpointsController
2022-03-08T10:40:02.022943014Z I0308 10:40:02.022925       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.022951523Z I0308 10:40:02.022941       1 reflector.go:243] Listing and watching *v1.Secret from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023097       1 reflector.go:207] Starting reflector *v1.Namespace (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023104       1 reflector.go:243] Listing and watching *v1.Namespace from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023161       1 reflector.go:207] Starting reflector *v1.ServiceAccount (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023166       1 reflector.go:243] Listing and watching *v1.ServiceAccount from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023213       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023218       1 reflector.go:243] Listing and watching *v1.ConfigMap from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023261       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023266       1 reflector.go:243] Listing and watching *v1.Secret from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023304       1 reflector.go:207] Starting reflector *v1.Endpoints (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023314748Z I0308 10:40:02.023308       1 reflector.go:243] Listing and watching *v1.Endpoints from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023388680Z I0308 10:40:02.023350       1 reflector.go:207] Starting reflector *v1.Pod (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023388680Z I0308 10:40:02.023358       1 reflector.go:243] Listing and watching *v1.Pod from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.023575864Z I0308 10:40:02.023560       1 base_controller.go:66] Waiting for caches to sync for UnsupportedConfigOverridesController
2022-03-08T10:40:02.024128841Z I0308 10:40:02.024096       1 base_controller.go:66] Waiting for caches to sync for StatusSyncer_etcd
2022-03-08T10:40:02.024360150Z I0308 10:40:02.024326       1 base_controller.go:66] Waiting for caches to sync for ConfigObserver
2022-03-08T10:40:02.024394280Z I0308 10:40:02.024383       1 base_controller.go:66] Waiting for caches to sync for ClusterMemberController
2022-03-08T10:40:02.024425968Z I0308 10:40:02.024418       1 base_controller.go:66] Waiting for caches to sync for EtcdMembersController
2022-03-08T10:40:02.024443194Z I0308 10:40:02.024437       1 shared_informer.go:270] caches populated
2022-03-08T10:40:02.024458731Z I0308 10:40:02.024452       1 base_controller.go:72] Caches are synced for EtcdMembersController 
2022-03-08T10:40:02.024487928Z I0308 10:40:02.024468       1 base_controller.go:109] Starting #1 worker of EtcdMembersController controller ...
2022-03-08T10:40:02.024554096Z I0308 10:40:02.024544       1 base_controller.go:66] Waiting for caches to sync for BootstrapTeardownController
2022-03-08T10:40:02.024580415Z I0308 10:40:02.024573       1 base_controller.go:66] Waiting for caches to sync for UnsupportedConfigOverridesController
2022-03-08T10:40:02.024626197Z I0308 10:40:02.024618       1 base_controller.go:66] Waiting for caches to sync for ScriptController
2022-03-08T10:40:02.024680296Z I0308 10:40:02.024672       1 envvarcontroller.go:161] Starting EnvVarController
2022-03-08T10:40:02.024708074Z I0308 10:40:02.024701       1 base_controller.go:66] Waiting for caches to sync for RevisionController
2022-03-08T10:40:02.024781270Z I0308 10:40:02.024773       1 base_controller.go:66] Waiting for caches to sync for InstallerStateController
2022-03-08T10:40:02.024804892Z I0308 10:40:02.024798       1 base_controller.go:66] Waiting for caches to sync for StaticPodStateController
2022-03-08T10:40:02.024827814Z I0308 10:40:02.024821       1 base_controller.go:66] Waiting for caches to sync for PruneController
2022-03-08T10:40:02.024851786Z I0308 10:40:02.024845       1 base_controller.go:66] Waiting for caches to sync for NodeController
2022-03-08T10:40:02.025213947Z I0308 10:40:02.025191       1 reflector.go:207] Starting reflector *v1.Infrastructure (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.025213947Z I0308 10:40:02.025202       1 reflector.go:243] Listing and watching *v1.Infrastructure from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.025373549Z I0308 10:40:02.025354       1 reflector.go:207] Starting reflector *v1.ClusterOperator (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.025373549Z I0308 10:40:02.025365       1 reflector.go:243] Listing and watching *v1.ClusterOperator from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.025505685Z I0308 10:40:02.025490       1 reflector.go:191] Reflector from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156 configured with expectedType of *unstructured.Unstructured with empty GroupVersionKind.
2022-03-08T10:40:02.025519913Z I0308 10:40:02.025502       1 reflector.go:207] Starting reflector *unstructured.Unstructured (12h0m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.025519913Z I0308 10:40:02.025508       1 reflector.go:243] Listing and watching *unstructured.Unstructured from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.025883717Z I0308 10:40:02.025855       1 reflector.go:207] Starting reflector *v1.Network (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.025883717Z I0308 10:40:02.025870       1 reflector.go:243] Listing and watching *v1.Network from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:40:02.026031227Z I0308 10:40:02.026011       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'ReportEtcdMembersErrorUpdatingStatus' etcds.operator.openshift.io "cluster" not found
2022-03-08T10:40:02.026344375Z I0308 10:40:02.026323       1 base_controller.go:66] Waiting for caches to sync for InstallerController
2022-03-08T10:40:02.027736991Z E0308 10:40:02.025055       1 base_controller.go:250] "EtcdMembersController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.028127572Z I0308 10:40:02.028108       1 base_controller.go:66] Waiting for caches to sync for StaticResourceController
2022-03-08T10:40:02.043203779Z E0308 10:40:02.041767       1 base_controller.go:250] "EtcdMembersController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.057153338Z E0308 10:40:02.056934       1 base_controller.go:250] "EtcdMembersController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.074108445Z I0308 10:40:02.073963       1 shared_informer.go:270] caches populated
2022-03-08T10:40:02.074108445Z I0308 10:40:02.073988       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file 
2022-03-08T10:40:02.074108445Z I0308 10:40:02.074029       1 shared_informer.go:270] caches populated
2022-03-08T10:40:02.074108445Z I0308 10:40:02.074034       1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController 
2022-03-08T10:40:02.074157778Z I0308 10:40:02.074145       1 shared_informer.go:270] caches populated
2022-03-08T10:40:02.074157778Z I0308 10:40:02.074150       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file 
2022-03-08T10:40:02.078112006Z I0308 10:40:02.077793       1 tlsconfig.go:178] loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "aggregator-signer" [] issuer="<self>" (2022-03-08 10:10:28 +0000 UTC to 2022-03-09 10:10:28 +0000 UTC (now=2022-03-08 10:40:02.077748219 +0000 UTC))
2022-03-08T10:40:02.078416803Z I0308 10:40:02.078393       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-etcd-operator.svc" [serving] validServingFor=[metrics.openshift-etcd-operator.svc,metrics.openshift-etcd-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:21 +0000 UTC to 2024-03-07 10:39:22 +0000 UTC (now=2022-03-08 10:40:02.078367481 +0000 UTC))
2022-03-08T10:40:02.078825216Z I0308 10:40:02.078798       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736001" (2022-03-08 09:40:01 +0000 UTC to 2023-03-08 09:40:01 +0000 UTC (now=2022-03-08 10:40:02.078786089 +0000 UTC))
2022-03-08T10:40:02.081099868Z I0308 10:40:02.079708       1 tlsconfig.go:178] loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "admin-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:26 +0000 UTC to 2032-03-05 10:10:26 +0000 UTC (now=2022-03-08 10:40:02.079690839 +0000 UTC))
2022-03-08T10:40:02.081197203Z I0308 10:40:02.081179       1 tlsconfig.go:178] loaded client CA [1/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:02.081156407 +0000 UTC))
2022-03-08T10:40:02.081232953Z I0308 10:40:02.081218       1 tlsconfig.go:178] loaded client CA [2/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-control-plane-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2023-03-08 10:10:30 +0000 UTC (now=2022-03-08 10:40:02.081211322 +0000 UTC))
2022-03-08T10:40:02.081270148Z I0308 10:40:02.081259       1 tlsconfig.go:178] loaded client CA [3/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-apiserver-to-kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:31 +0000 UTC to 2023-03-08 10:10:31 +0000 UTC (now=2022-03-08 10:40:02.08124505 +0000 UTC))
2022-03-08T10:40:02.081300968Z I0308 10:40:02.081289       1 tlsconfig.go:178] loaded client CA [4/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-bootstrap-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:27 +0000 UTC to 2032-03-05 10:10:27 +0000 UTC (now=2022-03-08 10:40:02.081282034 +0000 UTC))
2022-03-08T10:40:02.081333307Z I0308 10:40:02.081322       1 tlsconfig.go:178] loaded client CA [5/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "aggregator-signer" [] issuer="<self>" (2022-03-08 10:10:28 +0000 UTC to 2022-03-09 10:10:28 +0000 UTC (now=2022-03-08 10:40:02.081313037 +0000 UTC))
2022-03-08T10:40:02.081589698Z E0308 10:40:02.081568       1 base_controller.go:250] "EtcdMembersController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.081666217Z I0308 10:40:02.081653       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-etcd-operator.svc" [serving] validServingFor=[metrics.openshift-etcd-operator.svc,metrics.openshift-etcd-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:21 +0000 UTC to 2024-03-07 10:39:22 +0000 UTC (now=2022-03-08 10:40:02.081644056 +0000 UTC))
2022-03-08T10:40:02.081914498Z I0308 10:40:02.081898       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736001" (2022-03-08 09:40:01 +0000 UTC to 2023-03-08 09:40:01 +0000 UTC (now=2022-03-08 10:40:02.081889414 +0000 UTC))
2022-03-08T10:40:02.121134073Z I0308 10:40:02.119390       1 shared_informer.go:270] caches populated
2022-03-08T10:40:02.121134073Z I0308 10:40:02.119443       1 base_controller.go:72] Caches are synced for LoggingSyncer 
2022-03-08T10:40:02.121134073Z I0308 10:40:02.119454       1 base_controller.go:109] Starting #1 worker of LoggingSyncer controller ...
2022-03-08T10:40:02.121134073Z I0308 10:40:02.120384       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorLogLevelChange' Operator log level changed from "Debug" to "Normal"
2022-03-08T10:40:02.122478746Z E0308 10:40:02.122128       1 base_controller.go:250] "EtcdMembersController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.122478746Z I0308 10:40:02.122166       1 base_controller.go:72] Caches are synced for RemoveStaleConditionsController 
2022-03-08T10:40:02.122478746Z I0308 10:40:02.122171       1 base_controller.go:109] Starting #1 worker of RemoveStaleConditionsController controller ...
2022-03-08T10:40:02.125237020Z I0308 10:40:02.124395       1 base_controller.go:72] Caches are synced for UnsupportedConfigOverridesController 
2022-03-08T10:40:02.125237020Z I0308 10:40:02.124417       1 base_controller.go:109] Starting #1 worker of UnsupportedConfigOverridesController controller ...
2022-03-08T10:40:02.125237020Z I0308 10:40:02.125079       1 base_controller.go:72] Caches are synced for NodeController 
2022-03-08T10:40:02.125237020Z I0308 10:40:02.125172       1 base_controller.go:109] Starting #1 worker of NodeController controller ...
2022-03-08T10:40:02.125237020Z I0308 10:40:02.125183       1 base_controller.go:72] Caches are synced for BootstrapTeardownController 
2022-03-08T10:40:02.125237020Z I0308 10:40:02.125198       1 base_controller.go:109] Starting #1 worker of BootstrapTeardownController controller ...
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126386       1 base_controller.go:72] Caches are synced for PruneController 
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126405       1 base_controller.go:72] Caches are synced for UnsupportedConfigOverridesController 
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126416       1 base_controller.go:109] Starting #1 worker of UnsupportedConfigOverridesController controller ...
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126430       1 base_controller.go:109] Starting #1 worker of PruneController controller ...
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126723       1 prune_controller.go:134] no revision IDs currently eligible to prune
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126747       1 prune_controller.go:286] No excluded revisions to prune, skipping
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126777       1 base_controller.go:72] Caches are synced for StatusSyncer_etcd 
2022-03-08T10:40:02.127095148Z I0308 10:40:02.126784       1 base_controller.go:109] Starting #1 worker of StatusSyncer_etcd controller ...
2022-03-08T10:40:02.129250534Z I0308 10:40:02.127810       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.219:2379 https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.89.186:2379]: context deadline exceeded\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:02.137682016Z I0308 10:40:02.134703       1 prune_controller.go:134] no revision IDs currently eligible to prune
2022-03-08T10:40:02.137682016Z I0308 10:40:02.134721       1 prune_controller.go:286] No excluded revisions to prune, skipping
2022-03-08T10:40:02.137682016Z E0308 10:40:02.135686       1 base_controller.go:250] "BootstrapTeardownController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.137682016Z E0308 10:40:02.135887       1 base_controller.go:250] "BootstrapTeardownController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.137682016Z I0308 10:40:02.137198       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:02.138759419Z I0308 10:40:02.137909       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "EtcdMembersControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.219:2379 https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.89.186:2379]: context deadline exceeded\nBootstrapTeardownDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.219:2379 https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.89.186:2379]: context deadline exceeded\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready" to "EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.219:2379 https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.89.186:2379]: context deadline exceeded\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready"
2022-03-08T10:40:02.141145314Z E0308 10:40:02.140956       1 base_controller.go:250] "BootstrapTeardownController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.142178173Z I0308 10:40:02.141700       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:02.142178173Z I0308 10:40:02.141965       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "EtcdMembersControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.219:2379 https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.89.186:2379]: context deadline exceeded\nBootstrapTeardownDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.219:2379 https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.89.186:2379]: context deadline exceeded\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready" to "EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready"
2022-03-08T10:40:02.151673478Z I0308 10:40:02.149291       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.219:2379 https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.89.186:2379]: context deadline exceeded\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready" to "EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready"
2022-03-08T10:40:02.162086824Z E0308 10:40:02.161696       1 base_controller.go:250] "BootstrapTeardownController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.202601219Z E0308 10:40:02.202357       1 base_controller.go:250] "BootstrapTeardownController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.203102716Z E0308 10:40:02.202898       1 base_controller.go:250] "EtcdMembersController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.284454766Z E0308 10:40:02.284379       1 base_controller.go:250] "BootstrapTeardownController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.366131308Z E0308 10:40:02.364179       1 base_controller.go:250] "EtcdMembersController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.428865647Z I0308 10:40:02.422915       1 base_controller.go:72] Caches are synced for StaticResourceController 
2022-03-08T10:40:02.428865647Z I0308 10:40:02.422954       1 base_controller.go:109] Starting #1 worker of StaticResourceController controller ...
2022-03-08T10:40:02.428865647Z I0308 10:40:02.428517       1 base_controller.go:72] Caches are synced for StaticResourceController 
2022-03-08T10:40:02.428865647Z I0308 10:40:02.428542       1 base_controller.go:109] Starting #1 worker of StaticResourceController controller ...
2022-03-08T10:40:02.445930571Z E0308 10:40:02.445315       1 base_controller.go:250] "BootstrapTeardownController" controller failed to sync "key", err: node lister not synced
2022-03-08T10:40:02.622969704Z I0308 10:40:02.622903       1 base_controller.go:72] Caches are synced for EtcdEndpointsController 
2022-03-08T10:40:02.623056063Z I0308 10:40:02.623043       1 base_controller.go:109] Starting #1 worker of EtcdEndpointsController controller ...
2022-03-08T10:40:02.689435549Z WARNING: 2022/03/08 10:40:02 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:02.689678148Z WARNING: 2022/03/08 10:40:02 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:02.689750588Z WARNING: 2022/03/08 10:40:02 grpc: addrConn.createTransport failed to connect to {https://10.0.94.219:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.219:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:02.716080139Z I0308 10:40:02.716026       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:02.716389674Z WARNING: 2022/03/08 10:40:02 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:02.716612949Z WARNING: 2022/03/08 10:40:02 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:02.716756612Z WARNING: 2022/03/08 10:40:02 grpc: addrConn.createTransport failed to connect to {https://10.0.94.219:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.219:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:02.725327845Z I0308 10:40:02.725270       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready" to "EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:02.769597040Z I0308 10:40:02.769546       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"BootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:02.777631798Z I0308 10:40:02.777580       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "EtcdMembersControllerDegraded: node lister not synced\nBootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "BootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:02.841100967Z I0308 10:40:02.841033       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:02.851896992Z I0308 10:40:02.846730       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "BootstrapTeardownDegraded: node lister not synced\nStaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:02.922966365Z I0308 10:40:02.922904       1 base_controller.go:72] Caches are synced for EtcdCertSignerController 
2022-03-08T10:40:02.922966365Z I0308 10:40:02.922940       1 base_controller.go:109] Starting #1 worker of EtcdCertSignerController controller ...
2022-03-08T10:40:02.924772008Z I0308 10:40:02.924744       1 base_controller.go:72] Caches are synced for RevisionController 
2022-03-08T10:40:02.924772008Z I0308 10:40:02.924765       1 base_controller.go:109] Starting #1 worker of RevisionController controller ...
2022-03-08T10:40:03.022819145Z I0308 10:40:03.022779       1 base_controller.go:72] Caches are synced for TargetConfigController 
2022-03-08T10:40:03.022881686Z I0308 10:40:03.022871       1 base_controller.go:109] Starting #1 worker of TargetConfigController controller ...
2022-03-08T10:40:03.022916263Z I0308 10:40:03.022899       1 base_controller.go:72] Caches are synced for HostEndpoints2Controller 
2022-03-08T10:40:03.022923665Z I0308 10:40:03.022915       1 base_controller.go:109] Starting #1 worker of HostEndpoints2Controller controller ...
2022-03-08T10:40:03.023167518Z E0308 10:40:03.023155       1 base_controller.go:250] "TargetConfigController" controller failed to sync "key", err: missing env var values
2022-03-08T10:40:03.024779516Z I0308 10:40:03.024734       1 envvarcontroller.go:167] caches synced
2022-03-08T10:40:03.024836932Z I0308 10:40:03.024762       1 base_controller.go:72] Caches are synced for ScriptController 
2022-03-08T10:40:03.024862282Z I0308 10:40:03.024853       1 base_controller.go:109] Starting #1 worker of ScriptController controller ...
2022-03-08T10:40:03.219948127Z I0308 10:40:03.219860       1 request.go:645] Throttling request took 1.196439501s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?limit=500&resourceVersion=0
2022-03-08T10:40:03.224527186Z I0308 10:40:03.224442       1 base_controller.go:72] Caches are synced for ConfigObserver 
2022-03-08T10:40:03.224706344Z I0308 10:40:03.224689       1 base_controller.go:109] Starting #1 worker of ConfigObserver controller ...
2022-03-08T10:40:03.224742203Z I0308 10:40:03.224722       1 base_controller.go:72] Caches are synced for ClusterMemberController 
2022-03-08T10:40:03.224759089Z I0308 10:40:03.224740       1 base_controller.go:109] Starting #1 worker of ClusterMemberController controller ...
2022-03-08T10:40:03.225003546Z I0308 10:40:03.224984       1 base_controller.go:72] Caches are synced for StaticPodStateController 
2022-03-08T10:40:03.225003546Z I0308 10:40:03.224994       1 base_controller.go:109] Starting #1 worker of StaticPodStateController controller ...
2022-03-08T10:40:03.225013427Z I0308 10:40:03.225006       1 base_controller.go:72] Caches are synced for InstallerStateController 
2022-03-08T10:40:03.225013427Z I0308 10:40:03.225010       1 base_controller.go:109] Starting #1 worker of InstallerStateController controller ...
2022-03-08T10:40:03.226531223Z I0308 10:40:03.226423       1 base_controller.go:72] Caches are synced for InstallerController 
2022-03-08T10:40:03.226592700Z I0308 10:40:03.226575       1 base_controller.go:109] Starting #1 worker of InstallerController controller ...
2022-03-08T10:40:03.238486535Z I0308 10:40:03.238429       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'MemberAdd' adding new peer https://10.0.94.219:2380
2022-03-08T10:40:03.238552831Z I0308 10:40:03.238539       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FoundPodToScale' found pod to add to etcd membership: etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:03.239432517Z WARNING: 2022/03/08 10:40:03 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:03.239785273Z WARNING: 2022/03/08 10:40:03 grpc: addrConn.createTransport failed to connect to {https://10.0.94.219:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.219:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:03.239852011Z WARNING: 2022/03/08 10:40:03 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:03.521427312Z I0308 10:40:03.521326       1 base_controller.go:72] Caches are synced for ResourceSyncController 
2022-03-08T10:40:03.521538895Z I0308 10:40:03.521523       1 base_controller.go:109] Starting #1 worker of ResourceSyncController controller ...
2022-03-08T10:40:03.802819091Z WARNING: 2022/03/08 10:40:03 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:03.803790077Z I0308 10:40:03.803751       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 2","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:03.804249201Z WARNING: 2022/03/08 10:40:03 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:03.837303177Z I0308 10:40:03.837244       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nClusterMemberControllerDegraded: failed to get etcd client: failed to make etcd client for endpoints [https://10.0.94.119:2379 https://10.0.95.76:2379 https://10.0.94.219:2379 https://10.0.89.186:2379]: context deadline exceeded\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:03.839459116Z E0308 10:40:03.839403       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:04.220007131Z I0308 10:40:04.219917       1 request.go:645] Throttling request took 1.194587827s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-scripts
2022-03-08T10:40:04.803487883Z WARNING: 2022/03/08 10:40:04 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:04.805088621Z WARNING: 2022/03/08 10:40:04 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:05.028710906Z I0308 10:40:05.028625       1 clustermembercontroller.go:170] skipping unready pod "etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com" because it is already an etcd member: &etcdserverpb.Member{ID:0x827eb816f33178d7, Name:"master-2.case03116926.lab.upshift.rdu2.redhat.com", PeerURLs:[]string{"https://10.0.94.219:2380"}, ClientURLs:[]string{"https://10.0.94.219:2379"}, IsLearner:false}
2022-03-08T10:40:05.228094285Z E0308 10:40:05.228003       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:06.219977599Z I0308 10:40:06.219922       1 request.go:645] Throttling request took 1.181609201s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-scripts
2022-03-08T10:40:06.264601626Z WARNING: 2022/03/08 10:40:06 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:06.408287061Z WARNING: 2022/03/08 10:40:06 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:06.624934254Z E0308 10:40:06.624875       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:07.228461886Z I0308 10:40:07.228363       1 installer_controller.go:318] "master-2.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:07.228461886Z  NodeName: (string) (len=49) "master-2.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:07.228461886Z  CurrentRevision: (int32) 2,
2022-03-08T10:40:07.228461886Z  TargetRevision: (int32) 0,
2022-03-08T10:40:07.228461886Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:07.228461886Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:07.228461886Z }
2022-03-08T10:40:07.228461886Z  because static pod is ready
2022-03-08T10:40:07.242237827Z I0308 10:40:07.242173       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:07.247809380Z I0308 10:40:07.247729       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeCurrentRevisionChanged' Updated node "master-2.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 2 because static pod is ready
2022-03-08T10:40:07.259529918Z I0308 10:40:07.259464       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Progressing message changed from "NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 2" to "NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2",Available changed from False to True ("StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available")
2022-03-08T10:40:07.420028909Z I0308 10:40:07.419979       1 request.go:645] Throttling request took 1.597020366s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/restore-etcd-pod
2022-03-08T10:40:07.642442954Z I0308 10:40:07.642396       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:07.653116530Z I0308 10:40:07.653041       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: context canceled\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:08.025417445Z E0308 10:40:08.025357       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:08.357274981Z WARNING: 2022/03/08 10:40:08 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:08.359632516Z WARNING: 2022/03/08 10:40:08 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:08.394829200Z WARNING: 2022/03/08 10:40:08 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:08.396153851Z WARNING: 2022/03/08 10:40:08 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:08.420856817Z I0308 10:40:08.420771       1 request.go:645] Throttling request took 1.180098313s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/serviceaccounts/installer-sa
2022-03-08T10:40:09.395397408Z WARNING: 2022/03/08 10:40:09 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:09.395884169Z WARNING: 2022/03/08 10:40:09 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:09.619940795Z I0308 10:40:09.619844       1 request.go:645] Throttling request took 1.554004077s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-monitoring/secrets
2022-03-08T10:40:09.624110745Z E0308 10:40:09.624011       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:10.108209927Z WARNING: 2022/03/08 10:40:10 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:10.108286605Z WARNING: 2022/03/08 10:40:10 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:10.432518421Z I0308 10:40:10.432434       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/revision-pruner-2-master-2.case03116926.lab.upshift.rdu2.redhat.com -n openshift-etcd because it was missing
2022-03-08T10:40:10.434950582Z WARNING: 2022/03/08 10:40:10 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:10.434950582Z WARNING: 2022/03/08 10:40:10 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:10.620044874Z I0308 10:40:10.619941       1 request.go:645] Throttling request took 1.795896659s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:10.622387819Z I0308 10:40:10.622349       1 installer_controller.go:350] node master-0.case03116926.lab.upshift.rdu2.redhat.com static pod not found and needs new revision 2
2022-03-08T10:40:10.622515023Z I0308 10:40:10.622463       1 installer_controller.go:359] "master-0.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:10.622515023Z  NodeName: (string) (len=49) "master-0.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:10.622515023Z  CurrentRevision: (int32) 0,
2022-03-08T10:40:10.622515023Z  TargetRevision: (int32) 2,
2022-03-08T10:40:10.622515023Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:10.622515023Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:10.622515023Z }
2022-03-08T10:40:10.637743118Z I0308 10:40:10.637679       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeTargetRevisionChanged' Updating node "master-0.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 2 because node master-0.case03116926.lab.upshift.rdu2.redhat.com static pod not found
2022-03-08T10:40:11.225657791Z E0308 10:40:11.225574       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:11.435899636Z WARNING: 2022/03/08 10:40:11 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:11.435899636Z WARNING: 2022/03/08 10:40:11 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:11.820007698Z I0308 10:40:11.819929       1 request.go:645] Throttling request took 1.387475954s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/revision-pruner-2-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:12.707411476Z WARNING: 2022/03/08 10:40:12 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:12.707589729Z WARNING: 2022/03/08 10:40:12 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:12.847813949Z I0308 10:40:12.847754       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:12.848672907Z WARNING: 2022/03/08 10:40:12 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:12.849029030Z WARNING: 2022/03/08 10:40:12 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:12.867104225Z I0308 10:40:12.867029       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pod/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:39:53 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:13.019968626Z I0308 10:40:13.019881       1 request.go:645] Throttling request took 1.633799593s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-monitoring/secrets
2022-03-08T10:40:13.025361381Z E0308 10:40:13.025302       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:13.831121483Z I0308 10:40:13.829270       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/installer-2-master-0.case03116926.lab.upshift.rdu2.redhat.com -n openshift-etcd because it was missing
2022-03-08T10:40:13.849004991Z WARNING: 2022/03/08 10:40:13 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:13.849547879Z WARNING: 2022/03/08 10:40:13 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:14.219947494Z I0308 10:40:14.219875       1 request.go:645] Throttling request took 1.385393102s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:15.025163848Z E0308 10:40:15.024941       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:15.219949625Z I0308 10:40:15.219888       1 request.go:645] Throttling request took 1.796805763s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:15.598328722Z WARNING: 2022/03/08 10:40:15 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:15.622925238Z I0308 10:40:15.622851       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:15.749852523Z WARNING: 2022/03/08 10:40:15 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:16.420138901Z I0308 10:40:16.420066       1 request.go:645] Throttling request took 1.193700868s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:16.824497653Z E0308 10:40:16.824434       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:17.295078856Z WARNING: 2022/03/08 10:40:17 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:17.295671799Z WARNING: 2022/03/08 10:40:17 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:17.347496221Z WARNING: 2022/03/08 10:40:17 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:17.347594829Z WARNING: 2022/03/08 10:40:17 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:17.620025922Z I0308 10:40:17.619955       1 request.go:645] Throttling request took 1.18794374s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:17.823363053Z I0308 10:40:17.823273       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:18.350212885Z WARNING: 2022/03/08 10:40:18 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:18.351198560Z WARNING: 2022/03/08 10:40:18 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:19.024008911Z E0308 10:40:19.023931       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:19.420011363Z I0308 10:40:19.419934       1 request.go:645] Throttling request took 1.079900345s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:19.856247897Z I0308 10:40:19.856202       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:19.935315846Z WARNING: 2022/03/08 10:40:19 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:20.048456977Z WARNING: 2022/03/08 10:40:20 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:21.234154476Z I0308 10:40:21.227031       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:22.025256285Z WARNING: 2022/03/08 10:40:22 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:22.423267392Z E0308 10:40:22.423207       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:22.650935647Z I0308 10:40:22.650879       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:22.651571393Z WARNING: 2022/03/08 10:40:22 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:22.652205210Z WARNING: 2022/03/08 10:40:22 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:22.667586183Z I0308 10:40:22.667526       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pods \"etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:23.174410312Z WARNING: 2022/03/08 10:40:23 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:23.174410312Z WARNING: 2022/03/08 10:40:23 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:23.820000874Z I0308 10:40:23.819897       1 request.go:645] Throttling request took 1.171219687s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-scripts
2022-03-08T10:40:24.174160555Z WARNING: 2022/03/08 10:40:24 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:24.174605706Z WARNING: 2022/03/08 10:40:24 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:24.443157611Z I0308 10:40:24.443119       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because static pod is pending
2022-03-08T10:40:24.820107170Z I0308 10:40:24.819969       1 request.go:645] Throttling request took 1.196719648s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:25.832480672Z WARNING: 2022/03/08 10:40:25 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:25.889009328Z WARNING: 2022/03/08 10:40:25 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:26.019911702Z I0308 10:40:26.019841       1 request.go:645] Throttling request took 1.195996038s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:26.047850459Z WARNING: 2022/03/08 10:40:26 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:26.048340062Z I0308 10:40:26.048289       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd-metrics\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcdctl\" is not ready: PodInitializing: \nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:26.048524669Z WARNING: 2022/03/08 10:40:26 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:26.060225912Z I0308 10:40:26.060132       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd-metrics\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcdctl\" is not ready: PodInitializing: \nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:26.129663495Z WARNING: 2022/03/08 10:40:26 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:26.129980492Z WARNING: 2022/03/08 10:40:26 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:27.129878459Z WARNING: 2022/03/08 10:40:27 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.94.119:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:27.131497131Z WARNING: 2022/03/08 10:40:27 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:27.154729137Z I0308 10:40:27.154669       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FoundPodToScale' found pod to add to etcd membership: etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:27.155100176Z I0308 10:40:27.155050       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'MemberAdd' adding new peer https://10.0.94.119:2380
2022-03-08T10:40:27.219973300Z I0308 10:40:27.219905       1 request.go:645] Throttling request took 1.180480125s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/serviceaccounts/installer-sa
2022-03-08T10:40:28.262568203Z I0308 10:40:28.260216       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because static pod is pending
2022-03-08T10:40:28.423702159Z I0308 10:40:28.419976       1 request.go:645] Throttling request took 1.592090858s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod
2022-03-08T10:40:28.668386519Z WARNING: 2022/03/08 10:40:28 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:28.886084564Z E0308 10:40:28.882671       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:29.421668262Z I0308 10:40:29.421183       1 request.go:645] Throttling request took 1.159173077s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/installer-2-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:29.840704066Z I0308 10:40:29.840657       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:26 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:29.855509204Z I0308 10:40:29.855407       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd-metrics\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcdctl\" is not ready: PodInitializing: \nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:26 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:30.620010699Z I0308 10:40:30.619942       1 request.go:645] Throttling request took 1.195691939s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/installer-2-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:31.136236200Z WARNING: 2022/03/08 10:40:31 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:31.819972994Z I0308 10:40:31.819891       1 request.go:645] Throttling request took 1.596311652s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-scripts
2022-03-08T10:40:32.026836426Z I0308 10:40:32.026776       1 installer_controller.go:318] "master-0.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:32.026836426Z  NodeName: (string) (len=49) "master-0.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:32.026836426Z  CurrentRevision: (int32) 2,
2022-03-08T10:40:32.026836426Z  TargetRevision: (int32) 0,
2022-03-08T10:40:32.026836426Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:32.026836426Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:32.026836426Z }
2022-03-08T10:40:32.026836426Z  because static pod is ready
2022-03-08T10:40:32.041283020Z I0308 10:40:32.041222       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeCurrentRevisionChanged' Updated node "master-0.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 2 because static pod is ready
2022-03-08T10:40:32.057932490Z I0308 10:40:32.057876       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:26 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 1 nodes are at revision 0; 2 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:32.072996888Z I0308 10:40:32.067001       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Progressing message changed from "NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 2" to "NodeInstallerProgressing: 1 nodes are at revision 0; 2 nodes are at revision 2",Available message changed from "StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available" to "StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available"
2022-03-08T10:40:33.220230255Z I0308 10:40:33.220140       1 request.go:645] Throttling request took 1.174376154s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/revision-pruner-2-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:34.041530999Z I0308 10:40:34.041469       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 1 nodes are at revision 0; 2 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:34.046185296Z WARNING: 2022/03/08 10:40:34 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:34.060412305Z I0308 10:40:34.060352       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pod/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:26 +0000 UTC but not ready: unknown reason\nStaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:34.066024772Z WARNING: 2022/03/08 10:40:34 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:34.419930257Z I0308 10:40:34.419863       1 request.go:645] Throttling request took 1.597316395s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/installer-2-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:35.067282149Z WARNING: 2022/03/08 10:40:35 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:35.266253549Z WARNING: 2022/03/08 10:40:35 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:35.619929519Z I0308 10:40:35.619875       1 request.go:645] Throttling request took 1.397185s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd
2022-03-08T10:40:35.658098409Z WARNING: 2022/03/08 10:40:35 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:35.824812145Z I0308 10:40:35.824713       1 installer_controller.go:318] "master-0.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:35.824812145Z  NodeName: (string) (len=49) "master-0.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:35.824812145Z  CurrentRevision: (int32) 2,
2022-03-08T10:40:35.824812145Z  TargetRevision: (int32) 0,
2022-03-08T10:40:35.824812145Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:35.824812145Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:35.824812145Z }
2022-03-08T10:40:35.824812145Z  because static pod is ready
2022-03-08T10:40:36.039284462Z I0308 10:40:36.033969       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/revision-pruner-2-master-0.case03116926.lab.upshift.rdu2.redhat.com -n openshift-etcd because it was missing
2022-03-08T10:40:36.659287504Z WARNING: 2022/03/08 10:40:36 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:37.220320469Z I0308 10:40:37.220252       1 request.go:645] Throttling request took 1.186282896s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:38.255265200Z WARNING: 2022/03/08 10:40:38 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:38.420014215Z I0308 10:40:38.419905       1 request.go:645] Throttling request took 1.196088674s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:39.022836513Z I0308 10:40:39.022763       1 installer_controller.go:350] node master-1.case03116926.lab.upshift.rdu2.redhat.com static pod not found and needs new revision 2
2022-03-08T10:40:39.022899891Z I0308 10:40:39.022830       1 installer_controller.go:359] "master-1.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:39.022899891Z  NodeName: (string) (len=49) "master-1.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:39.022899891Z  CurrentRevision: (int32) 0,
2022-03-08T10:40:39.022899891Z  TargetRevision: (int32) 2,
2022-03-08T10:40:39.022899891Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:39.022899891Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:39.022899891Z }
2022-03-08T10:40:39.037715435Z I0308 10:40:39.037633       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeTargetRevisionChanged' Updating node "master-1.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 2 because node master-1.case03116926.lab.upshift.rdu2.redhat.com static pod not found
2022-03-08T10:40:40.219940292Z I0308 10:40:40.219822       1 request.go:645] Throttling request took 1.181111993s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/revision-pruner-2-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:40.664282531Z WARNING: 2022/03/08 10:40:40 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:40.824486767Z E0308 10:40:40.824420       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:40:41.219957951Z I0308 10:40:41.219860       1 request.go:645] Throttling request took 1.596056714s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:41.436200899Z I0308 10:40:41.436138       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/installer-2-master-1.case03116926.lab.upshift.rdu2.redhat.com -n openshift-etcd because it was missing
2022-03-08T10:40:42.419992486Z I0308 10:40:42.419857       1 request.go:645] Throttling request took 1.177504509s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:42.622813348Z I0308 10:40:42.622744       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:44.259202415Z WARNING: 2022/03/08 10:40:44 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:44.307251948Z WARNING: 2022/03/08 10:40:44 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:44.347542771Z WARNING: 2022/03/08 10:40:44 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:44.422528377Z I0308 10:40:44.422414       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:45.348628928Z WARNING: 2022/03/08 10:40:45 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:45.423640592Z I0308 10:40:45.423568       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:46.812542436Z WARNING: 2022/03/08 10:40:46 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:47.315874446Z WARNING: 2022/03/08 10:40:47 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:47.356573529Z WARNING: 2022/03/08 10:40:47 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:47.399779622Z WARNING: 2022/03/08 10:40:47 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:48.223516008Z I0308 10:40:48.223410       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:48.400652771Z WARNING: 2022/03/08 10:40:48 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:49.728802749Z WARNING: 2022/03/08 10:40:49 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:49.781264966Z I0308 10:40:49.781191       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because 
2022-03-08T10:40:49.854361376Z I0308 10:40:49.854286       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"NodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 1 nodes are at revision 0; 2 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:49.871249581Z I0308 10:40:49.871189       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pods \"etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "NodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:50.592662803Z WARNING: 2022/03/08 10:40:50 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:51.019900321Z I0308 10:40:51.019804       1 request.go:645] Throttling request took 1.162831459s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/config
2022-03-08T10:40:51.594274728Z WARNING: 2022/03/08 10:40:51 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:52.019922657Z I0308 10:40:52.019864       1 request.go:645] Throttling request took 1.195140458s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/revision-pruner-2-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:52.822905709Z I0308 10:40:52.822823       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because static pod is pending
2022-03-08T10:40:53.338195941Z WARNING: 2022/03/08 10:40:53 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:54.222607447Z I0308 10:40:54.222504       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because static pod is pending
2022-03-08T10:40:54.336876798Z WARNING: 2022/03/08 10:40:54 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:54.451499902Z I0308 10:40:54.451431       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd-metrics\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcdctl\" is not ready: PodInitializing: \nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 1 nodes are at revision 0; 2 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:54.466055158Z I0308 10:40:54.465978       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "NodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd-metrics\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcdctl\" is not ready: PodInitializing: \nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:54.643140032Z WARNING: 2022/03/08 10:40:54 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:55.619970260Z I0308 10:40:55.619895       1 request.go:645] Throttling request took 1.172806173s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:55.644050505Z WARNING: 2022/03/08 10:40:55 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:55.647550887Z WARNING: 2022/03/08 10:40:55 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:56.342947333Z WARNING: 2022/03/08 10:40:56 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:56.619951253Z I0308 10:40:56.619893       1 request.go:645] Throttling request took 1.596475163s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod
2022-03-08T10:40:56.649134945Z WARNING: 2022/03/08 10:40:56 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:56.673525870Z I0308 10:40:56.673393       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'FoundPodToScale' found pod to add to etcd membership: etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:56.674181226Z I0308 10:40:56.674100       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'MemberAdd' adding new peer https://10.0.95.76:2380
2022-03-08T10:40:56.678508269Z WARNING: 2022/03/08 10:40:56 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.95.76:2379: connect: connection refused". Reconnecting...
2022-03-08T10:40:57.601795844Z I0308 10:40:57.601733       1 tlsconfig.go:178] loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "admin-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:26 +0000 UTC to 2032-03-05 10:10:26 +0000 UTC (now=2022-03-08 10:40:57.601717502 +0000 UTC))
2022-03-08T10:40:57.601795844Z I0308 10:40:57.601775       1 tlsconfig.go:178] loaded client CA [1/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.601758454 +0000 UTC))
2022-03-08T10:40:57.601832311Z I0308 10:40:57.601788       1 tlsconfig.go:178] loaded client CA [2/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-control-plane-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2023-03-08 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.601783052 +0000 UTC))
2022-03-08T10:40:57.601832311Z I0308 10:40:57.601800       1 tlsconfig.go:178] loaded client CA [3/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-apiserver-to-kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:31 +0000 UTC to 2023-03-08 10:10:31 +0000 UTC (now=2022-03-08 10:40:57.601795038 +0000 UTC))
2022-03-08T10:40:57.601832311Z I0308 10:40:57.601814       1 tlsconfig.go:178] loaded client CA [4/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-bootstrap-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:27 +0000 UTC to 2032-03-05 10:10:27 +0000 UTC (now=2022-03-08 10:40:57.601809087 +0000 UTC))
2022-03-08T10:40:57.601832311Z I0308 10:40:57.601826       1 tlsconfig.go:178] loaded client CA [5/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-csr-signer_@1646735926" [] issuer="kubelet-signer" (2022-03-08 10:38:45 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.601821336 +0000 UTC))
2022-03-08T10:40:57.601883685Z I0308 10:40:57.601837       1 tlsconfig.go:178] loaded client CA [6/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "openshift-kube-apiserver-operator_node-system-admin-signer@1646735931" [] issuer="<self>" (2022-03-08 10:38:50 +0000 UTC to 2023-03-08 10:38:51 +0000 UTC (now=2022-03-08 10:40:57.601831615 +0000 UTC))
2022-03-08T10:40:57.601883685Z I0308 10:40:57.601858       1 tlsconfig.go:178] loaded client CA [7/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "aggregator-signer" [] issuer="<self>" (2022-03-08 10:10:28 +0000 UTC to 2022-03-09 10:10:28 +0000 UTC (now=2022-03-08 10:40:57.601853765 +0000 UTC))
2022-03-08T10:40:57.602099042Z I0308 10:40:57.602078       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-etcd-operator.svc" [serving] validServingFor=[metrics.openshift-etcd-operator.svc,metrics.openshift-etcd-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:21 +0000 UTC to 2024-03-07 10:39:22 +0000 UTC (now=2022-03-08 10:40:57.602055656 +0000 UTC))
2022-03-08T10:40:57.602275568Z I0308 10:40:57.602259       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736001" (2022-03-08 09:40:01 +0000 UTC to 2023-03-08 09:40:01 +0000 UTC (now=2022-03-08 10:40:57.602252386 +0000 UTC))
2022-03-08T10:40:57.620850774Z I0308 10:40:57.620549       1 request.go:645] Throttling request took 1.393590176s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:57.823673045Z I0308 10:40:57.823602       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because static pod is pending
2022-03-08T10:40:58.445112497Z I0308 10:40:58.445039       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:56 +0000 UTC but not ready: unknown reason\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 1 nodes are at revision 0; 2 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:58.460859789Z I0308 10:40:58.460791       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd-metrics\" is not ready: PodInitializing: \nStaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcdctl\" is not ready: PodInitializing: \nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:56 +0000 UTC but not ready: unknown reason\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:40:59.375856852Z WARNING: 2022/03/08 10:40:59 grpc: addrConn.createTransport failed to connect to {https://10.0.95.76:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: context canceled". Reconnecting...
2022-03-08T10:40:59.376647164Z I0308 10:40:59.376563       1 clustermembercontroller.go:170] skipping unready pod "etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com" because it is already an etcd member: &etcdserverpb.Member{ID:0xb983e031d3964a8, Name:"master-1.case03116926.lab.upshift.rdu2.redhat.com", PeerURLs:[]string{"https://10.0.95.76:2380"}, ClientURLs:[]string{"https://10.0.95.76:2379"}, IsLearner:false}
2022-03-08T10:40:59.379458621Z I0308 10:40:59.379405       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:40:59.416032856Z I0308 10:40:59.415978       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:40:59.457961096Z I0308 10:40:59.457889       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:40:59.459675272Z I0308 10:40:59.459634       1 clustermembercontroller.go:170] skipping unready pod "etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com" because it is already an etcd member: &etcdserverpb.Member{ID:0xb983e031d3964a8, Name:"master-1.case03116926.lab.upshift.rdu2.redhat.com", PeerURLs:[]string{"https://10.0.95.76:2380"}, ClientURLs:[]string{"https://10.0.95.76:2379"}, IsLearner:false}
2022-03-08T10:40:59.619913940Z I0308 10:40:59.619811       1 request.go:645] Throttling request took 1.178991699s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-scripts
2022-03-08T10:41:00.620035520Z I0308 10:41:00.619917       1 request.go:645] Throttling request took 1.396113784s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:01.045260490Z I0308 10:41:01.045193       1 clustermembercontroller.go:170] skipping unready pod "etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com" because it is already an etcd member: &etcdserverpb.Member{ID:0xb983e031d3964a8, Name:"master-1.case03116926.lab.upshift.rdu2.redhat.com", PeerURLs:[]string{"https://10.0.95.76:2380"}, ClientURLs:[]string{"https://10.0.95.76:2379"}, IsLearner:false}
2022-03-08T10:41:01.621015601Z I0308 10:41:01.620010       1 request.go:645] Throttling request took 1.396387322s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:01.625578347Z I0308 10:41:01.625492       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 2, but has not made progress because static pod is pending
2022-03-08T10:41:02.064079329Z I0308 10:41:02.064017       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:56 +0000 UTC but not ready: unknown reason\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:39:14Z","message":"NodeInstallerProgressing: 1 nodes are at revision 0; 2 nodes are at revision 2","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 4 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:02.082567668Z I0308 10:41:02.082452       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Available message changed from "StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 1 members are available" to "StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 4 members are available"
2022-03-08T10:41:02.100641899Z I0308 10:41:02.100559       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:02.116999472Z I0308 10:41:02.116892       1 clustermembercontroller.go:170] skipping unready pod "etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com" because it is already an etcd member: &etcdserverpb.Member{ID:0xb983e031d3964a8, Name:"master-1.case03116926.lab.upshift.rdu2.redhat.com", PeerURLs:[]string{"https://10.0.95.76:2380"}, ClientURLs:[]string{"https://10.0.95.76:2379"}, IsLearner:false}
2022-03-08T10:41:02.194653835Z I0308 10:41:02.194576       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:02.440031430Z I0308 10:41:02.439942       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:02.625018070Z E0308 10:41:02.624824       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:41:02.819960418Z I0308 10:41:02.819906       1 request.go:645] Throttling request took 1.394673876s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/revision-pruner-2-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:04.019993831Z I0308 10:41:04.019852       1 request.go:645] Throttling request took 1.578602617s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd
2022-03-08T10:41:04.425642720Z E0308 10:41:04.425578       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:41:05.020548146Z I0308 10:41:05.020332       1 request.go:645] Throttling request took 1.796032725s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod
2022-03-08T10:41:05.403074091Z I0308 10:41:05.402986       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:06.219990423Z I0308 10:41:06.219910       1 request.go:645] Throttling request took 1.594567999s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/revision-pruner-2-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:06.423453550Z I0308 10:41:06.423381       1 installer_controller.go:318] "master-1.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:41:06.423453550Z  NodeName: (string) (len=49) "master-1.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:41:06.423453550Z  CurrentRevision: (int32) 2,
2022-03-08T10:41:06.423453550Z  TargetRevision: (int32) 0,
2022-03-08T10:41:06.423453550Z  LastFailedRevision: (int32) 0,
2022-03-08T10:41:06.423453550Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:41:06.423453550Z }
2022-03-08T10:41:06.423453550Z  because static pod is ready
2022-03-08T10:41:06.435147659Z I0308 10:41:06.435032       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeCurrentRevisionChanged' Updated node "master-1.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 2 because static pod is ready
2022-03-08T10:41:06.438042711Z I0308 10:41:06.437692       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:56 +0000 UTC but not ready: unknown reason\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:06Z","message":"NodeInstallerProgressing: 3 nodes are at revision 2\nEtcdMembersProgressing: No unstarted etcd members found","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 3 nodes are active; 3 nodes are at revision 2\nEtcdMembersAvailable: 4 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:06.453634061Z I0308 10:41:06.453510       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Progressing changed from True to False ("NodeInstallerProgressing: 3 nodes are at revision 2\nEtcdMembersProgressing: No unstarted etcd members found"),Available message changed from "StaticPodsAvailable: 2 nodes are active; 1 nodes are at revision 0; 2 nodes are at revision 2\nEtcdMembersAvailable: 4 members are available" to "StaticPodsAvailable: 3 nodes are active; 3 nodes are at revision 2\nEtcdMembersAvailable: 4 members are available"
2022-03-08T10:41:06.460534026Z I0308 10:41:06.460364       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:06.848270125Z I0308 10:41:06.848200       1 status_controller.go:172] clusteroperator/etcd diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"NodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:06Z","message":"NodeInstallerProgressing: 3 nodes are at revision 2\nEtcdMembersProgressing: No unstarted etcd members found","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:40:07Z","message":"StaticPodsAvailable: 3 nodes are active; 3 nodes are at revision 2\nEtcdMembersAvailable: 4 members are available","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:47Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:06.860441791Z I0308 10:41:06.860359       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/etcd changed: Degraded message changed from "StaticPodsDegraded: pod/etcd-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"etcd\" is running since 2022-03-08 10:40:56 +0000 UTC but not ready: unknown reason\nNodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found" to "NodeControllerDegraded: All master nodes are ready\nEtcdMembersDegraded: No unhealthy members found"
2022-03-08T10:41:06.873868120Z I0308 10:41:06.873797       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:07.428266060Z I0308 10:41:07.428212       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapUpdated' Updated ConfigMap/revision-status-2 -n openshift-etcd:
2022-03-08T10:41:07.428266060Z cause by changes in data.status
2022-03-08T10:41:07.620024668Z I0308 10:41:07.619895       1 request.go:645] Throttling request took 1.183917008s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/serviceaccounts/installer-sa
2022-03-08T10:41:08.408530685Z I0308 10:41:08.408442       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:08.619995524Z I0308 10:41:08.619918       1 request.go:645] Throttling request took 1.396121188s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/revision-pruner-2-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:09.620025336Z I0308 10:41:09.619925       1 request.go:645] Throttling request took 1.396425742s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-etcd/pods/etcd-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:11.431647025Z WARNING: 2022/03/08 10:41:11 grpc: addrConn.createTransport failed to connect to {https://10.0.94.119:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: context canceled". Reconnecting...
2022-03-08T10:41:11.439742760Z I0308 10:41:11.439653       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:11.830708084Z I0308 10:41:11.830636       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/revision-pruner-2-master-1.case03116926.lab.upshift.rdu2.redhat.com -n openshift-etcd because it was missing
2022-03-08T10:41:14.459296254Z I0308 10:41:14.459204       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:17.533009328Z I0308 10:41:17.532884       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:20.603603472Z I0308 10:41:20.603553       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:23.528113792Z I0308 10:41:23.527975       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:26.499253593Z I0308 10:41:26.499160       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:29.504263968Z I0308 10:41:29.504136       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:32.541256230Z I0308 10:41:32.541108       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:35.593662266Z I0308 10:41:35.593229       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:38.586733187Z I0308 10:41:38.586396       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:41.591584016Z I0308 10:41:41.591401       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:43.595204407Z E0308 10:41:43.595041       1 base_controller.go:250] "FSyncController" controller failed to sync "key", err: unable to retrieve prometheus-k8 bearer token
2022-03-08T10:41:44.634677257Z I0308 10:41:44.634547       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:47.615449133Z I0308 10:41:47.615327       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:50.616674309Z I0308 10:41:50.616609       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:52.640237386Z I0308 10:41:52.640152       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:54.506120860Z I0308 10:41:54.505364       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:57.559339477Z I0308 10:41:57.559271       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DelayingBootstrapTeardown' cluster-bootstrap is not yet finished
2022-03-08T10:41:58.294804265Z I0308 10:41:58.294499       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RemoveBootstrapEtcd' removing etcd-bootstrap member
2022-03-08T10:41:58.294804265Z I0308 10:41:58.294683       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'MemberRemove' removing member "etcd-bootstrap"
2022-03-08T10:41:58.347425035Z WARNING: 2022/03/08 10:41:58 grpc: addrConn.createTransport failed to connect to {https://10.0.89.186:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: read tcp 10.130.0.4:47828->10.0.89.186:2379: read: connection reset by peer". Reconnecting...
2022-03-08T10:41:58.371681449Z E0308 10:41:58.371416       1 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"etcd-operator.16da61c659b9e090", GenerateName:"", Namespace:"openshift-etcd-operator", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-etcd-operator", Name:"etcd-operator", UID:"8520abe5-2170-45b9-b29e-f3df07b4ad2b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}, Reason:"RemoveBootstrapEtcd", Message:"removing etcd-bootstrap member", Source:v1.EventSource{Component:"openshift-cluster-etcd-operator-bootstrap-teardown-controller-bootstrapteardowncontroller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xc081ea9d9187c490, ext:116903848696, loc:(*time.Location)(0x37dd3e0)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc081ea9d9187c490, ext:116903848696, loc:(*time.Location)(0x37dd3e0)}}, Count:1, Type:"Normal", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'rpc error: code = Unavailable desc = transport is closing' (will not retry!)
2022-03-08T10:41:59.493521798Z WARNING: 2022/03/08 10:41:59 grpc: addrConn.createTransport failed to connect to {https://10.0.89.186:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: authentication handshake failed: read tcp 10.130.0.4:47842->10.0.89.186:2379: read: connection reset by peer". Reconnecting...
2022-03-08T10:42:01.280968644Z WARNING: 2022/03/08 10:42:01 grpc: addrConn.createTransport failed to connect to {https://10.0.89.186:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.89.186:2379: connect: connection refused". Reconnecting...
2022-03-08T10:42:03.705934070Z WARNING: 2022/03/08 10:42:03 grpc: addrConn.createTransport failed to connect to {https://10.0.89.186:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.89.186:2379: connect: connection refused". Reconnecting...
2022-03-08T10:42:08.047743558Z WARNING: 2022/03/08 10:42:08 grpc: addrConn.createTransport failed to connect to {https://10.0.89.186:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.89.186:2379: connect: connection refused". Reconnecting...
2022-03-08T10:42:14.268100020Z WARNING: 2022/03/08 10:42:14 grpc: addrConn.createTransport failed to connect to {https://10.0.89.186:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.89.186:2379: connect: connection refused". Reconnecting...
2022-03-08T10:42:25.796608166Z WARNING: 2022/03/08 10:42:25 grpc: addrConn.createTransport failed to connect to {https://10.0.89.186:2379  <nil> 0 <nil>}. Err :connection error: desc = "transport: Error while dialing dial tcp 10.0.89.186:2379: connect: connection refused". Reconnecting...
2022-03-08T10:42:37.164420983Z E0308 10:42:37.164304       1 leaderelection.go:321] error retrieving resource lock openshift-etcd-operator/openshift-cluster-etcd-operator-lock: Get "https://172.30.0.1:443/api/v1/namespaces/openshift-etcd-operator/configmaps/openshift-cluster-etcd-operator-lock?timeout=35s": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:42:37.164420983Z I0308 10:42:37.164401       1 leaderelection.go:278] failed to renew lease openshift-etcd-operator/openshift-cluster-etcd-operator-lock: timed out waiting for the condition
2022-03-08T10:42:37.164608121Z I0308 10:42:37.164544       1 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"", Name:"", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' etcd-operator-859798c7f4-kb5lq_02b0f5d9-9c2c-41fc-ad32-5e15a246e928 stopped leading
2022-03-08T10:42:37.164897022Z E0308 10:42:37.164864       1 leaderelection.go:297] Failed to release lock: resource name may not be empty
2022-03-08T10:42:37.164997457Z I0308 10:42:37.164980       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165024104Z I0308 10:42:37.165009       1 base_controller.go:166] Shutting down ClusterMemberController ...
2022-03-08T10:42:37.165091428Z I0308 10:42:37.165051       1 base_controller.go:166] Shutting down ConfigObserver ...
2022-03-08T10:42:37.165091428Z I0308 10:42:37.165078       1 base_controller.go:166] Shutting down EtcdEndpointsController ...
2022-03-08T10:42:37.165091428Z I0308 10:42:37.165087       1 base_controller.go:166] Shutting down ScriptController ...
2022-03-08T10:42:37.165109673Z I0308 10:42:37.165089       1 base_controller.go:166] Shutting down StaticResourceController ...
2022-03-08T10:42:37.165109673Z I0308 10:42:37.165099       1 base_controller.go:166] Shutting down StaticResourceController ...
2022-03-08T10:42:37.165121956Z I0308 10:42:37.165107       1 envvarcontroller.go:177] Shutting down EnvVarController
2022-03-08T10:42:37.165121956Z I0308 10:42:37.165110       1 base_controller.go:166] Shutting down StatusSyncer_etcd ...
2022-03-08T10:42:37.165121956Z I0308 10:42:37.165118       1 base_controller.go:166] Shutting down HostEndpoints2Controller ...
2022-03-08T10:42:37.165131214Z I0308 10:42:37.165114       1 reflector.go:213] Stopping reflector *v1.Node (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165131214Z I0308 10:42:37.165118       1 base_controller.go:166] Shutting down PruneController ...
2022-03-08T10:42:37.165131214Z I0308 10:42:37.165126       1 base_controller.go:166] Shutting down UnsupportedConfigOverridesController ...
2022-03-08T10:42:37.165140591Z I0308 10:42:37.165128       1 base_controller.go:144] All StatusSyncer_etcd post start hooks have been terminated
2022-03-08T10:42:37.165140591Z I0308 10:42:37.165132       1 base_controller.go:166] Shutting down BootstrapTeardownController ...
2022-03-08T10:42:37.165140591Z I0308 10:42:37.165136       1 base_controller.go:166] Shutting down ResourceSyncController ...
2022-03-08T10:42:37.165150307Z I0308 10:42:37.165136       1 base_controller.go:166] Shutting down TargetConfigController ...
2022-03-08T10:42:37.165150307Z I0308 10:42:37.165141       1 base_controller.go:166] Shutting down NodeController ...
2022-03-08T10:42:37.165150307Z I0308 10:42:37.165147       1 base_controller.go:166] Shutting down RevisionController ...
2022-03-08T10:42:37.165172062Z I0308 10:42:37.165148       1 base_controller.go:166] Shutting down UnsupportedConfigOverridesController ...
2022-03-08T10:42:37.165172062Z I0308 10:42:37.165154       1 base_controller.go:166] Shutting down RemoveStaleConditionsController ...
2022-03-08T10:42:37.165172062Z I0308 10:42:37.165156       1 base_controller.go:166] Shutting down InstallerController ...
2022-03-08T10:42:37.165172062Z I0308 10:42:37.165155       1 base_controller.go:166] Shutting down EtcdCertSignerController ...
2022-03-08T10:42:37.165172062Z I0308 10:42:37.165168       1 base_controller.go:166] Shutting down InstallerStateController ...
2022-03-08T10:42:37.165183204Z I0308 10:42:37.165168       1 base_controller.go:166] Shutting down LoggingSyncer ...
2022-03-08T10:42:37.165183204Z I0308 10:42:37.165180       1 base_controller.go:166] Shutting down StaticPodStateController ...
2022-03-08T10:42:37.165215560Z I0308 10:42:37.165195       1 reflector.go:213] Stopping reflector *v1.Endpoints (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165228872Z I0308 10:42:37.165212       1 base_controller.go:113] Shutting down worker of StatusSyncer_etcd controller ...
2022-03-08T10:42:37.165228872Z I0308 10:42:37.165218       1 base_controller.go:103] All StatusSyncer_etcd workers have been terminated
2022-03-08T10:42:37.165242369Z I0308 10:42:37.165228       1 base_controller.go:113] Shutting down worker of PruneController controller ...
2022-03-08T10:42:37.165242369Z I0308 10:42:37.165228       1 base_controller.go:113] Shutting down worker of ConfigObserver controller ...
2022-03-08T10:42:37.165242369Z I0308 10:42:37.165236       1 base_controller.go:103] All ConfigObserver workers have been terminated
2022-03-08T10:42:37.165242369Z I0308 10:42:37.165239       1 base_controller.go:113] Shutting down worker of UnsupportedConfigOverridesController controller ...
2022-03-08T10:42:37.165252587Z I0308 10:42:37.165243       1 base_controller.go:103] All UnsupportedConfigOverridesController workers have been terminated
2022-03-08T10:42:37.165252587Z I0308 10:42:37.165250       1 base_controller.go:113] Shutting down worker of NodeController controller ...
2022-03-08T10:42:37.165262494Z I0308 10:42:37.165254       1 base_controller.go:103] All NodeController workers have been terminated
2022-03-08T10:42:37.165262494Z I0308 10:42:37.165257       1 base_controller.go:113] Shutting down worker of HostEndpoints2Controller controller ...
2022-03-08T10:42:37.165277899Z I0308 10:42:37.165261       1 base_controller.go:113] Shutting down worker of UnsupportedConfigOverridesController controller ...
2022-03-08T10:42:37.165277899Z I0308 10:42:37.165265       1 base_controller.go:103] All UnsupportedConfigOverridesController workers have been terminated
2022-03-08T10:42:37.165277899Z I0308 10:42:37.165271       1 base_controller.go:113] Shutting down worker of RevisionController controller ...
2022-03-08T10:42:37.165277899Z I0308 10:42:37.165275       1 base_controller.go:103] All RevisionController workers have been terminated
2022-03-08T10:42:37.165285315Z I0308 10:42:37.165278       1 base_controller.go:113] Shutting down worker of RemoveStaleConditionsController controller ...
2022-03-08T10:42:37.165291573Z I0308 10:42:37.165286       1 base_controller.go:103] All RemoveStaleConditionsController workers have been terminated
2022-03-08T10:42:37.165297935Z I0308 10:42:37.165291       1 base_controller.go:113] Shutting down worker of EtcdCertSignerController controller ...
2022-03-08T10:42:37.165297935Z I0308 10:42:37.165296       1 base_controller.go:103] All EtcdCertSignerController workers have been terminated
2022-03-08T10:42:37.165307851Z I0308 10:42:37.165301       1 base_controller.go:113] Shutting down worker of LoggingSyncer controller ...
2022-03-08T10:42:37.165314152Z I0308 10:42:37.165307       1 base_controller.go:103] All LoggingSyncer workers have been terminated
2022-03-08T10:42:37.165314152Z I0308 10:42:37.165262       1 base_controller.go:103] All HostEndpoints2Controller workers have been terminated
2022-03-08T10:42:37.165359288Z I0308 10:42:37.165323       1 reflector.go:213] Stopping reflector *v1.ServiceAccount (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165359288Z I0308 10:42:37.165232       1 base_controller.go:103] All PruneController workers have been terminated
2022-03-08T10:42:37.165367590Z I0308 10:42:37.165359       1 reflector.go:213] Stopping reflector *v1.Pod (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165405667Z I0308 10:42:37.165393       1 reflector.go:213] Stopping reflector *v1.ClusterOperator (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165447995Z I0308 10:42:37.165434       1 base_controller.go:166] Shutting down EtcdMembersController ...
2022-03-08T10:42:37.165447995Z I0308 10:42:37.165442       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165497703Z I0308 10:42:37.165475       1 reflector.go:213] Stopping reflector *unstructured.Unstructured (12h0m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165543238Z I0308 10:42:37.165528       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165567897Z I0308 10:42:37.165557       1 reflector.go:213] Stopping reflector *v1.Etcd (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165605361Z I0308 10:42:37.165576       1 base_controller.go:113] Shutting down worker of InstallerController controller ...
2022-03-08T10:42:37.165605361Z I0308 10:42:37.165580       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165605361Z I0308 10:42:37.165583       1 base_controller.go:103] All InstallerController workers have been terminated
2022-03-08T10:42:37.165605361Z I0308 10:42:37.165592       1 base_controller.go:113] Shutting down worker of ClusterMemberController controller ...
2022-03-08T10:42:37.165605361Z I0308 10:42:37.165596       1 base_controller.go:103] All ClusterMemberController workers have been terminated
2022-03-08T10:42:37.165605361Z I0308 10:42:37.165596       1 reflector.go:213] Stopping reflector *v1.ClusterRoleBinding (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165655824Z I0308 10:42:37.165643       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165663993Z I0308 10:42:37.165655       1 base_controller.go:113] Shutting down worker of ResourceSyncController controller ...
2022-03-08T10:42:37.165663993Z I0308 10:42:37.165659       1 base_controller.go:103] All ResourceSyncController workers have been terminated
2022-03-08T10:42:37.165679245Z I0308 10:42:37.165668       1 base_controller.go:166] Shutting down FSyncController ...
2022-03-08T10:42:37.165679245Z I0308 10:42:37.165674       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165728341Z I0308 10:42:37.165714       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165752312Z I0308 10:42:37.165742       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165808530Z I0308 10:42:37.165788       1 reflector.go:213] Stopping reflector *v1.Network (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165833834Z I0308 10:42:37.165823       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165897841Z I0308 10:42:37.165883       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165906541Z I0308 10:42:37.165897       1 controller_manager.go:54] NodeController controller terminated
2022-03-08T10:42:37.165906541Z I0308 10:42:37.165902       1 controller_manager.go:54] UnsupportedConfigOverridesController controller terminated
2022-03-08T10:42:37.165912895Z I0308 10:42:37.165905       1 controller_manager.go:54] RevisionController controller terminated
2022-03-08T10:42:37.165912895Z I0308 10:42:37.165910       1 controller_manager.go:54] LoggingSyncer controller terminated
2022-03-08T10:42:37.165937777Z I0308 10:42:37.165916       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:37.165937777Z I0308 10:42:37.165926       1 controller_manager.go:54] InstallerController controller terminated
2022-03-08T10:42:37.165937777Z W0308 10:42:37.165885       1 leaderelection.go:75] leader election lost
