2022-03-08T10:40:06.293584324Z I0308 10:40:06.293445       1 cmd.go:200] Using service-serving-cert provided certificates
2022-03-08T10:40:06.294042427Z I0308 10:40:06.294005       1 observer_polling.go:159] Starting file observer
2022-03-08T10:40:06.318173795Z I0308 10:40:06.318088       1 builder.go:238] kube-controller-manager-operator version v0.0.0-alpha.0-858-gb05961c9-b05961c979fff825cc17fbd66660f7409cc82f28
2022-03-08T10:40:06.627157048Z W0308 10:40:06.627116       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.
2022-03-08T10:40:06.627217182Z W0308 10:40:06.627204       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.
2022-03-08T10:40:06.627642134Z I0308 10:40:06.627571       1 leaderelection.go:243] attempting to acquire leader lease  openshift-kube-controller-manager-operator/kube-controller-manager-operator-lock...
2022-03-08T10:40:06.633201187Z I0308 10:40:06.632240       1 secure_serving.go:197] Serving securely on [::]:8443
2022-03-08T10:40:06.633308221Z I0308 10:40:06.633289       1 tlsconfig.go:240] Starting DynamicServingCertificateController
2022-03-08T10:40:06.633378747Z I0308 10:40:06.633359       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
2022-03-08T10:40:06.633390954Z I0308 10:40:06.633379       1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
2022-03-08T10:40:06.633408800Z I0308 10:40:06.633387       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
2022-03-08T10:40:06.633417332Z I0308 10:40:06.633403       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
2022-03-08T10:40:06.633543148Z I0308 10:40:06.633523       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
2022-03-08T10:40:06.633543148Z I0308 10:40:06.633532       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
2022-03-08T10:40:06.633603974Z I0308 10:40:06.633586       1 dynamic_serving_content.go:130] Starting serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key
2022-03-08T10:40:06.641259016Z I0308 10:40:06.641210       1 leaderelection.go:253] successfully acquired lease openshift-kube-controller-manager-operator/kube-controller-manager-operator-lock
2022-03-08T10:40:06.642134952Z I0308 10:40:06.642091       1 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator-lock", UID:"144301bc-ecf9-42f5-ac6f-c69bde1213ff", APIVersion:"v1", ResourceVersion:"10103", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' kube-controller-manager-operator-7bb9b6d8b8-h8hjn_d927429a-bf3d-4a4f-a9de-203494db50c9 became leader
2022-03-08T10:40:06.675193501Z I0308 10:40:06.674958       1 base_controller.go:66] Waiting for caches to sync for RemoveStaleConditionsController
2022-03-08T10:40:06.675329134Z I0308 10:40:06.675305       1 targetconfigcontroller.go:724] Starting TargetConfigController
2022-03-08T10:40:06.675340639Z I0308 10:40:06.675323       1 satokensigner_controller.go:344] Starting SATokenSignerController
2022-03-08T10:40:06.675362858Z I0308 10:40:06.675351       1 base_controller.go:66] Waiting for caches to sync for ConfigObserver
2022-03-08T10:40:06.675597099Z I0308 10:40:06.675577       1 base_controller.go:66] Waiting for caches to sync for StatusSyncer_kube-controller-manager
2022-03-08T10:40:06.675609441Z I0308 10:40:06.675599       1 base_controller.go:66] Waiting for caches to sync for ResourceSyncController
2022-03-08T10:40:06.675632853Z I0308 10:40:06.675616       1 base_controller.go:66] Waiting for caches to sync for CertRotationController
2022-03-08T10:40:06.676229483Z I0308 10:40:06.676208       1 base_controller.go:66] Waiting for caches to sync for LoggingSyncer
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676375       1 base_controller.go:66] Waiting for caches to sync for RevisionController
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676532       1 base_controller.go:66] Waiting for caches to sync for PruneController
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676706       1 base_controller.go:66] Waiting for caches to sync for InstallerStateController
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676727       1 base_controller.go:66] Waiting for caches to sync for StaticPodStateController
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676759       1 base_controller.go:66] Waiting for caches to sync for InstallerController
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676802       1 base_controller.go:66] Waiting for caches to sync for UnsupportedConfigOverridesController
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676842       1 base_controller.go:66] Waiting for caches to sync for NodeController
2022-03-08T10:40:06.679099895Z I0308 10:40:06.676945       1 base_controller.go:66] Waiting for caches to sync for StaticResourceController
2022-03-08T10:40:06.683204315Z I0308 10:40:06.682246       1 base_controller.go:66] Waiting for caches to sync for StaticResourceController
2022-03-08T10:40:06.733628968Z I0308 10:40:06.733574       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file 
2022-03-08T10:40:06.733628968Z I0308 10:40:06.733574       1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController 
2022-03-08T10:40:06.733628968Z I0308 10:40:06.733610       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file 
2022-03-08T10:40:06.775392458Z I0308 10:40:06.775191       1 base_controller.go:72] Caches are synced for RemoveStaleConditionsController 
2022-03-08T10:40:06.775392458Z I0308 10:40:06.775248       1 base_controller.go:109] Starting #1 worker of RemoveStaleConditionsController controller ...
2022-03-08T10:40:06.775709530Z I0308 10:40:06.775672       1 base_controller.go:72] Caches are synced for StatusSyncer_kube-controller-manager 
2022-03-08T10:40:06.775709530Z I0308 10:40:06.775685       1 base_controller.go:109] Starting #1 worker of StatusSyncer_kube-controller-manager controller ...
2022-03-08T10:40:06.776347229Z I0308 10:40:06.776324       1 base_controller.go:72] Caches are synced for LoggingSyncer 
2022-03-08T10:40:06.776383854Z I0308 10:40:06.776375       1 base_controller.go:109] Starting #1 worker of LoggingSyncer controller ...
2022-03-08T10:40:06.776461989Z I0308 10:40:06.776425       1 base_controller.go:72] Caches are synced for RevisionController 
2022-03-08T10:40:06.776480121Z I0308 10:40:06.776462       1 base_controller.go:109] Starting #1 worker of RevisionController controller ...
2022-03-08T10:40:06.776658713Z I0308 10:40:06.776632       1 base_controller.go:72] Caches are synced for PruneController 
2022-03-08T10:40:06.776658713Z I0308 10:40:06.776646       1 base_controller.go:109] Starting #1 worker of PruneController controller ...
2022-03-08T10:40:06.776816894Z I0308 10:40:06.776799       1 base_controller.go:72] Caches are synced for StaticPodStateController 
2022-03-08T10:40:06.776856856Z I0308 10:40:06.776837       1 base_controller.go:109] Starting #1 worker of StaticPodStateController controller ...
2022-03-08T10:40:06.776913209Z I0308 10:40:06.776845       1 base_controller.go:72] Caches are synced for UnsupportedConfigOverridesController 
2022-03-08T10:40:06.776913209Z I0308 10:40:06.776908       1 base_controller.go:109] Starting #1 worker of UnsupportedConfigOverridesController controller ...
2022-03-08T10:40:06.776974674Z I0308 10:40:06.776854       1 base_controller.go:72] Caches are synced for InstallerController 
2022-03-08T10:40:06.776974674Z I0308 10:40:06.776970       1 base_controller.go:109] Starting #1 worker of InstallerController controller ...
2022-03-08T10:40:06.777199111Z I0308 10:40:06.776861       1 base_controller.go:72] Caches are synced for InstallerStateController 
2022-03-08T10:40:06.777199111Z I0308 10:40:06.777190       1 base_controller.go:109] Starting #1 worker of InstallerStateController controller ...
2022-03-08T10:40:06.777250111Z I0308 10:40:06.777234       1 base_controller.go:72] Caches are synced for StaticResourceController 
2022-03-08T10:40:06.777278087Z I0308 10:40:06.777268       1 base_controller.go:109] Starting #1 worker of StaticResourceController controller ...
2022-03-08T10:40:06.777372208Z I0308 10:40:06.777347       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:06.777406624Z E0308 10:40:06.777391       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:06.778935249Z I0308 10:40:06.778897       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RevisionTriggered' new revision 5 triggered by "configmap/serviceaccount-ca has changed"
2022-03-08T10:40:06.782618810Z I0308 10:40:06.782579       1 base_controller.go:72] Caches are synced for StaticResourceController 
2022-03-08T10:40:06.782695590Z I0308 10:40:06.782681       1 base_controller.go:109] Starting #1 worker of StaticResourceController controller ...
2022-03-08T10:40:06.783053130Z I0308 10:40:06.783012       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:06.783228634Z E0308 10:40:06.783191       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:06.791382196Z I0308 10:40:06.791095       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nInstallerControllerDegraded: missing required resources: configmaps: client-ca" to "StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nInstallerControllerDegraded: missing required resources: configmaps: client-ca\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled"
2022-03-08T10:40:06.794565011Z E0308 10:40:06.794178       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:06.794565011Z I0308 10:40:06.794256       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:06.814956837Z I0308 10:40:06.814887       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:06.815040474Z E0308 10:40:06.814932       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:06.855784826Z I0308 10:40:06.855713       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:06.855868330Z E0308 10:40:06.855836       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:06.877133834Z I0308 10:40:06.877021       1 base_controller.go:72] Caches are synced for NodeController 
2022-03-08T10:40:06.877133834Z I0308 10:40:06.877100       1 base_controller.go:109] Starting #1 worker of NodeController controller ...
2022-03-08T10:40:06.936990065Z I0308 10:40:06.936873       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:06.937047213Z E0308 10:40:06.937003       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:07.097790328Z E0308 10:40:07.097725       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:07.098236409Z I0308 10:40:07.098153       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:07.419756471Z E0308 10:40:07.419668       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:07.424042988Z I0308 10:40:07.420252       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:07.476117484Z I0308 10:40:07.475404       1 base_controller.go:72] Caches are synced for ConfigObserver 
2022-03-08T10:40:07.476117484Z I0308 10:40:07.475432       1 base_controller.go:109] Starting #1 worker of ConfigObserver controller ...
2022-03-08T10:40:07.675746001Z I0308 10:40:07.675694       1 base_controller.go:72] Caches are synced for ResourceSyncController 
2022-03-08T10:40:07.675746001Z I0308 10:40:07.675719       1 base_controller.go:109] Starting #1 worker of ResourceSyncController controller ...
2022-03-08T10:40:07.675746001Z I0308 10:40:07.675726       1 base_controller.go:72] Caches are synced for CertRotationController 
2022-03-08T10:40:07.675811256Z I0308 10:40:07.675744       1 base_controller.go:109] Starting #1 worker of CertRotationController controller ...
2022-03-08T10:40:07.869166154Z I0308 10:40:07.868717       1 request.go:645] Throttling request took 1.170650542s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config
2022-03-08T10:40:08.060735127Z I0308 10:40:08.060530       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RequiredInstallerResourcesMissing' configmaps: client-ca
2022-03-08T10:40:08.060735127Z E0308 10:40:08.060603       1 base_controller.go:250] "InstallerController" controller failed to sync "key", err: missing required resources: configmaps: client-ca
2022-03-08T10:40:09.068683703Z I0308 10:40:09.068632       1 request.go:645] Throttling request took 1.392963404s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-apiserver/pods?labelSelector=app%3Dopenshift-kube-apiserver
2022-03-08T10:40:09.282878708Z I0308 10:40:09.282686       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/client-ca -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:09.876680947Z I0308 10:40:09.876598       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/kube-controller-cert-syncer-kubeconfig-5 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:10.068762950Z I0308 10:40:10.068643       1 request.go:645] Throttling request took 1.162734291s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/services/kube-controller-manager
2022-03-08T10:40:10.273108843Z E0308 10:40:10.273037       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:10.273172716Z I0308 10:40:10.273113       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:10.470888632Z I0308 10:40:10.470775       1 installer_controller.go:350] node master-0.case03116926.lab.upshift.rdu2.redhat.com static pod not found and needs new revision 4
2022-03-08T10:40:10.470888632Z I0308 10:40:10.470862       1 installer_controller.go:359] "master-0.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:10.470888632Z  NodeName: (string) (len=49) "master-0.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:10.470888632Z  CurrentRevision: (int32) 0,
2022-03-08T10:40:10.470888632Z  TargetRevision: (int32) 4,
2022-03-08T10:40:10.470888632Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:10.470888632Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:10.470888632Z }
2022-03-08T10:40:10.490552538Z I0308 10:40:10.490380       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeTargetRevisionChanged' Updating node "master-0.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 4 because node master-0.case03116926.lab.upshift.rdu2.redhat.com static pod not found
2022-03-08T10:40:10.514329650Z I0308 10:40:10.514264       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 4","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 4","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:10.526421676Z I0308 10:40:10.526303       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nInstallerControllerDegraded: missing required resources: configmaps: client-ca\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled" to "StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled"
2022-03-08T10:40:10.872115245Z I0308 10:40:10.871936       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'MissingVersion' no image found for operand pod
2022-03-08T10:40:11.080212173Z I0308 10:40:11.080114       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/serviceaccount-ca-5 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:11.268733109Z I0308 10:40:11.268622       1 request.go:645] Throttling request took 1.195957307s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/kube-controller-manager-sa
2022-03-08T10:40:12.468660087Z I0308 10:40:12.468559       1 request.go:645] Throttling request took 1.388742423s, request: POST:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps
2022-03-08T10:40:12.488269932Z I0308 10:40:12.488093       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/service-ca-5 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:12.875813782Z E0308 10:40:12.875760       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:12.875899957Z I0308 10:40:12.875860       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:13.286138842Z I0308 10:40:13.286049       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/installer-4-master-0.case03116926.lab.upshift.rdu2.redhat.com -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:13.468713454Z I0308 10:40:13.468625       1 request.go:645] Throttling request took 1.392482408s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager-operator/configmaps/csr-signer-ca
2022-03-08T10:40:13.878619761Z I0308 10:40:13.878559       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/recycler-config-5 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:14.468716090Z I0308 10:40:14.468657       1 request.go:645] Throttling request took 1.182733563s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:14.672591867Z I0308 10:40:14.672512       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 4, but has not made progress because 
2022-03-08T10:40:15.074538996Z I0308 10:40:15.074414       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'MissingVersion' no image found for operand pod
2022-03-08T10:40:15.285387821Z I0308 10:40:15.285326       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/service-account-private-key-5 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:15.668693218Z I0308 10:40:15.668642       1 request.go:645] Throttling request took 1.387828343s, request: GET:https://172.30.0.1:443/api/v1/namespaces/default/endpoints/kubernetes
2022-03-08T10:40:15.675815276Z E0308 10:40:15.675731       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:15.675873371Z I0308 10:40:15.675811       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:16.792395620Z I0308 10:40:16.792324       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/serving-cert-5 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:16.868700144Z I0308 10:40:16.868641       1 request.go:645] Throttling request took 1.397455476s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/kube-controller-manager-sa
2022-03-08T10:40:17.483876910Z I0308 10:40:17.483817       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 4, but has not made progress because 
2022-03-08T10:40:17.868696397Z I0308 10:40:17.868652       1 request.go:645] Throttling request took 1.395282392s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:18.080314244Z I0308 10:40:18.080262       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/localhost-recovery-client-token-5 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:18.101371159Z I0308 10:40:18.101318       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RevisionCreate' Revision 4 created because configmap/serviceaccount-ca has changed
2022-03-08T10:40:18.473776593Z E0308 10:40:18.473726       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:18.474269677Z I0308 10:40:18.473921       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:18.868764053Z I0308 10:40:18.868680       1 request.go:645] Throttling request took 1.190172836s, request: PUT:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/serviceaccount-ca
2022-03-08T10:40:18.882377001Z I0308 10:40:18.882308       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapUpdated' Updated ConfigMap/serviceaccount-ca -n openshift-kube-controller-manager:
2022-03-08T10:40:18.882377001Z cause by changes in data.ca-bundle.crt
2022-03-08T10:40:18.887881727Z I0308 10:40:18.887825       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RevisionTriggered' new revision 6 triggered by "configmap/serviceaccount-ca has changed"
2022-03-08T10:40:19.074116729Z I0308 10:40:19.074045       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'MissingVersion' no image found for operand pod
2022-03-08T10:40:20.068707006Z I0308 10:40:20.068637       1 request.go:645] Throttling request took 1.389859616s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/installer-4-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:20.516848145Z I0308 10:40:20.516746       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/revision-status-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:21.109113950Z E0308 10:40:21.108620       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:21.109172452Z I0308 10:40:21.109117       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:21.269326622Z I0308 10:40:21.269222       1 request.go:645] Throttling request took 1.321592082s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:21.479104283Z I0308 10:40:21.478188       1 installer_controller.go:331] "master-0.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 4, but has not made progress because static pod is pending
2022-03-08T10:40:21.503210835Z I0308 10:40:21.502346       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:21.530163012Z I0308 10:40:21.530025       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Progressing message changed from "NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 4" to "NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5",Available message changed from "StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 4" to "StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5"
2022-03-08T10:40:21.877373998Z I0308 10:40:21.877314       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapUpdated' Updated ConfigMap/revision-status-5 -n openshift-kube-controller-manager:
2022-03-08T10:40:21.877373998Z cause by changes in data.status
2022-03-08T10:40:22.468737340Z I0308 10:40:22.468624       1 request.go:645] Throttling request took 1.359897632s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-apiserver/pods?labelSelector=app%3Dopenshift-kube-apiserver
2022-03-08T10:40:23.485158357Z I0308 10:40:23.485099       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/kube-controller-manager-pod-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:23.668714325Z I0308 10:40:23.668652       1 request.go:645] Throttling request took 1.595881336s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:23.672807000Z I0308 10:40:23.672747       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"versions":[{"name":"raw-internal","version":"4.6.52"},{"name":"kube-controller-manager","version":"1.19.16"},{"name":"operator","version":"4.6.52"}]}}
2022-03-08T10:40:23.673031616Z I0308 10:40:23.672799       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorVersionChanged' clusteroperator/kube-controller-manager version "kube-controller-manager" changed from "" to "1.19.16"
2022-03-08T10:40:23.673049182Z I0308 10:40:23.673018       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorVersionChanged' clusteroperator/kube-controller-manager version "operator" changed from "" to "4.6.52"
2022-03-08T10:40:23.689103800Z I0308 10:40:23.689034       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}],"versions":[{"name":"raw-internal","version":"4.6.52"},{"name":"operator","version":"4.6.52"},{"name":"kube-controller-manager","version":"1.19.16"}]}}
2022-03-08T10:40:23.689456995Z I0308 10:40:23.689427       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: status.versions changed from [{"raw-internal" "4.6.52"}] to [{"raw-internal" "4.6.52"} {"kube-controller-manager" "1.19.16"} {"operator" "4.6.52"}]
2022-03-08T10:40:23.689559967Z I0308 10:40:23.689532       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorVersionChanged' clusteroperator/kube-controller-manager version "operator" changed from "" to "4.6.52"
2022-03-08T10:40:23.689640272Z I0308 10:40:23.689619       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorVersionChanged' clusteroperator/kube-controller-manager version "kube-controller-manager" changed from "" to "1.19.16"
2022-03-08T10:40:23.694223495Z I0308 10:40:23.694177       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:23.694444826Z I0308 10:40:23.694417       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled" to "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled",status.versions changed from [{"raw-internal" "4.6.52"}] to [{"raw-internal" "4.6.52"} {"operator" "4.6.52"} {"kube-controller-manager" "1.19.16"}]
2022-03-08T10:40:23.704898722Z I0308 10:40:23.704833       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled" to "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled"
2022-03-08T10:40:24.072255818Z E0308 10:40:24.072202       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:24.072316146Z I0308 10:40:24.072287       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:24.273263692Z I0308 10:40:24.273206       1 installer_controller.go:318] "master-0.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:24.273263692Z  NodeName: (string) (len=49) "master-0.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:24.273263692Z  CurrentRevision: (int32) 0,
2022-03-08T10:40:24.273263692Z  TargetRevision: (int32) 0,
2022-03-08T10:40:24.273263692Z  LastFailedRevision: (int32) 4,
2022-03-08T10:40:24.273263692Z  LastFailedRevisionErrors: ([]string) (len=1 cap=1) {
2022-03-08T10:40:24.273263692Z   (string) (len=93) "static pod of revision 4 has been installed, but is not ready while new revision 5 is pending"
2022-03-08T10:40:24.273263692Z  }
2022-03-08T10:40:24.273263692Z }
2022-03-08T10:40:24.273263692Z  because new revision pending
2022-03-08T10:40:24.292008097Z I0308 10:40:24.290430       1 installer_controller.go:398] revision-status-0 configmap not found, skipping update revision status
2022-03-08T10:40:24.295173375Z I0308 10:40:24.292826       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:24.329106778Z I0308 10:40:24.329031       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled" to "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled"
2022-03-08T10:40:24.670108710Z I0308 10:40:24.670016       1 request.go:645] Throttling request took 1.184951039s, request: POST:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps
2022-03-08T10:40:24.681126441Z I0308 10:40:24.680259       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/config-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:25.684211932Z I0308 10:40:25.684141       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapUpdated' Updated ConfigMap/revision-status-4 -n openshift-kube-controller-manager:
2022-03-08T10:40:25.684211932Z cause by changes in data.status
2022-03-08T10:40:25.868748568Z I0308 10:40:25.868641       1 request.go:645] Throttling request took 1.577129449s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/revision-pruner-4-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:26.478631975Z I0308 10:40:26.478577       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/cluster-policy-controller-config-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:27.068653165Z I0308 10:40:27.068580       1 request.go:645] Throttling request took 1.757612608s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/services/kube-controller-manager
2022-03-08T10:40:27.274169167Z E0308 10:40:27.274113       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:27.274636565Z I0308 10:40:27.274585       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:27.686911627Z I0308 10:40:27.686776       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/revision-pruner-4-master-0.case03116926.lab.upshift.rdu2.redhat.com -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:28.068771714Z I0308 10:40:28.068688       1 request.go:645] Throttling request took 1.795005791s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle
2022-03-08T10:40:28.217358742Z I0308 10:40:28.217300       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:28.261192294Z I0308 10:40:28.259978       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending\nTargetConfigControllerDegraded: \"configmap/trusted-ca-bundle\": Get \"https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/trusted-ca-bundle\": context canceled" to "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending"
2022-03-08T10:40:28.284053099Z I0308 10:40:28.283985       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/controller-manager-kubeconfig-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:28.550517329Z I0308 10:40:28.550417       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:28.562866713Z I0308 10:40:28.562818       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending" to "StaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending"
2022-03-08T10:40:29.069330391Z I0308 10:40:29.069213       1 request.go:645] Throttling request took 1.590795493s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:29.082938519Z I0308 10:40:29.081548       1 installer_controller.go:350] node master-1.case03116926.lab.upshift.rdu2.redhat.com static pod not found and needs new revision 5
2022-03-08T10:40:29.082938519Z I0308 10:40:29.081599       1 installer_controller.go:359] "master-1.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:29.082938519Z  NodeName: (string) (len=49) "master-1.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:29.082938519Z  CurrentRevision: (int32) 0,
2022-03-08T10:40:29.082938519Z  TargetRevision: (int32) 5,
2022-03-08T10:40:29.082938519Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:29.082938519Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:29.082938519Z }
2022-03-08T10:40:29.125184797Z I0308 10:40:29.125115       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeTargetRevisionChanged' Updating node "master-1.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 5 because node master-1.case03116926.lab.upshift.rdu2.redhat.com static pod not found
2022-03-08T10:40:30.271076990Z I0308 10:40:30.268916       1 request.go:645] Throttling request took 1.720254897s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:30.678113461Z E0308 10:40:30.678024       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:30.678390542Z I0308 10:40:30.678153       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:31.468845286Z I0308 10:40:31.468652       1 request.go:645] Throttling request took 1.792647505s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/installer-sa
2022-03-08T10:40:31.876525300Z I0308 10:40:31.876436       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/kube-controller-cert-syncer-kubeconfig-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:32.668681761Z I0308 10:40:32.668614       1 request.go:645] Throttling request took 1.797304523s, request: POST:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods
2022-03-08T10:40:32.679815604Z I0308 10:40:32.679760       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/installer-5-master-1.case03116926.lab.upshift.rdu2.redhat.com -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:33.278929347Z I0308 10:40:33.278871       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/serviceaccount-ca-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:33.497745015Z I0308 10:40:33.497694       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:33.506992006Z I0308 10:40:33.506830       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending" to "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending"
2022-03-08T10:40:33.668726680Z I0308 10:40:33.668621       1 request.go:645] Throttling request took 1.376864028s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/services/kube-controller-manager
2022-03-08T10:40:33.871705538Z I0308 10:40:33.871612       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:33.872000228Z E0308 10:40:33.871970       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:34.073423979Z I0308 10:40:34.073379       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 5, but has not made progress because 
2022-03-08T10:40:34.668750622Z I0308 10:40:34.668654       1 request.go:645] Throttling request took 1.390034941s, request: POST:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps
2022-03-08T10:40:34.678845927Z I0308 10:40:34.678770       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/service-ca-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:35.868879094Z I0308 10:40:35.868759       1 request.go:645] Throttling request took 1.792561662s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/installer-5-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:36.476532976Z I0308 10:40:36.476450       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapCreated' Created ConfigMap/recycler-config-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:37.068858233Z I0308 10:40:37.068734       1 request.go:645] Throttling request took 1.392824536s, request: GET:https://172.30.0.1:443/api/v1/namespaces/default/endpoints/kubernetes
2022-03-08T10:40:37.072281299Z I0308 10:40:37.072194       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:37.072370106Z E0308 10:40:37.072219       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:37.272345662Z I0308 10:40:37.272225       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 5, but has not made progress because 
2022-03-08T10:40:37.878041433Z I0308 10:40:37.877966       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/service-account-private-key-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:38.068833551Z I0308 10:40:38.068741       1 request.go:645] Throttling request took 1.393048623s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:39.069016614Z I0308 10:40:39.068911       1 request.go:645] Throttling request took 1.191211316s, request: POST:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/secrets
2022-03-08T10:40:39.077719513Z I0308 10:40:39.077642       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/serving-cert-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:39.671690585Z E0308 10:40:39.671625       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:39.671690585Z I0308 10:40:39.671660       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:39.872252584Z I0308 10:40:39.872163       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 5, but has not made progress because 
2022-03-08T10:40:40.268667993Z I0308 10:40:40.268617       1 request.go:645] Throttling request took 1.191276139s, request: POST:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/secrets
2022-03-08T10:40:40.276742777Z I0308 10:40:40.276682       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SecretCreated' Created Secret/localhost-recovery-client-token-6 -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:40.288803501Z I0308 10:40:40.288742       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RevisionCreate' Revision 5 created because configmap/serviceaccount-ca has changed
2022-03-08T10:40:41.268867806Z I0308 10:40:41.268817       1 request.go:645] Throttling request took 1.395565529s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/installer-5-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:42.468734596Z I0308 10:40:42.468634       1 request.go:645] Throttling request took 1.396790495s, request: GET:https://172.30.0.1:443/api/v1/namespaces/default/endpoints/kubernetes
2022-03-08T10:40:42.472192332Z I0308 10:40:42.472133       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:42.472401316Z E0308 10:40:42.472375       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:43.668713707Z I0308 10:40:43.668618       1 request.go:645] Throttling request took 1.196085346s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-apiserver/pods?labelSelector=app%3Dopenshift-kube-apiserver
2022-03-08T10:40:44.072465310Z I0308 10:40:44.072386       1 installer_controller.go:331] "master-1.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 5, but has not made progress because static pod is pending
2022-03-08T10:40:44.112018655Z I0308 10:40:44.111910       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:44.127512870Z I0308 10:40:44.127414       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Progressing message changed from "NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 5" to "NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6",Available message changed from "StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 5" to "StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6"
2022-03-08T10:40:44.668784575Z I0308 10:40:44.668667       1 request.go:645] Throttling request took 1.176520705s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/services/kube-controller-manager
2022-03-08T10:40:44.872141327Z E0308 10:40:44.872015       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:44.872598935Z I0308 10:40:44.872540       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:45.869043613Z I0308 10:40:45.868952       1 request.go:645] Throttling request took 1.499305261s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:46.094699470Z I0308 10:40:46.094632       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:46.107798184Z I0308 10:40:46.107505       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com\" not found\nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending" to "StaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending"
2022-03-08T10:40:46.673753107Z I0308 10:40:46.673607       1 installer_controller.go:318] "master-1.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:46.673753107Z  NodeName: (string) (len=49) "master-1.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:46.673753107Z  CurrentRevision: (int32) 0,
2022-03-08T10:40:46.673753107Z  TargetRevision: (int32) 0,
2022-03-08T10:40:46.673753107Z  LastFailedRevision: (int32) 5,
2022-03-08T10:40:46.673753107Z  LastFailedRevisionErrors: ([]string) (len=1 cap=1) {
2022-03-08T10:40:46.673753107Z   (string) (len=93) "static pod of revision 5 has been installed, but is not ready while new revision 6 is pending"
2022-03-08T10:40:46.673753107Z  }
2022-03-08T10:40:46.673753107Z }
2022-03-08T10:40:46.673753107Z  because new revision pending
2022-03-08T10:40:46.687827718Z I0308 10:40:46.687755       1 installer_controller.go:398] revision-status-0 configmap not found, skipping update revision status
2022-03-08T10:40:46.695353697Z I0308 10:40:46.695257       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:46.721799017Z I0308 10:40:46.719975       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending" to "StaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending"
2022-03-08T10:40:47.068784848Z I0308 10:40:47.068668       1 request.go:645] Throttling request took 1.190643583s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:48.068783080Z I0308 10:40:48.068666       1 request.go:645] Throttling request took 1.594074251s, request: GET:https://172.30.0.1:443/api/v1/namespaces/default/endpoints/kubernetes
2022-03-08T10:40:48.072866604Z E0308 10:40:48.072773       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:48.073301605Z I0308 10:40:48.073259       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:48.279224973Z I0308 10:40:48.279107       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ConfigMapUpdated' Updated ConfigMap/revision-status-5 -n openshift-kube-controller-manager:
2022-03-08T10:40:48.279224973Z cause by changes in data.status
2022-03-08T10:40:49.068855530Z I0308 10:40:49.068745       1 request.go:645] Throttling request took 1.3947468s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/installer-sa
2022-03-08T10:40:50.093900263Z I0308 10:40:50.092769       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:45Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:50.104312513Z I0308 10:40:50.103663       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending" to "StaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending"
2022-03-08T10:40:50.268752164Z I0308 10:40:50.268651       1 request.go:645] Throttling request took 1.395280153s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/revision-pruner-5-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:50.675358409Z E0308 10:40:50.675287       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:50.675420863Z I0308 10:40:50.675374       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:51.468802249Z I0308 10:40:51.468686       1 request.go:645] Throttling request took 1.373852413s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/installer-sa
2022-03-08T10:40:51.684821584Z I0308 10:40:51.684738       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/revision-pruner-5-master-1.case03116926.lab.upshift.rdu2.redhat.com -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:52.271195548Z I0308 10:40:52.271132       1 installer_controller.go:350] node master-2.case03116926.lab.upshift.rdu2.redhat.com static pod not found and needs new revision 6
2022-03-08T10:40:52.271195548Z I0308 10:40:52.271185       1 installer_controller.go:359] "master-2.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:40:52.271195548Z  NodeName: (string) (len=49) "master-2.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:40:52.271195548Z  CurrentRevision: (int32) 0,
2022-03-08T10:40:52.271195548Z  TargetRevision: (int32) 6,
2022-03-08T10:40:52.271195548Z  LastFailedRevision: (int32) 0,
2022-03-08T10:40:52.271195548Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:40:52.271195548Z }
2022-03-08T10:40:52.288100538Z I0308 10:40:52.287967       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:52Z","message":"StaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"NodeInstaller_InstallerPodFailed::StaticPods_Error","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:52.289055495Z I0308 10:40:52.288936       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeTargetRevisionChanged' Updating node "master-2.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 6 because node master-2.case03116926.lab.upshift.rdu2.redhat.com static pod not found
2022-03-08T10:40:52.308118820Z I0308 10:40:52.307343       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded changed from False to True ("StaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending")
2022-03-08T10:40:52.469007489Z I0308 10:40:52.468729       1 request.go:645] Throttling request took 1.396421012s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/configmaps/recycler-config
2022-03-08T10:40:53.471885270Z I0308 10:40:53.471790       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:53.471995966Z E0308 10:40:53.471966       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:53.668854906Z I0308 10:40:53.668675       1 request.go:645] Throttling request took 1.379660801s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/installer-6-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:54.868860096Z I0308 10:40:54.868718       1 request.go:645] Throttling request took 1.588454052s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager
2022-03-08T10:40:55.280964407Z I0308 10:40:55.280888       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/installer-6-master-2.case03116926.lab.upshift.rdu2.redhat.com -n openshift-kube-controller-manager because it was missing
2022-03-08T10:40:56.069737672Z I0308 10:40:56.069494       1 request.go:645] Throttling request took 1.395014383s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/revision-pruner-4-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:40:56.472249928Z E0308 10:40:56.472178       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:40:56.472300804Z I0308 10:40:56.472235       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:40:56.671107102Z I0308 10:40:56.671016       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because 
2022-03-08T10:40:57.268759089Z I0308 10:40:57.268678       1 request.go:645] Throttling request took 1.396288863s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593631       1 tlsconfig.go:178] loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "admin-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:26 +0000 UTC to 2032-03-05 10:10:26 +0000 UTC (now=2022-03-08 10:40:57.59360929 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593661       1 tlsconfig.go:178] loaded client CA [1/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.593655389 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593673       1 tlsconfig.go:178] loaded client CA [2/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-control-plane-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2023-03-08 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.593668296 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593694       1 tlsconfig.go:178] loaded client CA [3/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-apiserver-to-kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:31 +0000 UTC to 2023-03-08 10:10:31 +0000 UTC (now=2022-03-08 10:40:57.593685905 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593709       1 tlsconfig.go:178] loaded client CA [4/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-bootstrap-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:27 +0000 UTC to 2032-03-05 10:10:27 +0000 UTC (now=2022-03-08 10:40:57.5937027 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593726       1 tlsconfig.go:178] loaded client CA [5/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-csr-signer_@1646735926" [] issuer="kubelet-signer" (2022-03-08 10:38:45 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.593717922 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593743       1 tlsconfig.go:178] loaded client CA [6/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "openshift-kube-apiserver-operator_node-system-admin-signer@1646735931" [] issuer="<self>" (2022-03-08 10:38:50 +0000 UTC to 2023-03-08 10:38:51 +0000 UTC (now=2022-03-08 10:40:57.593733119 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.593759       1 tlsconfig.go:178] loaded client CA [7/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "aggregator-signer" [] issuer="<self>" (2022-03-08 10:10:28 +0000 UTC to 2022-03-09 10:10:28 +0000 UTC (now=2022-03-08 10:40:57.593753026 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.594008       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-kube-controller-manager-operator.svc" [serving] validServingFor=[metrics.openshift-kube-controller-manager-operator.svc,metrics.openshift-kube-controller-manager-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:10 +0000 UTC to 2024-03-07 10:39:11 +0000 UTC (now=2022-03-08 10:40:57.59399796 +0000 UTC))
2022-03-08T10:40:57.597582455Z I0308 10:40:57.594266       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736006" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736006" (2022-03-08 09:40:06 +0000 UTC to 2023-03-08 09:40:06 +0000 UTC (now=2022-03-08 10:40:57.594250406 +0000 UTC))
2022-03-08T10:40:58.468656498Z I0308 10:40:58.468574       1 request.go:645] Throttling request took 1.195536692s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller
2022-03-08T10:40:58.871935390Z I0308 10:40:58.871850       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because 
2022-03-08T10:41:03.272346453Z I0308 10:41:03.272269       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because 
2022-03-08T10:41:06.071410498Z E0308 10:41:06.071364       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:06.071545598Z I0308 10:41:06.071522       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:08.872980558Z E0308 10:41:08.872917       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:08.873304495Z I0308 10:41:08.873274       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:15.473123105Z I0308 10:41:15.471398       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because 
2022-03-08T10:41:21.072195954Z I0308 10:41:21.072046       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because 
2022-03-08T10:41:23.697726706Z I0308 10:41:23.697656       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:52Z","message":"StaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"NodeInstaller_InstallerPodFailed::StaticPods_Error","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:23.714626372Z I0308 10:41:23.714568       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "StaticPodsDegraded: pods \"kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com\" not found\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending" to "StaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"cluster-policy-controller\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-cert-syncer\" is waiting: ContainerCreating: \nStaticPodsDegraded: pod/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com container \"kube-controller-manager-recovery-controller\" is waiting: ContainerCreating: \nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending"
2022-03-08T10:41:24.077812453Z I0308 10:41:24.077754       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because static pod is pending
2022-03-08T10:41:24.868716713Z I0308 10:41:24.868647       1 request.go:645] Throttling request took 1.17150763s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/revision-pruner-4-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:26.068767059Z I0308 10:41:26.068678       1 request.go:645] Throttling request took 1.395847434s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:26.472541595Z I0308 10:41:26.472445       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:26.472750615Z E0308 10:41:26.472721       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:27.073547012Z I0308 10:41:27.073271       1 request.go:645] Throttling request took 1.201197334s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/kube-controller-manager-sa
2022-03-08T10:41:27.472287520Z I0308 10:41:27.472217       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because static pod is pending
2022-03-08T10:41:32.668708606Z I0308 10:41:32.668615       1 request.go:645] Throttling request took 1.090855321s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/services/kube-controller-manager
2022-03-08T10:41:33.670371072Z I0308 10:41:33.669615       1 request.go:645] Throttling request took 1.080403809s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-apiserver/pods?labelSelector=app%3Dopenshift-kube-apiserver
2022-03-08T10:41:34.507472577Z I0308 10:41:34.507394       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:41:34Z","message":"NodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"StaticPodsAvailable: 0 nodes are active; 3 nodes are at revision 0; 0 nodes have achieved new revision 6","reason":"StaticPods_ZeroNodesActive","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:34.546923557Z I0308 10:41:34.546836       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded changed from True to False ("NodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending")
2022-03-08T10:41:34.677906286Z I0308 10:41:34.677833       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because static pod is pending
2022-03-08T10:41:34.868758872Z I0308 10:41:34.868659       1 request.go:645] Throttling request took 1.194892028s, request: GET:https://172.30.0.1:443/api/v1/namespaces/default/endpoints/kubernetes
2022-03-08T10:41:34.880573454Z E0308 10:41:34.880512       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:34.880697809Z I0308 10:41:34.880666       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:35.877394812Z I0308 10:41:35.869709       1 request.go:645] Throttling request took 1.330818586s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:37.068829990Z I0308 10:41:37.068727       1 request.go:645] Throttling request took 1.446634904s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/installer-sa
2022-03-08T10:41:37.887277714Z I0308 10:41:37.887181       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:37.887432654Z E0308 10:41:37.887396       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:38.268754878Z I0308 10:41:38.268692       1 request.go:645] Throttling request took 1.484525961s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/services/kube-controller-manager
2022-03-08T10:41:38.872254636Z I0308 10:41:38.872195       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because static pod is pending
2022-03-08T10:41:39.469304413Z I0308 10:41:39.468722       1 request.go:645] Throttling request took 1.193101276s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/kube-controller-manager-sa
2022-03-08T10:41:40.274732037Z E0308 10:41:40.272872       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:40.274732037Z I0308 10:41:40.273282       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:40.668753640Z I0308 10:41:40.668621       1 request.go:645] Throttling request took 1.193933099s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/localhost-recovery-client
2022-03-08T10:41:42.268742707Z I0308 10:41:42.268663       1 request.go:645] Throttling request took 1.150432422s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-apiserver/pods?labelSelector=app%3Dopenshift-kube-apiserver
2022-03-08T10:41:42.571101416Z I0308 10:41:42.570993       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because static pod is pending
2022-03-08T10:41:43.268756737Z I0308 10:41:43.268660       1 request.go:645] Throttling request took 1.160936078s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/serviceaccounts/installer-sa
2022-03-08T10:41:43.479940905Z I0308 10:41:43.479883       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:43.480314433Z E0308 10:41:43.480286       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:45.473291883Z E0308 10:41:45.473222       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:45.473365065Z I0308 10:41:45.473288       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:45.675004989Z I0308 10:41:45.674933       1 installer_controller.go:331] "master-2.case03116926.lab.upshift.rdu2.redhat.com" is in transition to 6, but has not made progress because static pod is pending
2022-03-08T10:41:47.268724953Z I0308 10:41:47.268632       1 request.go:645] Throttling request took 1.016182203s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/installer-6-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:47.480161735Z I0308 10:41:47.480077       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:47.480386908Z E0308 10:41:47.480344       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:48.268776216Z I0308 10:41:48.268714       1 request.go:645] Throttling request took 1.196123434s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:49.468694683Z I0308 10:41:49.468636       1 request.go:645] Throttling request took 1.191370507s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:49.671743340Z I0308 10:41:49.671699       1 installer_controller.go:318] "master-2.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:41:49.671743340Z  NodeName: (string) (len=49) "master-2.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:41:49.671743340Z  CurrentRevision: (int32) 6,
2022-03-08T10:41:49.671743340Z  TargetRevision: (int32) 0,
2022-03-08T10:41:49.671743340Z  LastFailedRevision: (int32) 0,
2022-03-08T10:41:49.671743340Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:41:49.671743340Z }
2022-03-08T10:41:49.671743340Z  because static pod is ready
2022-03-08T10:41:49.692001996Z I0308 10:41:49.691944       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeCurrentRevisionChanged' Updated node "master-2.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 6 because static pod is ready
2022-03-08T10:41:49.692106234Z I0308 10:41:49.692084       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:41:34Z","message":"NodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:41:49Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 6","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:49.704551661Z I0308 10:41:49.704496       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Progressing message changed from "NodeInstallerProgressing: 3 nodes are at revision 0; 0 nodes have achieved new revision 6" to "NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 6",Available changed from False to True ("StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 6")
2022-03-08T10:41:49.872493544Z I0308 10:41:49.872425       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:49.872578275Z E0308 10:41:49.872497       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:50.868703630Z I0308 10:41:50.868621       1 request.go:645] Throttling request took 1.176086528s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:51.868727388Z I0308 10:41:51.868660       1 request.go:645] Throttling request took 1.396922758s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-1.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:52.471755213Z I0308 10:41:52.471669       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerStuck' unexpected addresses: 10.0.89.186
2022-03-08T10:41:52.471847537Z E0308 10:41:52.471729       1 satokensigner_controller.go:392] key failed with : unexpected addresses: 10.0.89.186
2022-03-08T10:41:53.068708046Z I0308 10:41:53.068638       1 request.go:645] Throttling request took 1.195726381s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/kube-controller-manager-master-2.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:54.290181836Z I0308 10:41:54.290110       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'PodCreated' Created Pod/revision-pruner-6-master-2.case03116926.lab.upshift.rdu2.redhat.com -n openshift-kube-controller-manager because it was missing
2022-03-08T10:41:55.872374077Z I0308 10:41:55.872329       1 installer_controller.go:350] node master-0.case03116926.lab.upshift.rdu2.redhat.com with revision 0 is the oldest not matching its expected revision 0 and needs new revision 6
2022-03-08T10:41:55.872523758Z I0308 10:41:55.872500       1 installer_controller.go:359] "master-0.case03116926.lab.upshift.rdu2.redhat.com" moving to (v1.NodeStatus) {
2022-03-08T10:41:55.872523758Z  NodeName: (string) (len=49) "master-0.case03116926.lab.upshift.rdu2.redhat.com",
2022-03-08T10:41:55.872523758Z  CurrentRevision: (int32) 0,
2022-03-08T10:41:55.872523758Z  TargetRevision: (int32) 6,
2022-03-08T10:41:55.872523758Z  LastFailedRevision: (int32) 4,
2022-03-08T10:41:55.872523758Z  LastFailedRevisionErrors: ([]string) <nil>
2022-03-08T10:41:55.872523758Z }
2022-03-08T10:41:55.899219555Z I0308 10:41:55.899149       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'NodeTargetRevisionChanged' Updating node "master-0.case03116926.lab.upshift.rdu2.redhat.com" from revision 0 to 6 because node master-0.case03116926.lab.upshift.rdu2.redhat.com with revision 0 is the oldest not matching its expected revision 0
2022-03-08T10:41:55.904632441Z I0308 10:41:55.904544       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:41:34Z","message":"NodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: ; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:41:49Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 6","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:55.920972636Z I0308 10:41:55.920923       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded message changed from "NodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: static pod of revision 4 has been installed, but is not ready while new revision 5 is pending; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending" to "NodeControllerDegraded: All master nodes are ready\nNodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: ; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending"
2022-03-08T10:41:56.268681603Z I0308 10:41:56.268620       1 request.go:645] Throttling request took 1.190040641s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller
2022-03-08T10:41:57.268689805Z I0308 10:41:57.268627       1 request.go:645] Throttling request took 1.366629676s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods/revision-pruner-4-master-0.case03116926.lab.upshift.rdu2.redhat.com
2022-03-08T10:41:57.675260510Z I0308 10:41:57.675181       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SATokenSignerControllerOK' found expected kube-apiserver endpoints
2022-03-08T10:41:58.269208683Z I0308 10:41:58.269134       1 request.go:645] Throttling request took 1.585713748s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager-operator/configmaps/csr-controller-ca
2022-03-08T10:41:59.468849107Z I0308 10:41:59.468700       1 request.go:645] Throttling request took 1.137689874s, request: POST:https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager-operator/secrets
2022-03-08T10:42:24.805209816Z I0308 10:42:24.805083       1 status_controller.go:172] clusteroperator/kube-controller-manager diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:42:24Z","message":"NodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: ; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending","reason":"NodeInstaller_InstallerPodFailed","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:54Z","message":"NodeInstallerProgressing: 2 nodes are at revision 0; 1 nodes are at revision 6","reason":"NodeInstaller","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:41:49Z","message":"StaticPodsAvailable: 1 nodes are active; 2 nodes are at revision 0; 1 nodes are at revision 6","reason":"AsExpected","status":"True","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:45Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:42:32.672870361Z I0308 10:42:32.672744       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'InstallerPodFailed' Failed to create installer pod for revision 6 on node "master-0.case03116926.lab.upshift.rdu2.redhat.com": context deadline exceeded
2022-03-08T10:42:32.672968517Z I0308 10:42:32.672951       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PodCreateFailed' Failed to create Pod/installer-6-master-0.case03116926.lab.upshift.rdu2.redhat.com -n openshift-kube-controller-manager: context deadline exceeded
2022-03-08T10:42:33.476261982Z I0308 10:42:33.476172       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'SecretCreateFailed' Failed to create Secret/next-service-account-private-key -n openshift-kube-controller-manager-operator: Timeout: request did not complete within requested timeout 34s
2022-03-08T10:42:41.977147832Z E0308 10:42:41.974276       1 leaderelection.go:321] error retrieving resource lock openshift-kube-controller-manager-operator/kube-controller-manager-operator-lock: Get "https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager-operator/configmaps/kube-controller-manager-operator-lock?timeout=35s": net/http: request canceled (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974376       1 leaderelection.go:278] failed to renew lease openshift-kube-controller-manager-operator/kube-controller-manager-operator-lock: timed out waiting for the condition
2022-03-08T10:42:41.977147832Z E0308 10:42:41.974430       1 leaderelection.go:297] Failed to release lock: resource name may not be empty
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974514       1 base_controller.go:166] Shutting down StatusSyncer_kube-controller-manager ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974567       1 base_controller.go:166] Shutting down RemoveStaleConditionsController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974643       1 base_controller.go:113] Shutting down worker of RemoveStaleConditionsController controller ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974643       1 reflector.go:213] Stopping reflector *v1.Network (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974655       1 base_controller.go:103] All RemoveStaleConditionsController workers have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974686       1 base_controller.go:144] All StatusSyncer_kube-controller-manager post start hooks have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974699       1 base_controller.go:166] Shutting down CertRotationController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974703       1 base_controller.go:144] All CertRotationController post start hooks have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974713       1 base_controller.go:166] Shutting down ResourceSyncController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974721       1 satokensigner_controller.go:372] Shutting down SATokenSignerController
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974735       1 base_controller.go:166] Shutting down ConfigObserver ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974734       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974748       1 base_controller.go:166] Shutting down NodeController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974762       1 base_controller.go:166] Shutting down StaticResourceController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974771       1 base_controller.go:166] Shutting down StaticResourceController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974780       1 base_controller.go:166] Shutting down InstallerStateController ...
2022-03-08T10:42:41.977147832Z E0308 10:42:41.974880       1 base_controller.go:250] "InstallerStateController" controller failed to sync "key", err: Get "https://172.30.0.1:443/api/v1/namespaces/openshift-kube-controller-manager/pods?labelSelector=app%3Dinstaller": context canceled
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974908       1 base_controller.go:113] Shutting down worker of InstallerStateController controller ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974913       1 base_controller.go:103] All InstallerStateController workers have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974918       1 controller_manager.go:54] InstallerStateController controller terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974926       1 base_controller.go:166] Shutting down InstallerController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974937       1 base_controller.go:166] Shutting down UnsupportedConfigOverridesController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.974946       1 base_controller.go:166] Shutting down StaticPodStateController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975189       1 reflector.go:213] Stopping reflector *v1.Node (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975231       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975252       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975266       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975281       1 reflector.go:213] Stopping reflector *v1.Role (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975300       1 reflector.go:213] Stopping reflector *v1.Role (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975345       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975361       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975376       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975393       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975407       1 reflector.go:213] Stopping reflector *v1.ConfigMap (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975423       1 reflector.go:213] Stopping reflector *v1.RoleBinding (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975444       1 reflector.go:213] Stopping reflector *v1.RoleBinding (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975458       1 reflector.go:213] Stopping reflector *v1.Pod (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975486       1 reflector.go:213] Stopping reflector *v1.Namespace (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975503       1 reflector.go:213] Stopping reflector *v1.ClusterOperator (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975685       1 reflector.go:213] Stopping reflector *v1.Secret (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975703       1 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"", Name:"", UID:"", APIVersion:"v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' kube-controller-manager-operator-7bb9b6d8b8-h8hjn_d927429a-bf3d-4a4f-a9de-203494db50c9 stopped leading
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975729       1 reflector.go:213] Stopping reflector *v1.Proxy (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975836       1 base_controller.go:113] Shutting down worker of StatusSyncer_kube-controller-manager controller ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975840       1 base_controller.go:166] Shutting down PruneController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975844       1 base_controller.go:103] All StatusSyncer_kube-controller-manager workers have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975858       1 base_controller.go:166] Shutting down RevisionController ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975858       1 reflector.go:213] Stopping reflector *v1.FeatureGate (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975867       1 base_controller.go:166] Shutting down LoggingSyncer ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975888       1 reflector.go:213] Stopping reflector *v1.Infrastructure (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975906       1 base_controller.go:113] Shutting down worker of CertRotationController controller ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975912       1 base_controller.go:103] All CertRotationController workers have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975923       1 base_controller.go:113] Shutting down worker of ResourceSyncController controller ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975927       1 base_controller.go:103] All ResourceSyncController workers have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975944       1 base_controller.go:113] Shutting down worker of ConfigObserver controller ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975948       1 base_controller.go:103] All ConfigObserver workers have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975958       1 base_controller.go:113] Shutting down worker of NodeController controller ...
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975963       1 base_controller.go:103] All NodeController workers have been terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975968       1 controller_manager.go:54] NodeController controller terminated
2022-03-08T10:42:41.977147832Z I0308 10:42:41.975984       1 base_controller.go:113] Shutting down worker of UnsupportedConfigOverridesController cont2022-03-08T10:42:41.977221887Z roller ...
2022-03-08T10:42:41.977221887Z I0308 10:42:41.975995       1 base_controller.go:103] All UnsupportedConfigOverridesController workers have been terminated
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976000       1 controller_manager.go:54] UnsupportedConfigOverridesController controller terminated
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976091       1 reflector.go:213] Stopping reflector *v1.ServiceAccount (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976106       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/kube-controller-manager changed: Degraded changed from False to True ("NodeInstallerDegraded: 1 nodes are failing on revision 4:\nNodeInstallerDegraded: ; 1 nodes are failing on revision 5:\nNodeInstallerDegraded: static pod of revision 5 has been installed, but is not ready while new revision 6 is pending")
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976131       1 reflector.go:213] Stopping reflector *v1.ClusterRole (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976146       1 reflector.go:213] Stopping reflector *unstructured.Unstructured (12h0m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976172       1 base_controller.go:113] Shutting down worker of LoggingSyncer controller ...
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976178       1 base_controller.go:103] All LoggingSyncer workers have been terminated
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976183       1 controller_manager.go:54] LoggingSyncer controller terminated
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976197       1 base_controller.go:113] Shutting down worker of RevisionController controller ...
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976202       1 base_controller.go:103] All RevisionController workers have been terminated
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976206       1 controller_manager.go:54] RevisionController controller terminated
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976221       1 reflector.go:213] Stopping reflector *v1.ClusterRoleBinding (10m0s) from k8s.io/client-go@v0.19.0/tools/cache/reflector.go:156
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976236       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-kube-controller-manager-operator", Name:"kube-controller-manager-operator", UID:"eab5c6dd-ee48-42b1-9a41-e97928d02331", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'MissingVersion' no image found for operand pod
2022-03-08T10:42:41.977221887Z I0308 10:42:41.976479       1 targetconfigcontroller.go:731] Shutting down TargetConfigController
2022-03-08T10:42:41.977221887Z W0308 10:42:41.976688       1 leaderelection.go:75] leader election lost
