2022-03-08T10:40:00.492683660Z Copying system trust bundle
2022-03-08T10:40:00.563672513Z I0308 10:40:00.563512       1 cmd.go:200] Using service-serving-cert provided certificates
2022-03-08T10:40:00.564806775Z I0308 10:40:00.564772       1 observer_polling.go:159] Starting file observer
2022-03-08T10:40:00.566625472Z I0308 10:40:00.564729       1 observer_polling.go:159] Starting file observer
2022-03-08T10:40:00.587370606Z I0308 10:40:00.587315       1 builder.go:238] cluster-authentication-operator version v4.6.0-202111301700.p0.g938196e.assembly.stream-0-g90f7d7f-
2022-03-08T10:40:00.591024911Z I0308 10:40:00.588781       1 dynamic_serving_content.go:111] Loaded a new cert/key pair for "serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"
2022-03-08T10:40:01.034792742Z I0308 10:40:01.034753       1 requestheader_controller.go:244] Loaded a new request header values for RequestHeaderAuthRequestController
2022-03-08T10:40:01.044651060Z W0308 10:40:01.044591       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected.
2022-03-08T10:40:01.044651060Z W0308 10:40:01.044620       1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected.
2022-03-08T10:40:01.045514817Z I0308 10:40:01.045484       1 leaderelection.go:243] attempting to acquire leader lease  openshift-authentication-operator/cluster-authentication-operator-lock...
2022-03-08T10:40:01.049126735Z I0308 10:40:01.049064       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-authentication-operator.svc" [serving] validServingFor=[metrics.openshift-authentication-operator.svc,metrics.openshift-authentication-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:16 +0000 UTC to 2024-03-07 10:39:17 +0000 UTC (now=2022-03-08 10:40:01.049034514 +0000 UTC))
2022-03-08T10:40:01.049337431Z I0308 10:40:01.049307       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736000" (2022-03-08 09:40:00 +0000 UTC to 2023-03-08 09:40:00 +0000 UTC (now=2022-03-08 10:40:01.049296054 +0000 UTC))
2022-03-08T10:40:01.049351826Z I0308 10:40:01.049339       1 secure_serving.go:197] Serving securely on [::]:8443
2022-03-08T10:40:01.049383094Z I0308 10:40:01.049370       1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
2022-03-08T10:40:01.049391652Z I0308 10:40:01.049386       1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
2022-03-08T10:40:01.049434775Z I0308 10:40:01.049407       1 dynamic_serving_content.go:130] Starting serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key
2022-03-08T10:40:01.049446926Z I0308 10:40:01.049440       1 tlsconfig.go:240] Starting DynamicServingCertificateController
2022-03-08T10:40:01.049526870Z I0308 10:40:01.049508       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
2022-03-08T10:40:01.049526870Z I0308 10:40:01.049518       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
2022-03-08T10:40:01.049544491Z I0308 10:40:01.049533       1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
2022-03-08T10:40:01.049544491Z I0308 10:40:01.049539       1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
2022-03-08T10:40:01.049735066Z I0308 10:40:01.049708       1 reflector.go:207] Starting reflector *v1.ConfigMap (12h0m0s) from k8s.io/apiserver/pkg/authentication/request/headerrequest/requestheader_controller.go:172
2022-03-08T10:40:01.050680932Z I0308 10:40:01.050650       1 reflector.go:207] Starting reflector *v1.ConfigMap (12h0m0s) from k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206
2022-03-08T10:40:01.051073564Z I0308 10:40:01.051055       1 leaderelection.go:253] successfully acquired lease openshift-authentication-operator/cluster-authentication-operator-lock
2022-03-08T10:40:01.051530514Z I0308 10:40:01.051504       1 reflector.go:207] Starting reflector *v1.ConfigMap (12h0m0s) from k8s.io/apiserver/pkg/server/dynamiccertificates/configmap_cafile_content.go:206
2022-03-08T10:40:01.051545093Z I0308 10:40:01.051522       1 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"openshift-authentication-operator", Name:"cluster-authentication-operator-lock", UID:"6d0e64cd-3317-4999-8fb7-f64b551fa305", APIVersion:"v1", ResourceVersion:"9834", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' authentication-operator-6fcbcf79b6-5h2gz_b6c8ca0e-47ff-4466-98c3-77bac1018604 became leader
2022-03-08T10:40:01.102569087Z I0308 10:40:01.102503       1 base_controller.go:66] Waiting for caches to sync for LoggingSyncer
2022-03-08T10:40:01.102700182Z I0308 10:40:01.102674       1 reflector.go:207] Starting reflector *v1.Route (20m0s) from github.com/openshift/client-go/route/informers/externalversions/factory.go:101
2022-03-08T10:40:01.102859454Z I0308 10:40:01.102822       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.103078701Z I0308 10:40:01.103021       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.103170041Z I0308 10:40:01.103148       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.103327399Z I0308 10:40:01.103298       1 reflector.go:207] Starting reflector *v1.Authentication (30s) from github.com/openshift/client-go/operator/informers/externalversions/factory.go:101
2022-03-08T10:40:01.103389141Z I0308 10:40:01.103376       1 reflector.go:207] Starting reflector *v1.Node (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.103554423Z I0308 10:40:01.103534       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.103719923Z I0308 10:40:01.103700       1 reflector.go:207] Starting reflector *v1.IngressController (30s) from github.com/openshift/client-go/operator/informers/externalversions/factory.go:101
2022-03-08T10:40:01.103743396Z I0308 10:40:01.103728       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.103880315Z I0308 10:40:01.103830       1 base_controller.go:66] Waiting for caches to sync for OAuthRouteCheckEndpointAccessibleController
2022-03-08T10:40:01.103952329Z I0308 10:40:01.103938       1 reflector.go:207] Starting reflector *v1.DaemonSet (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.104044544Z I0308 10:40:01.104033       1 reflector.go:207] Starting reflector *v1.OAuth (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.104119340Z I0308 10:40:01.104103       1 base_controller.go:66] Waiting for caches to sync for OAuthServiceCheckEndpointAccessibleController
2022-03-08T10:40:01.104135747Z I0308 10:40:01.104127       1 base_controller.go:66] Waiting for caches to sync for OAuthServiceEndpointsCheckEndpointAccessibleController
2022-03-08T10:40:01.104157050Z I0308 10:40:01.104139       1 base_controller.go:66] Waiting for caches to sync for IngressNodesAvailableController
2022-03-08T10:40:01.104157050Z I0308 10:40:01.104154       1 base_controller.go:66] Waiting for caches to sync for RemoveStaleConditionsController
2022-03-08T10:40:01.104174260Z I0308 10:40:01.104165       1 base_controller.go:66] Waiting for caches to sync for IngressStateController
2022-03-08T10:40:01.104188684Z I0308 10:40:01.104180       1 base_controller.go:66] Waiting for caches to sync for ConfigObserver
2022-03-08T10:40:01.104196390Z I0308 10:40:01.104192       1 base_controller.go:66] Waiting for caches to sync for MangeOAuthAPIController
2022-03-08T10:40:01.104219548Z I0308 10:40:01.104210       1 workload.go:139] Starting OAuthAPIServerControllerWorkloadController
2022-03-08T10:40:01.104233855Z I0308 10:40:01.104225       1 base_controller.go:66] Waiting for caches to sync for ResourceSyncController
2022-03-08T10:40:01.104247822Z I0308 10:40:01.104238       1 base_controller.go:66] Waiting for caches to sync for UnsupportedConfigOverridesController
2022-03-08T10:40:01.104287408Z I0308 10:40:01.104273       1 reflector.go:207] Starting reflector *v1.ClusterRoleBinding (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.104304000Z I0308 10:40:01.104295       1 base_controller.go:66] Waiting for caches to sync for APIServiceController_openshift-apiserver
2022-03-08T10:40:01.104318146Z I0308 10:40:01.104309       1 base_controller.go:66] Waiting for caches to sync for SecretRevisionPruneController
2022-03-08T10:40:01.104326428Z I0308 10:40:01.104322       1 base_controller.go:66] Waiting for caches to sync for RevisionController
2022-03-08T10:40:01.104482650Z I0308 10:40:01.104463       1 reflector.go:207] Starting reflector *v1.Node (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.104482650Z I0308 10:40:01.104477       1 base_controller.go:66] Waiting for caches to sync for StaticResourceController
2022-03-08T10:40:01.104495000Z I0308 10:40:01.104489       1 base_controller.go:66] Waiting for caches to sync for EncryptionConditionController
2022-03-08T10:40:01.104510457Z I0308 10:40:01.104500       1 base_controller.go:66] Waiting for caches to sync for NamespaceFinalizerController_openshift-oauth-apiserver
2022-03-08T10:40:01.104635805Z I0308 10:40:01.104621       1 reflector.go:207] Starting reflector *v1.Ingress (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.104730686Z I0308 10:40:01.104717       1 base_controller.go:66] Waiting for caches to sync for EncryptionKeyController
2022-03-08T10:40:01.104739742Z I0308 10:40:01.104730       1 base_controller.go:66] Waiting for caches to sync for EncryptionStateController
2022-03-08T10:40:01.104746060Z I0308 10:40:01.104738       1 base_controller.go:66] Waiting for caches to sync for EncryptionPruneController
2022-03-08T10:40:01.104753542Z I0308 10:40:01.104749       1 base_controller.go:66] Waiting for caches to sync for EncryptionMigrationController
2022-03-08T10:40:01.104931872Z I0308 10:40:01.104873       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.104931872Z I0308 10:40:01.104898       1 reflector.go:207] Starting reflector *v1.ServiceAccount (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.105057986Z I0308 10:40:01.105044       1 reflector.go:207] Starting reflector *v1.Authentication (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.105151878Z I0308 10:40:01.105136       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.105161469Z E0308 10:40:01.105154       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server could not find the requested resource (get routes.route.openshift.io)
2022-03-08T10:40:01.105288482Z I0308 10:40:01.105273       1 reflector.go:207] Starting reflector *v1.Proxy (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.105447750Z I0308 10:40:01.105425       1 reflector.go:207] Starting reflector *v1.ClusterOperator (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.105730182Z I0308 10:40:01.105707       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.105771063Z I0308 10:40:01.105760       1 reflector.go:207] Starting reflector *v1.APIServer (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.105937937Z I0308 10:40:01.105909       1 reflector.go:207] Starting reflector *v1.ServiceAccount (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106023645Z I0308 10:40:01.106012       1 reflector.go:207] Starting reflector *v1.Pod (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106059910Z I0308 10:40:01.106049       1 reflector.go:207] Starting reflector *v1.Console (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.106136862Z I0308 10:40:01.106125       1 reflector.go:207] Starting reflector *v1.Endpoints (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106241885Z I0308 10:40:01.106226       1 reflector.go:207] Starting reflector *v1.Service (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106328125Z I0308 10:40:01.106315       1 reflector.go:207] Starting reflector *v1.Infrastructure (20m0s) from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
2022-03-08T10:40:01.106380695Z I0308 10:40:01.106366       1 reflector.go:207] Starting reflector *v1.Deployment (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106619879Z I0308 10:40:01.106596       1 reflector.go:207] Starting reflector *v1.Namespace (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106646456Z I0308 10:40:01.106630       1 reflector.go:207] Starting reflector *v1.APIService (10m0s) from k8s.io/kube-aggregator/pkg/client/informers/externalversions/factory.go:117
2022-03-08T10:40:01.106692973Z I0308 10:40:01.106681       1 base_controller.go:66] Waiting for caches to sync for StatusSyncer_authentication
2022-03-08T10:40:01.106726273Z I0308 10:40:01.106717       1 base_controller.go:66] Waiting for caches to sync for ConfigObserver
2022-03-08T10:40:01.106744621Z I0308 10:40:01.106730       1 reflector.go:207] Starting reflector *v1.Service (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106765221Z I0308 10:40:01.106758       1 base_controller.go:66] Waiting for caches to sync for Deployment
2022-03-08T10:40:01.106787795Z I0308 10:40:01.106781       1 base_controller.go:66] Waiting for caches to sync for ManagementStateController
2022-03-08T10:40:01.106814421Z I0308 10:40:01.106807       1 base_controller.go:66] Waiting for caches to sync for MetadataController
2022-03-08T10:40:01.106845399Z I0308 10:40:01.106826       1 reflector.go:207] Starting reflector *v1.Endpoints (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.106947297Z I0308 10:40:01.106930       1 base_controller.go:66] Waiting for caches to sync for PayloadConfig
2022-03-08T10:40:01.106987406Z I0308 10:40:01.106978       1 base_controller.go:66] Waiting for caches to sync for RouterCertsDomainValidationController
2022-03-08T10:40:01.107025725Z I0308 10:40:01.107018       1 base_controller.go:66] Waiting for caches to sync for ServiceCAController
2022-03-08T10:40:01.107329490Z I0308 10:40:01.107257       1 base_controller.go:66] Waiting for caches to sync for StaticResourceController
2022-03-08T10:40:01.107371977Z I0308 10:40:01.107362       1 base_controller.go:66] Waiting for caches to sync for TargetVersion
2022-03-08T10:40:01.107400205Z I0308 10:40:01.107392       1 base_controller.go:66] Waiting for caches to sync for WellKnownReadyController
2022-03-08T10:40:01.107615562Z I0308 10:40:01.107589       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.107789612Z I0308 10:40:01.107771       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.107898151Z I0308 10:40:01.105052       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.108008508Z I0308 10:40:01.107992       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.108047344Z I0308 10:40:01.107026       1 reflector.go:207] Starting reflector *v1alpha1.StorageVersionMigration (30m0s) from sigs.k8s.io/kube-storage-version-migrator/pkg/clients/informer/factory.go:117
2022-03-08T10:40:01.108167696Z I0308 10:40:01.108150       1 reflector.go:207] Starting reflector *v1.Endpoints (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.108308247Z I0308 10:40:01.108290       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.108435       1 reflector.go:207] Starting reflector *v1.Service (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.108554       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.108703       1 reflector.go:207] Starting reflector *v1.Endpoints (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.108900       1 reflector.go:207] Starting reflector *v1.Secret (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.108917       1 reflector.go:207] Starting reflector *v1.Deployment (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.108440       1 reflector.go:207] Starting reflector *v1.ConfigMap (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.109203       1 reflector.go:207] Starting reflector *v1.Namespace (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.113545239Z I0308 10:40:01.109270       1 reflector.go:207] Starting reflector *v1.Pod (10m0s) from k8s.io/client-go/informers/factory.go:134
2022-03-08T10:40:01.149713575Z I0308 10:40:01.149593       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file 
2022-03-08T10:40:01.149713575Z I0308 10:40:01.149648       1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController 
2022-03-08T10:40:01.149793599Z I0308 10:40:01.149781       1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file 
2022-03-08T10:40:01.150300110Z I0308 10:40:01.150202       1 tlsconfig.go:178] loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "aggregator-signer" [] issuer="<self>" (2022-03-08 10:10:28 +0000 UTC to 2022-03-09 10:10:28 +0000 UTC (now=2022-03-08 10:40:01.150182241 +0000 UTC))
2022-03-08T10:40:01.150557246Z I0308 10:40:01.150517       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-authentication-operator.svc" [serving] validServingFor=[metrics.openshift-authentication-operator.svc,metrics.openshift-authentication-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:16 +0000 UTC to 2024-03-07 10:39:17 +0000 UTC (now=2022-03-08 10:40:01.150502108 +0000 UTC))
2022-03-08T10:40:01.150826457Z I0308 10:40:01.150795       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736000" (2022-03-08 09:40:00 +0000 UTC to 2023-03-08 09:40:00 +0000 UTC (now=2022-03-08 10:40:01.150782424 +0000 UTC))
2022-03-08T10:40:01.151352105Z I0308 10:40:01.151303       1 tlsconfig.go:178] loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "admin-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:26 +0000 UTC to 2032-03-05 10:10:26 +0000 UTC (now=2022-03-08 10:40:01.1512884 +0000 UTC))
2022-03-08T10:40:01.151352105Z I0308 10:40:01.151340       1 tlsconfig.go:178] loaded client CA [1/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:01.151331061 +0000 UTC))
2022-03-08T10:40:01.151369904Z I0308 10:40:01.151358       1 tlsconfig.go:178] loaded client CA [2/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-control-plane-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2023-03-08 10:10:30 +0000 UTC (now=2022-03-08 10:40:01.151349673 +0000 UTC))
2022-03-08T10:40:01.151400105Z I0308 10:40:01.151376       1 tlsconfig.go:178] loaded client CA [3/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-apiserver-to-kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:31 +0000 UTC to 2023-03-08 10:10:31 +0000 UTC (now=2022-03-08 10:40:01.151366327 +0000 UTC))
2022-03-08T10:40:01.151408028Z I0308 10:40:01.151397       1 tlsconfig.go:178] loaded client CA [4/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-bootstrap-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:27 +0000 UTC to 2032-03-05 10:10:27 +0000 UTC (now=2022-03-08 10:40:01.151390265 +0000 UTC))
2022-03-08T10:40:01.151427754Z I0308 10:40:01.151412       1 tlsconfig.go:178] loaded client CA [5/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "aggregator-signer" [] issuer="<self>" (2022-03-08 10:10:28 +0000 UTC to 2022-03-09 10:10:28 +0000 UTC (now=2022-03-08 10:40:01.151406057 +0000 UTC))
2022-03-08T10:40:01.151781468Z I0308 10:40:01.151713       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-authentication-operator.svc" [serving] validServingFor=[metrics.openshift-authentication-operator.svc,metrics.openshift-authentication-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:16 +0000 UTC to 2024-03-07 10:39:17 +0000 UTC (now=2022-03-08 10:40:01.151699476 +0000 UTC))
2022-03-08T10:40:01.152078895Z I0308 10:40:01.152035       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736000" (2022-03-08 09:40:00 +0000 UTC to 2023-03-08 09:40:00 +0000 UTC (now=2022-03-08 10:40:01.152024401 +0000 UTC))
2022-03-08T10:40:01.202819180Z I0308 10:40:01.202725       1 base_controller.go:72] Caches are synced for LoggingSyncer 
2022-03-08T10:40:01.202819180Z I0308 10:40:01.202784       1 base_controller.go:109] Starting #1 worker of LoggingSyncer controller ...
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204170       1 base_controller.go:72] Caches are synced for IngressNodesAvailableController 
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204203       1 base_controller.go:72] Caches are synced for RemoveStaleConditionsController 
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204221       1 base_controller.go:109] Starting #1 worker of RemoveStaleConditionsController controller ...
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204236       1 base_controller.go:72] Caches are synced for MangeOAuthAPIController 
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204208       1 base_controller.go:109] Starting #1 worker of IngressNodesAvailableController controller ...
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204243       1 base_controller.go:109] Starting #1 worker of MangeOAuthAPIController controller ...
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204256       1 base_controller.go:72] Caches are synced for UnsupportedConfigOverridesController 
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204262       1 base_controller.go:109] Starting #1 worker of UnsupportedConfigOverridesController controller ...
2022-03-08T10:40:01.204874958Z I0308 10:40:01.204456       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:01.206946929Z I0308 10:40:01.206900       1 base_controller.go:72] Caches are synced for StatusSyncer_authentication 
2022-03-08T10:40:01.206946929Z I0308 10:40:01.206921       1 base_controller.go:109] Starting #1 worker of StatusSyncer_authentication controller ...
2022-03-08T10:40:01.206982440Z I0308 10:40:01.206944       1 base_controller.go:72] Caches are synced for ManagementStateController 
2022-03-08T10:40:01.206982440Z I0308 10:40:01.206948       1 base_controller.go:109] Starting #1 worker of ManagementStateController controller ...
2022-03-08T10:40:01.607945625Z I0308 10:40:01.607033       1 base_controller.go:72] Caches are synced for RouterCertsDomainValidationController 
2022-03-08T10:40:01.607945625Z I0308 10:40:01.607061       1 base_controller.go:109] Starting #1 worker of RouterCertsDomainValidationController controller ...
2022-03-08T10:40:02.292830360Z I0308 10:40:02.292752       1 request.go:645] Throttling request took 1.186545794s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-authentication/endpoints?limit=500&resourceVersion=0
2022-03-08T10:40:02.305569424Z I0308 10:40:02.304956       1 base_controller.go:72] Caches are synced for OAuthServiceEndpointsCheckEndpointAccessibleController 
2022-03-08T10:40:02.305569424Z I0308 10:40:02.304985       1 base_controller.go:72] Caches are synced for IngressStateController 
2022-03-08T10:40:02.305569424Z I0308 10:40:02.305003       1 base_controller.go:109] Starting #1 worker of IngressStateController controller ...
2022-03-08T10:40:02.305569424Z I0308 10:40:02.304989       1 base_controller.go:109] Starting #1 worker of OAuthServiceEndpointsCheckEndpointAccessibleController controller ...
2022-03-08T10:40:02.305569424Z E0308 10:40:02.305189       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:02.310663131Z E0308 10:40:02.310606       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:02.321127459Z E0308 10:40:02.321070       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:02.341444919Z E0308 10:40:02.341379       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:02.383945270Z E0308 10:40:02.381648       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:02.462372901Z E0308 10:40:02.462306       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:02.504302015Z I0308 10:40:02.504224       1 base_controller.go:72] Caches are synced for OAuthServiceCheckEndpointAccessibleController 
2022-03-08T10:40:02.504302015Z I0308 10:40:02.504254       1 base_controller.go:109] Starting #1 worker of OAuthServiceCheckEndpointAccessibleController controller ...
2022-03-08T10:40:02.507222412Z I0308 10:40:02.507148       1 base_controller.go:72] Caches are synced for ServiceCAController 
2022-03-08T10:40:02.507222412Z I0308 10:40:02.507172       1 base_controller.go:109] Starting #1 worker of ServiceCAController controller ...
2022-03-08T10:40:02.622622662Z E0308 10:40:02.622543       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:02.655499397Z E0308 10:40:02.655421       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server could not find the requested resource (get routes.route.openshift.io)
2022-03-08T10:40:02.707527275Z I0308 10:40:02.707460       1 base_controller.go:72] Caches are synced for StaticResourceController 
2022-03-08T10:40:02.707527275Z I0308 10:40:02.707485       1 base_controller.go:109] Starting #1 worker of StaticResourceController controller ...
2022-03-08T10:40:02.943134480Z E0308 10:40:02.943070       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:03.497888301Z I0308 10:40:03.495936       1 request.go:645] Throttling request took 2.388075568s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-authentication-operator/secrets?limit=500&resourceVersion=0
2022-03-08T10:40:03.583740875Z E0308 10:40:03.583412       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:03.706990322Z I0308 10:40:03.706917       1 base_controller.go:72] Caches are synced for ConfigObserver 
2022-03-08T10:40:03.706990322Z I0308 10:40:03.706948       1 base_controller.go:109] Starting #1 worker of ConfigObserver controller ...
2022-03-08T10:40:03.806148889Z E0308 10:40:03.806053       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:03.806737695Z I0308 10:40:03.806202       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:03.806737695Z I0308 10:40:03.806721       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:03.838778369Z E0308 10:40:03.838692       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:03.839711976Z I0308 10:40:03.839423       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:03.839743093Z E0308 10:40:03.839717       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:03.842355298Z I0308 10:40:03.841113       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:03.842355298Z I0308 10:40:03.841765       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused"
2022-03-08T10:40:04.693536781Z I0308 10:40:04.692484       1 request.go:645] Throttling request took 3.583871691s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/configmaps?limit=500&resourceVersion=0
2022-03-08T10:40:04.863865597Z E0308 10:40:04.863762       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:04.903829689Z E0308 10:40:04.903744       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:04.904388875Z I0308 10:40:04.904349       1 base_controller.go:72] Caches are synced for APIServiceController_openshift-apiserver 
2022-03-08T10:40:04.904388875Z I0308 10:40:04.904367       1 base_controller.go:109] Starting #1 worker of APIServiceController_openshift-apiserver controller ...
2022-03-08T10:40:05.034878333Z I0308 10:40:05.034760       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:05.035418608Z I0308 10:40:05.035356       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server",Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused"
2022-03-08T10:40:05.063332323Z I0308 10:40:05.063226       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server"
2022-03-08T10:40:05.104355915Z I0308 10:40:05.104288       1 base_controller.go:72] Caches are synced for ConfigObserver 
2022-03-08T10:40:05.104355915Z I0308 10:40:05.104314       1 base_controller.go:109] Starting #1 worker of ConfigObserver controller ...
2022-03-08T10:40:05.104355915Z I0308 10:40:05.104333       1 base_controller.go:72] Caches are synced for RevisionController 
2022-03-08T10:40:05.104355915Z I0308 10:40:05.104340       1 base_controller.go:109] Starting #1 worker of RevisionController controller ...
2022-03-08T10:40:05.306962639Z I0308 10:40:05.305967       1 base_controller.go:72] Caches are synced for ResourceSyncController 
2022-03-08T10:40:05.306962639Z I0308 10:40:05.305990       1 base_controller.go:109] Starting #1 worker of ResourceSyncController controller ...
2022-03-08T10:40:05.449467302Z E0308 10:40:05.449409       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server could not find the requested resource (get routes.route.openshift.io)
2022-03-08T10:40:05.505824534Z I0308 10:40:05.504643       1 base_controller.go:72] Caches are synced for StaticResourceController 
2022-03-08T10:40:05.505824534Z I0308 10:40:05.504673       1 base_controller.go:109] Starting #1 worker of StaticResourceController controller ...
2022-03-08T10:40:05.695788237Z I0308 10:40:05.694940       1 request.go:645] Throttling request took 4.585604892s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/pods?limit=500&resourceVersion=0
2022-03-08T10:40:05.706283261Z I0308 10:40:05.704644       1 base_controller.go:72] Caches are synced for NamespaceFinalizerController_openshift-oauth-apiserver 
2022-03-08T10:40:05.706283261Z I0308 10:40:05.704670       1 base_controller.go:109] Starting #1 worker of NamespaceFinalizerController_openshift-oauth-apiserver controller ...
2022-03-08T10:40:05.706283261Z I0308 10:40:05.704980       1 base_controller.go:72] Caches are synced for SecretRevisionPruneController 
2022-03-08T10:40:05.706283261Z I0308 10:40:05.704987       1 base_controller.go:109] Starting #1 worker of SecretRevisionPruneController controller ...
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705032       1 base_controller.go:72] Caches are synced for EncryptionConditionController 
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705038       1 base_controller.go:109] Starting #1 worker of EncryptionConditionController controller ...
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705141       1 base_controller.go:72] Caches are synced for EncryptionMigrationController 
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705146       1 base_controller.go:109] Starting #1 worker of EncryptionMigrationController controller ...
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705159       1 base_controller.go:72] Caches are synced for EncryptionKeyController 
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705164       1 base_controller.go:109] Starting #1 worker of EncryptionKeyController controller ...
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705177       1 base_controller.go:72] Caches are synced for EncryptionStateController 
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705181       1 base_controller.go:109] Starting #1 worker of EncryptionStateController controller ...
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705194       1 base_controller.go:72] Caches are synced for EncryptionPruneController 
2022-03-08T10:40:05.706283261Z I0308 10:40:05.705198       1 base_controller.go:109] Starting #1 worker of EncryptionPruneController controller ...
2022-03-08T10:40:05.711953476Z E0308 10:40:05.709890       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.711953476Z I0308 10:40:05.710164       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.721913239Z E0308 10:40:05.720229       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.721913239Z I0308 10:40:05.720630       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.733752514Z E0308 10:40:05.733714       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.734581611Z I0308 10:40:05.734197       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.763168048Z E0308 10:40:05.763125       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.763644430Z I0308 10:40:05.763608       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.808413466Z E0308 10:40:05.808370       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.808795370Z I0308 10:40:05.808763       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.892549724Z E0308 10:40:05.892496       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.893096951Z I0308 10:40:05.893054       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:05.926307797Z E0308 10:40:05.926250       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:06.056471858Z E0308 10:40:06.056425       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:06.056592480Z I0308 10:40:06.056568       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:06.390675128Z E0308 10:40:06.390637       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:06.390857397Z I0308 10:40:06.390736       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:06.893880781Z I0308 10:40:06.892482       1 request.go:645] Throttling request took 1.187359372s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver
2022-03-08T10:40:06.950882963Z E0308 10:40:06.950526       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:07.042903030Z E0308 10:40:07.040710       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:07.042903030Z I0308 10:40:07.041125       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:07.974322423Z E0308 10:40:07.974283       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:08.329448849Z E0308 10:40:08.329394       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:08.329982546Z I0308 10:40:08.329942       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:08.998308982Z E0308 10:40:08.998256       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:10.022324704Z E0308 10:40:10.022275       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:10.893921291Z E0308 10:40:10.893828       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:10.894407971Z I0308 10:40:10.894363       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:11.046401291Z E0308 10:40:11.046350       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:11.809210038Z E0308 10:40:11.809167       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server could not find the requested resource (get routes.route.openshift.io)
2022-03-08T10:40:12.070336090Z E0308 10:40:12.070288       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:13.094300889Z E0308 10:40:13.094247       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:14.375294277Z E0308 10:40:14.375245       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:15.104093526Z E0308 10:40:15.104048       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:16.017452425Z E0308 10:40:16.017390       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:16.017690647Z I0308 10:40:16.017638       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:17.303811645Z E0308 10:40:17.303760       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:17.305919554Z I0308 10:40:17.304318       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:17.365663854Z E0308 10:40:17.363152       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:17.365663854Z I0308 10:40:17.363482       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:20.521181032Z E0308 10:40:20.518174       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: connect: connection refused
2022-03-08T10:40:23.627065074Z E0308 10:40:23.626915       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server could not find the requested resource (get routes.route.openshift.io)
2022-03-08T10:40:26.261614833Z E0308 10:40:26.261541       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:26.261991132Z I0308 10:40:26.261951       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:31.108277797Z E0308 10:40:31.108220       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:31.108996956Z I0308 10:40:31.108964       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:31.113162655Z E0308 10:40:31.113129       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:31.113456713Z I0308 10:40:31.113430       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:32.305318140Z E0308 10:40:32.305265       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:35.587135480Z E0308 10:40:35.587012       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:35.661929297Z E0308 10:40:35.661666       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:35.662035776Z I0308 10:40:35.662005       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:35.776212577Z I0308 10:40:35.776148       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:35.776651211Z E0308 10:40:35.776622       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:35.777681025Z I0308 10:40:35.777594       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:35.781987471Z E0308 10:40:35.781932       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:35.785096643Z I0308 10:40:35.784748       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:35.792197284Z I0308 10:40:35.792136       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
2022-03-08T10:40:35.824047328Z E0308 10:40:35.824005       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:40:35.826992664Z E0308 10:40:35.826946       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:35.827435481Z I0308 10:40:35.827405       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:35.828289615Z I0308 10:40:35.828253       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:35.835779273Z E0308 10:40:35.835727       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:35.836113669Z I0308 10:40:35.836043       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:35.840050570Z I0308 10:40:35.840000       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: connect: connection refused\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server"
2022-03-08T10:40:36.547726258Z E0308 10:40:36.547652       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server could not find the requested resource (get routes.route.openshift.io)
2022-03-08T10:40:40.824568304Z E0308 10:40:40.824496       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:40:45.838348822Z E0308 10:40:45.838271       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:45.838710290Z I0308 10:40:45.838683       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:45.839529479Z I0308 10:40:45.839491       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:45.841826279Z E0308 10:40:45.841763       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:45.842111398Z I0308 10:40:45.842072       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:45.851013840Z I0308 10:40:45.850967       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"
2022-03-08T10:40:45.863371025Z E0308 10:40:45.863320       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:40:45.864604241Z E0308 10:40:45.864525       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:45.864881850Z I0308 10:40:45.864848       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:45.865441401Z I0308 10:40:45.865408       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:45.868621072Z E0308 10:40:45.868574       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:45.868854996Z I0308 10:40:45.868820       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:45.877007213Z I0308 10:40:45.876959       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server"
2022-03-08T10:40:47.360529157Z E0308 10:40:47.360482       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:47.361099704Z I0308 10:40:47.361055       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:47.399544337Z E0308 10:40:47.399507       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:47.400015941Z I0308 10:40:47.399983       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:50.879141913Z E0308 10:40:50.879089       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:50.879556966Z I0308 10:40:50.879529       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:50.880436156Z I0308 10:40:50.880392       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:50.885184054Z E0308 10:40:50.885073       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:50.885503221Z I0308 10:40:50.885466       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:50.891413327Z I0308 10:40:50.891345       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
2022-03-08T10:40:50.906012634Z E0308 10:40:50.905962       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:40:50.907706782Z E0308 10:40:50.907664       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:40:50.907802154Z I0308 10:40:50.907785       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:40:50.908405824Z I0308 10:40:50.908368       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"AsExpected","status":"False","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:50.912047361Z E0308 10:40:50.912006       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:50.912098884Z I0308 10:40:50.912063       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:40:50.917994827Z I0308 10:40:50.917880       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server"
2022-03-08T10:40:51.215605434Z I0308 10:40:51.215533       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:40:51.226158345Z I0308 10:40:51.226097       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded changed from False to True ("OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server")
2022-03-08T10:40:55.906373105Z E0308 10:40:55.906304       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:40:57.673244362Z I0308 10:40:57.673190       1 tlsconfig.go:178] loaded client CA [0/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "admin-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:26 +0000 UTC to 2032-03-05 10:10:26 +0000 UTC (now=2022-03-08 10:40:57.673161263 +0000 UTC))
2022-03-08T10:40:57.673328422Z I0308 10:40:57.673313       1 tlsconfig.go:178] loaded client CA [1/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.673301963 +0000 UTC))
2022-03-08T10:40:57.673362126Z I0308 10:40:57.673350       1 tlsconfig.go:178] loaded client CA [2/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-control-plane-signer" [] issuer="<self>" (2022-03-08 10:10:30 +0000 UTC to 2023-03-08 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.673343335 +0000 UTC))
2022-03-08T10:40:57.673394036Z I0308 10:40:57.673381       1 tlsconfig.go:178] loaded client CA [3/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-apiserver-to-kubelet-signer" [] issuer="<self>" (2022-03-08 10:10:31 +0000 UTC to 2023-03-08 10:10:31 +0000 UTC (now=2022-03-08 10:40:57.673374767 +0000 UTC))
2022-03-08T10:40:57.673437287Z I0308 10:40:57.673422       1 tlsconfig.go:178] loaded client CA [4/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kubelet-bootstrap-kubeconfig-signer" [] issuer="<self>" (2022-03-08 10:10:27 +0000 UTC to 2032-03-05 10:10:27 +0000 UTC (now=2022-03-08 10:40:57.67341492 +0000 UTC))
2022-03-08T10:40:57.673471862Z I0308 10:40:57.673459       1 tlsconfig.go:178] loaded client CA [5/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "kube-csr-signer_@1646735926" [] issuer="kubelet-signer" (2022-03-08 10:38:45 +0000 UTC to 2022-03-09 10:10:30 +0000 UTC (now=2022-03-08 10:40:57.673450965 +0000 UTC))
2022-03-08T10:40:57.673506697Z I0308 10:40:57.673493       1 tlsconfig.go:178] loaded client CA [6/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "openshift-kube-apiserver-operator_node-system-admin-signer@1646735931" [] issuer="<self>" (2022-03-08 10:38:50 +0000 UTC to 2023-03-08 10:38:51 +0000 UTC (now=2022-03-08 10:40:57.673485167 +0000 UTC))
2022-03-08T10:40:57.673544331Z I0308 10:40:57.673530       1 tlsconfig.go:178] loaded client CA [7/"client-ca::kube-system::extension-apiserver-authentication::client-ca-file,client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"]: "aggregator-signer" [] issuer="<self>" (2022-03-08 10:10:28 +0000 UTC to 2022-03-09 10:10:28 +0000 UTC (now=2022-03-08 10:40:57.673522211 +0000 UTC))
2022-03-08T10:40:57.673862945Z I0308 10:40:57.673824       1 tlsconfig.go:200] loaded serving cert ["serving-cert::/var/run/secrets/serving-cert/tls.crt::/var/run/secrets/serving-cert/tls.key"]: "metrics.openshift-authentication-operator.svc" [serving] validServingFor=[metrics.openshift-authentication-operator.svc,metrics.openshift-authentication-operator.svc.cluster.local] issuer="openshift-service-serving-signer@1646735940" (2022-03-08 10:39:16 +0000 UTC to 2024-03-07 10:39:17 +0000 UTC (now=2022-03-08 10:40:57.673814397 +0000 UTC))
2022-03-08T10:40:57.730034507Z I0308 10:40:57.729987       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1646736001" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1646736000" (2022-03-08 09:40:00 +0000 UTC to 2023-03-08 09:40:00 +0000 UTC (now=2022-03-08 10:40:57.729963704 +0000 UTC))
2022-03-08T10:41:00.907853660Z E0308 10:41:00.906972       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:01.108745153Z I0308 10:41:01.108552       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:01.109712084Z E0308 10:41:01.109676       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:01.112499761Z E0308 10:41:01.112464       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:01.112955090Z I0308 10:41:01.112616       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:02.320770562Z E0308 10:41:02.320700       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:05.731981671Z E0308 10:41:05.731931       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:05.732056686Z I0308 10:41:05.732027       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:05.921095970Z E0308 10:41:05.921053       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:05.921437576Z I0308 10:41:05.921393       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:05.921937987Z I0308 10:41:05.921912       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:05.925511089Z E0308 10:41:05.925472       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:05.925745045Z I0308 10:41:05.925713       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:05.935451579Z I0308 10:41:05.935395       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"
2022-03-08T10:41:05.935701900Z E0308 10:41:05.935675       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:05.937081176Z E0308 10:41:05.937052       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:05.937410368Z I0308 10:41:05.937389       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:05.938192022Z I0308 10:41:05.938164       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:38:51Z","reason":"NoData","status":"Unknown","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:05.946303753Z E0308 10:41:05.946241       1 workload.go:170] key failed with : clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:05.946860961Z I0308 10:41:05.946722       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' clusteroperator/kube-apiserver is not Available
2022-03-08T10:41:05.952938571Z I0308 10:41:05.952854       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server"
2022-03-08T10:41:10.936251364Z E0308 10:41:10.936171       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:15.936743429Z E0308 10:41:15.936621       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:16.306349769Z E0308 10:41:16.306301       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server could not find the requested resource (get routes.route.openshift.io)
2022-03-08T10:41:17.554279401Z I0308 10:41:17.553945       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentCreated' Created Deployment.apps/apiserver -n openshift-oauth-apiserver because it was missing
2022-03-08T10:41:17.565614249Z I0308 10:41:17.565561       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (no pods found with labels \"apiserver=true,app=openshift-oauth-apiserver,oauth-apiserver-anti-affinity=true,revision=2\")","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:17Z","message":"APIServerDeploymentProgressing: deployment/apiserver.openshift-oauth-apiserver: observed generation is 0, desired generation is 1.","reason":"APIServerDeployment_NewGeneration","status":"True","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:17.566127569Z E0308 10:41:17.566094       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:17.578172204Z I0308 10:41:17.578003       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (no pods found with labels \"apiserver=true,app=openshift-oauth-apiserver,oauth-apiserver-anti-affinity=true,revision=2\")",Progressing changed from Unknown to True ("APIServerDeploymentProgressing: deployment/apiserver.openshift-oauth-apiserver: observed generation is 0, desired generation is 1."),Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node."
2022-03-08T10:41:18.113051912Z I0308 10:41:18.112972       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:18.709425996Z I0308 10:41:18.709323       1 request.go:645] Throttling request took 1.096061809s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver
2022-03-08T10:41:20.536614224Z E0308 10:41:20.536521       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:20.539423432Z I0308 10:41:20.539279       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, 2 containers are waiting in pending apiserver-b7585d69b-f6jdj pod, 2 containers are waiting in pending apiserver-b7585d69b-l9525 pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:20.564114895Z I0308 10:41:20.564038       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (no pods found with labels \"apiserver=true,app=openshift-oauth-apiserver,oauth-apiserver-anti-affinity=true,revision=2\")" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, 2 containers are waiting in pending apiserver-b7585d69b-f6jdj pod, 2 containers are waiting in pending apiserver-b7585d69b-l9525 pod)",Progressing changed from True to False ("")
2022-03-08T10:41:21.326016136Z I0308 10:41:21.325949       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:21.709481384Z I0308 10:41:21.709400       1 request.go:645] Throttling request took 1.138436319s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/secrets/etcd-client
2022-03-08T10:41:22.582344856Z E0308 10:41:22.582303       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:22.582977095Z I0308 10:41:22.582939       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, 2 containers are waiting in pending apiserver-b7585d69b-f6jdj pod, 2 containers are waiting in pending apiserver-b7585d69b-l9525 pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:22.607607803Z I0308 10:41:22.607007       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node." to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node."
2022-03-08T10:41:22.608924907Z E0308 10:41:22.608892       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:22.610273454Z E0308 10:41:22.610100       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:22.610380121Z I0308 10:41:22.610358       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, 2 containers are waiting in pending apiserver-b7585d69b-f6jdj pod, 2 containers are waiting in pending apiserver-b7585d69b-l9525 pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:22.627458834Z I0308 10:41:22.627412       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, 2 containers are waiting in pending apiserver-b7585d69b-f6jdj pod, 2 containers are waiting in pending apiserver-b7585d69b-l9525 pod)" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, 2 containers are waiting in pending apiserver-b7585d69b-f6jdj pod, 2 containers are waiting in pending apiserver-b7585d69b-l9525 pod)"
2022-03-08T10:41:23.709494936Z I0308 10:41:23.709437       1 request.go:645] Throttling request took 1.126946234s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/pods?labelSelector=apiserver%3Dtrue
2022-03-08T10:41:23.716564382Z I0308 10:41:23.715983       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:23.926124954Z E0308 10:41:23.926083       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:23.926185932Z I0308 10:41:23.926096       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, container is waiting in pending apiserver-b7585d69b-f6jdj pod, container is waiting in pending apiserver-b7585d69b-l9525 pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:23.943143075Z I0308 10:41:23.943094       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, 2 containers are waiting in pending apiserver-b7585d69b-f6jdj pod, 2 containers are waiting in pending apiserver-b7585d69b-l9525 pod)" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, container is waiting in pending apiserver-b7585d69b-f6jdj pod, container is waiting in pending apiserver-b7585d69b-l9525 pod)"
2022-03-08T10:41:24.909653241Z I0308 10:41:24.909598       1 request.go:645] Throttling request took 1.193452712s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/pods?labelSelector=apiserver%3Dtrue
2022-03-08T10:41:24.918790003Z I0308 10:41:24.918707       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:25.909928139Z I0308 10:41:25.909631       1 request.go:645] Throttling request took 1.193230352s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/services/api
2022-03-08T10:41:26.113910701Z I0308 10:41:26.112790       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:26.911741928Z I0308 10:41:26.911042       1 request.go:645] Throttling request took 1.198086463s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-authentication/services/oauth-openshift
2022-03-08T10:41:27.335141655Z E0308 10:41:27.335099       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:27.336340147Z I0308 10:41:27.336306       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-7p56k pod, container is not ready in apiserver-b7585d69b-f6jdj pod, container is not ready in apiserver-b7585d69b-l9525 pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:27.361348552Z I0308 10:41:27.361293       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (2 containers are waiting in pending apiserver-b7585d69b-7p56k pod, container is waiting in pending apiserver-b7585d69b-f6jdj pod, container is waiting in pending apiserver-b7585d69b-l9525 pod)" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-7p56k pod, container is not ready in apiserver-b7585d69b-f6jdj pod, container is not ready in apiserver-b7585d69b-l9525 pod)"
2022-03-08T10:41:27.624642478Z E0308 10:41:27.624572       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:27.625550248Z I0308 10:41:27.625432       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-7p56k pod, container is not ready in apiserver-b7585d69b-f6jdj pod, container is not ready in apiserver-b7585d69b-l9525 pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:27.642654728Z I0308 10:41:27.642584       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node." to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node."
2022-03-08T10:41:27.645261116Z E0308 10:41:27.645200       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:27.648956532Z E0308 10:41:27.648907       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:27.650738252Z I0308 10:41:27.650651       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-7p56k pod, container is not ready in apiserver-b7585d69b-f6jdj pod, container is not ready in apiserver-b7585d69b-l9525 pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node.","reason":"APIServerDeployment_NoPod::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:27.664746581Z I0308 10:41:27.663969       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-7p56k pod, container is not ready in apiserver-b7585d69b-f6jdj pod, container is not ready in apiserver-b7585d69b-l9525 pod)" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-7p56k pod, container is not ready in apiserver-b7585d69b-f6jdj pod, container is not ready in apiserver-b7585d69b-l9525 pod)"
2022-03-08T10:41:28.314924454Z I0308 10:41:28.312597       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:28.509412681Z I0308 10:41:28.509328       1 request.go:645] Throttling request took 1.136740853s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/secrets/etcd-client
2022-03-08T10:41:29.509391719Z I0308 10:41:29.509327       1 request.go:645] Throttling request took 1.196623373s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/pods?labelSelector=apiserver%3Dtrue
2022-03-08T10:41:29.522035815Z I0308 10:41:29.521979       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:30.509405936Z I0308 10:41:30.509348       1 request.go:645] Throttling request took 1.196633281s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/serviceaccounts/oauth-apiserver-sa
2022-03-08T10:41:30.733305836Z E0308 10:41:30.733209       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:30.734696374Z I0308 10:41:30.734651       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 1 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-f6jdj pod)","reason":"APIServerDeployment_UnavailablePod::IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:30.750356812Z I0308 10:41:30.750292       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 3 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-7p56k pod, container is not ready in apiserver-b7585d69b-f6jdj pod, container is not ready in apiserver-b7585d69b-l9525 pod)" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 1 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-f6jdj pod)",Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nAPIServerDeploymentAvailable: no apiserver.openshift-oauth-apiserver pods available on any node." to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"
2022-03-08T10:41:31.109207548Z E0308 10:41:31.108398       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:31.720700553Z I0308 10:41:31.720648       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:31.909450126Z I0308 10:41:31.909380       1 request.go:645] Throttling request took 1.148343449s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/secrets/etcd-client
2022-03-08T10:41:32.321011333Z E0308 10:41:32.320929       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:32.648387785Z E0308 10:41:32.648339       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:32.919796970Z I0308 10:41:32.919728       1 sync_take_over_apiservices_oauth_apiserver.go:68] the encryption deployer hasn't yet converged, retrying in 5 minutes
2022-03-08T10:41:33.110120492Z I0308 10:41:33.109897       1 request.go:645] Throttling request took 1.187649444s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/configmaps/etcd-serving-ca
2022-03-08T10:41:34.113662831Z I0308 10:41:34.110706       1 request.go:645] Throttling request took 1.589627646s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-authentication/services/oauth-openshift
2022-03-08T10:41:34.549750774Z I0308 10:41:34.548784       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorVersionChanged' clusteroperator/authentication version "oauth-apiserver" changed from "" to "4.6.52"
2022-03-08T10:41:34.549750774Z I0308 10:41:34.549004       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"versions":[{"name":"oauth-apiserver","version":"4.6.52"}]}}
2022-03-08T10:41:34.572957110Z I0308 10:41:34.572666       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: status.versions changed from [] to [{"oauth-apiserver" "4.6.52"}]
2022-03-08T10:41:34.589302921Z E0308 10:41:34.588934       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:34.600892782Z I0308 10:41:34.600801       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:34.626768887Z I0308 10:41:34.626701       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 1 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-f6jdj pod)" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server"
2022-03-08T10:41:34.629855473Z I0308 10:41:34.628409       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:34.666490974Z I0308 10:41:34.666276       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Degraded message changed from "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server\nAPIServerDeploymentDegraded: 1 of 3 requested instances are unavailable for apiserver.openshift-oauth-apiserver (container is not ready in apiserver-b7585d69b-f6jdj pod)" to "OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server"
2022-03-08T10:41:35.709336783Z I0308 10:41:35.709272       1 request.go:645] Throttling request took 1.041845286s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/secrets/etcd-client
2022-03-08T10:41:36.555685737Z E0308 10:41:36.555622       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:36.645983805Z I0308 10:41:36.645928       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'APIServicesToManageChanged' The new API Services list this operator will manage is [v1.oauth.openshift.io v1.user.openshift.io]
2022-03-08T10:41:36.685090346Z I0308 10:41:36.684875       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: '<unknown>Updated' Updated <unknown>/v1.oauth.openshift.io because it changed
2022-03-08T10:41:36.709601394Z I0308 10:41:36.709557       1 request.go:645] Throttling request took 1.111183936s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-authentication/serviceaccounts/oauth-openshift
2022-03-08T10:41:36.710155740Z I0308 10:41:36.710108       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: '<unknown>Updated' Updated <unknown>/v1.user.openshift.io because it changed
2022-03-08T10:41:37.650346276Z E0308 10:41:37.650294       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:37.911340609Z I0308 10:41:37.910978       1 request.go:645] Throttling request took 1.189266114s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-authentication/services/oauth-openshift
2022-03-08T10:41:39.109436733Z I0308 10:41:39.109330       1 request.go:645] Throttling request took 1.391962655s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/secrets/encryption-config-2
2022-03-08T10:41:40.109431547Z I0308 10:41:40.109341       1 request.go:645] Throttling request took 1.307377341s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-oauth-apiserver/services/api
2022-03-08T10:41:41.109424172Z I0308 10:41:41.109355       1 request.go:645] Throttling request took 1.189214627s, request: DELETE:https://172.30.0.1:443/api/v1/namespaces/openshift-config-managed/configmaps/oauth-openshift
2022-03-08T10:41:41.720356820Z I0308 10:41:41.714027       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "oauth.openshift.io.v1" failed with HTTP status code 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.720356820Z I0308 10:41:41.714541       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "user.openshift.io.v1" failed with HTTP status code 404 (the server could not find the requested resource)
2022-03-08T10:41:41.728422451Z E0308 10:41:41.728345       1 base_controller.go:250] "APIServiceController_openshift-apiserver" controller failed to sync "key", err: "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.728422451Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.730383136Z E0308 10:41:41.730309       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:41.731375988Z E0308 10:41:41.730789       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.731375988Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.732934896Z I0308 10:41:41.732151       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.732934896Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.732934896Z I0308 10:41:41.732704       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nAPIServicesAvailable: \"oauth.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\nAPIServicesAvailable: \"user.openshift.io.v1\" is not ready: 404 (the server could not find the requested resource)\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"APIServices_Error::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:41.738047262Z E0308 10:41:41.737988       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.738047262Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.738166410Z I0308 10:41:41.738043       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.738166410Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.748377338Z I0308 10:41:41.748308       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nAPIServicesAvailable: \"oauth.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\nAPIServicesAvailable: \"user.openshift.io.v1\" is not ready: 404 (the server could not find the requested resource)\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"
2022-03-08T10:41:41.756870621Z E0308 10:41:41.751870       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.756870621Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.756870621Z I0308 10:41:41.754667       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.756870621Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.772774028Z E0308 10:41:41.772654       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.772774028Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.772774028Z I0308 10:41:41.772724       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.772774028Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.812942263Z E0308 10:41:41.812886       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.812942263Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.813531994Z I0308 10:41:41.813042       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.813531994Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.893748943Z E0308 10:41:41.893698       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.893748943Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:41.893973963Z I0308 10:41:41.893834       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:41.893973963Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:42.055266759Z E0308 10:41:42.055122       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:42.055266759Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:42.055444884Z I0308 10:41:42.055384       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:42.055444884Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:42.375574797Z E0308 10:41:42.375373       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:42.375574797Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:42.375831458Z I0308 10:41:42.375738       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:42.375831458Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:42.662832794Z E0308 10:41:42.660711       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:43.016309952Z E0308 10:41:43.015604       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:43.016309952Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:43.016309952Z I0308 10:41:43.016073       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:43.016309952Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:44.296912202Z I0308 10:41:44.296014       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:44.296912202Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:44.296912202Z E0308 10:41:44.296058       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:44.296912202Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:46.748549567Z I0308 10:41:46.743568       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "oauth.openshift.io.v1" failed with HTTP status code 503 (the server is currently unable to handle the request)
2022-03-08T10:41:46.748549567Z E0308 10:41:46.745975       1 base_controller.go:250] "APIServiceController_openshift-apiserver" controller failed to sync "key", err: "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:46.748549567Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:46.748549567Z I0308 10:41:46.746656       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "user.openshift.io.v1" failed with HTTP status code 404 (the server could not find the requested resource)
2022-03-08T10:41:46.856385316Z E0308 10:41:46.856331       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:46.856385316Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:46.856574611Z I0308 10:41:46.856480       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:46.856574611Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:47.661320165Z E0308 10:41:47.661255       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:41:51.781968114Z I0308 10:41:51.779651       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "oauth.openshift.io.v1" failed with HTTP status code 503 (the server is currently unable to handle the request)
2022-03-08T10:41:51.976820945Z I0308 10:41:51.976768       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:51.976820945Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:51.976894665Z E0308 10:41:51.976810       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:51.976894665Z "user.openshift.io.v1" is not ready: 404 (the server could not find the requested resource)
2022-03-08T10:41:56.782233796Z I0308 10:41:56.782185       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "user.openshift.io.v1" failed with HTTP status code 503 (the server is currently unable to handle the request)
2022-03-08T10:41:56.800233517Z E0308 10:41:56.799696       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:41:56.828139085Z E0308 10:41:56.800785       1 base_controller.go:250] "APIServiceController_openshift-apiserver" controller failed to sync "key", err: "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:56.828139085Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:56.828139085Z E0308 10:41:56.801190       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:56.828139085Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:56.828139085Z I0308 10:41:56.805627       1 status_controller.go:172] clusteroperator/authentication diff {"status":{"conditions":[{"lastTransitionTime":"2022-03-08T10:40:51Z","message":"OAuthServiceCheckEndpointAccessibleControllerDegraded: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)\nOAuthServiceEndpointsCheckEndpointAccessibleControllerDegraded: oauth service endpoints are not ready\nIngressStateEndpointsDegraded: No subsets found for the endpoints of oauth-server","reason":"IngressStateEndpoints_MissingSubsets::OAuthServiceCheckEndpointAccessibleController_SyncError::OAuthServiceEndpointsCheckEndpointAccessibleController_SyncError","status":"True","type":"Degraded"},{"lastTransitionTime":"2022-03-08T10:41:20Z","reason":"AsExpected","status":"False","type":"Progressing"},{"lastTransitionTime":"2022-03-08T10:38:51Z","message":"OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nAPIServicesAvailable: \"oauth.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\nAPIServicesAvailable: \"user.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)","reason":"APIServices_Error::OAuthServiceCheckEndpointAccessibleController_EndpointUnavailable::OAuthServiceEndpointsCheckEndpointAccessibleController_EndpointUnavailable::ReadyIngressNodes_NoReadyIngressNodes","status":"False","type":"Available"},{"lastTransitionTime":"2022-03-08T10:38:53Z","reason":"AsExpected","status":"True","type":"Upgradeable"}]}}
2022-03-08T10:41:56.828139085Z I0308 10:41:56.821977       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:56.828139085Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:41:56.888293900Z I0308 10:41:56.888226       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nAPIServicesAvailable: \"oauth.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\nAPIServicesAvailable: \"user.openshift.io.v1\" is not ready: 404 (the server could not find the requested resource)\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)" to "OAuthServiceEndpointsCheckEndpointAccessibleControllerAvailable: Failed to get oauth-openshift enpoints\nAPIServicesAvailable: \"oauth.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\nAPIServicesAvailable: \"user.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\nReadyIngressNodesAvailable: Authentication requires functional ingress which requires at least one schedulable and ready node. Got 0 worker nodes, 3 master nodes, 0 custom target nodes (none are schedulable or ready for ingress pods).\nOAuthServiceCheckEndpointAccessibleControllerAvailable: Get \"https://172.30.82.175:443/healthz\": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)"
2022-03-08T10:41:58.510146246Z I0308 10:41:58.510077       1 request.go:645] Throttling request took 1.091657672s, request: GET:https://172.30.0.1:443/api/v1/namespaces/openshift-authentication/configmaps/v4-0-config-system-service-ca
2022-03-08T10:42:01.108863130Z E0308 10:42:01.108457       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:42:01.109124392Z E0308 10:42:01.109069       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:01.109124392Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:01.109144875Z I0308 10:42:01.109107       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:01.109144875Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:01.804724670Z E0308 10:42:01.804087       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:42:01.864632383Z I0308 10:42:01.864084       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "oauth.openshift.io.v1" failed with HTTP status code 503 (the server is currently unable to handle the request)
2022-03-08T10:42:02.217918613Z E0308 10:42:02.217098       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:02.217918613Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:02.217918613Z I0308 10:42:02.217173       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:02.217918613Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:02.323876402Z E0308 10:42:02.321696       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:42:06.804873585Z E0308 10:42:06.804676       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:42:06.866911289Z E0308 10:42:06.866270       1 base_controller.go:250] "APIServiceController_openshift-apiserver" controller failed to sync "key", err: "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:06.866911289Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:06.866911289Z I0308 10:42:06.866546       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'OpenShiftAPICheckFailed' "user.openshift.io.v1" failed with HTTP status code 503 (the server is currently unable to handle the request)
2022-03-08T10:42:11.805937079Z E0308 10:42:11.805120       1 base_controller.go:250] "OAuthServiceCheckEndpointAccessibleController" controller failed to sync "key", err: Get "https://172.30.82.175:443/healthz": dial tcp 172.30.82.175:443: i/o timeout (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:42:31.108408652Z E0308 10:42:31.108352       1 base_controller.go:250] "MangeOAuthAPIController" controller failed to sync "key", err: OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:31.108408652Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:31.108817456Z I0308 10:42:31.108786       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'PrereqNotReady' OAuth API servers are not ready. APIService check reporeted "oauth.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:31.108817456Z "user.openshift.io.v1" is not ready: 503 (the server is currently unable to handle the request)
2022-03-08T10:42:31.109158266Z E0308 10:42:31.109121       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:42:32.125463699Z I0308 10:42:32.125425       1 trace.go:205] Trace[323725821]: "Reflector ListAndWatch" name:github.com/openshift/client-go/route/informers/externalversions/factory.go:101 (08-Mar-2022 10:42:02.118) (total time: 30006ms):
2022-03-08T10:42:32.125463699Z Trace[323725821]: [30.006430533s] [30.006430533s] END
2022-03-08T10:42:32.125546412Z E0308 10:42:32.125532       1 reflector.go:127] github.com/openshift/client-go/route/informers/externalversions/factory.go:101: Failed to watch *v1.Route: failed to list *v1.Route: the server is currently unable to handle the request (get routes.route.openshift.io)
2022-03-08T10:42:32.321906253Z E0308 10:42:32.321826       1 base_controller.go:250] "OAuthServiceEndpointsCheckEndpointAccessibleController" controller failed to sync "key", err: oauth service endpoints are not ready
2022-03-08T10:42:32.392610049Z E0308 10:42:32.392531       1 workload.go:170] key failed with : context deadline exceeded
2022-03-08T10:42:35.117251283Z E0308 10:42:35.117003       1 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"authentication-operator.16da61c600bd828d", GenerateName:"", Namespace:"openshift-authentication-operator", SelfLink:"", UID:"", ResourceVersion:"14848", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}, Reason:"PrereqNotReady", Message:"OAuth API servers are not ready. APIService check reporeted \"oauth.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)\n\"user.openshift.io.v1\" is not ready: 503 (the server is currently unable to handle the request)", Source:v1.EventSource{Component:"oauth-apiserver-manage-oauth-api-controller-mangeoauthapicontroller", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63782332916, loc:(*time.Location)(0x36402c0)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc081ea9e467fe238, ext:120608268577, loc:(*time.Location)(0x36402c0)}}, Count:2, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'Timeout: request did not complete within requested timeout 34s' (will not retry!)
2022-03-08T10:42:35.867274995Z E0308 10:42:35.867125       1 event.go:264] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"authentication-operator.16da61c27d6692d0", GenerateName:"", Namespace:"openshift-authentication-operator", SelfLink:"", UID:"", ResourceVersion:"14514", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"ddbbb242-e96a-4c69-91c7-b167fc7d4340", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}, Reason:"OpenShiftAPICheckFailed", Message:"\"oauth.openshift.io.v1\" failed with HTTP status code 503 (the server is currently unable to handle the request)", Source:v1.EventSource{Component:"oauth-apiserver-apiservice-openshift-apiserver-controller-apiservicecontroller_openshift-apiserver", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63782332901, loc:(*time.Location)(0x36402c0)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xc081ea9e737cef1e, ext:121363049861, loc:(*time.Location)(0x36402c0)}}, Count:4, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'Timeout: request did not complete within requested timeout 34s' (will not retry!)
2022-03-08T10:42:36.360097697Z E0308 10:42:36.360026       1 leaderelection.go:321] error retrieving resource lock openshift-authentication-operator/cluster-authentication-operator-lock: Get "https://172.30.0.1:443/api/v1/namespaces/openshift-authentication-operator/configmaps/cluster-authentication-operator-lock?timeout=35s": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2022-03-08T10:42:36.360206681Z I0308 10:42:36.360189       1 leaderelection.go:278] failed to renew lease openshift-authentication-operator/cluster-authentication-operator-lock: timed out waiting for the condition
2022-03-08T10:42:36.360294240Z E0308 10:42:36.360281       1 leaderelection.go:297] Failed to release lock: resource name may not be empty
2022-03-08T10:42:36.360396740Z W0308 10:42:36.360380       1 leaderelection.go:75] leader election lost
