Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-24012

Tuned Node Profile takes up to 30 minutes post OpenShift Container Platform 4 - Node creation before it's being created

    XMLWordPrintable

Details

    • Important
    • No
    • False
    • Hide

      None

      Show
      None
    • Release Note Not Required
    • In Progress

    Description

      It was found that OpenShift Container Platform 4 - Node(s) are missing certain settings applied via tuned and when starting to investigate the problem it was found that it takes up to 30 minutes or more for the tuned profiles of this newly added OpenShift Container Platform 4 - Node for being created.

      When increasing the log level of cluster-node-tuning-operator pod we can see the following events being recorded.

      I1128 13:05:12.465193       1 controller.go:1121] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (add)
      I1128 13:05:12.465235       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:12.465247       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:12.465255       1 profilecalculator.go:131] Node's new-worker-X.example.com providerID=aws:///eu-central-1c/i-0874090641dd61eef
      I1128 13:05:12.465268       1 controller.go:300] sync(): Node new-worker-X.example.com label(s) changed
      I1128 13:05:12.465288       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:12.486200       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:12.486233       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:12.486242       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:12.486256       1 controller.go:300] sync(): Node new-worker-X.example.com label(s) changed
      I1128 13:05:12.486273       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:12.612063       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:12.612114       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:12.612127       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:12.612149       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:15.232435       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:15.232477       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:15.232541       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:15.232565       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:22.805108       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:22.805142       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:22.805151       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:22.805170       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:30.803481       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:30.803511       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:30.803519       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:30.803533       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:35.815894       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:35.815933       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:35.815942       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:35.815958       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:35.832338       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:35.832386       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:35.832395       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:35.832419       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:35.851291       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:35.851337       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:35.851349       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:35.851369       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:40.855159       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:40.855192       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:40.855201       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:40.855221       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:48.004741       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:48.004783       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:48.004815       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:48.004835       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:48.011986       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:48.012035       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:48.012047       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:48.012067       1 controller.go:300] sync(): Node new-worker-X.example.com label(s) changed
      I1128 13:05:48.012090       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:53.475798       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:53.475842       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:53.475855       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:53.475876       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:56.097269       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:56.097299       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:56.097309       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:56.097329       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:05:58.497782       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:05:58.497838       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:05:58.497847       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:05:58.497864       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:06:06.117201       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:06:06.117235       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:06:06.117254       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:06:06.117271       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:06:08.008992       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:06:08.009031       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:06:08.009041       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:06:08.009059       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:06:09.685949       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:06:09.685988       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:06:09.685997       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:06:09.686015       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:06:11.163882       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:06:11.163929       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:06:11.163941       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:06:11.163965       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:06:19.730972       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:06:19.731005       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:06:19.731013       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:06:19.731028       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:06:23.713627       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:06:23.713665       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:06:23.713675       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:06:23.713693       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:07:52.133190       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:07:52.133227       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:07:52.133235       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:07:52.133268       1 controller.go:300] sync(): Node new-worker-X.example.com label(s) changed
      I1128 13:07:52.133285       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:07:55.779247       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:07:55.779278       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:07:55.779286       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:07:55.779324       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:07:55.799941       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:07:55.799975       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:07:55.799983       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:07:55.800021       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:07:56.062048       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:07:56.062081       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:07:56.062089       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:07:56.062126       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:09:58.224261       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:09:58.224294       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:09:58.224303       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:09:58.224333       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:10:08.146467       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:10:08.146504       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:10:08.146513       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:10:08.146549       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:10:29.293368       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:10:29.293402       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:10:29.293410       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:10:29.293440       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:11:38.765691       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:11:38.781424       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:11:38.781432       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:11:38.781471       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:15:35.022263       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:15:35.022303       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:15:35.022312       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:15:35.022349       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:20:41.252897       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:20:41.252942       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:20:41.252951       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:20:41.252988       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:21:38.768157       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:21:38.781098       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:21:38.781103       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:21:38.781133       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:25:47.684402       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:25:47.684445       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:25:47.684457       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:25:47.684494       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:25:53.336668       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:25:53.336700       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:25:53.336709       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:25:53.336738       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:25:57.754420       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:25:57.754453       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:25:57.754462       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:25:57.754491       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:26:03.987123       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:26:03.987188       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:26:03.987203       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:26:03.987258       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:26:38.231524       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:26:38.231558       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:26:38.231566       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:26:38.231602       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:27:08.845310       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:27:08.845349       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:27:08.845358       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:27:08.845398       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:27:49.797881       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:27:49.797919       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:27:49.797928       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:27:49.797958       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:27:49.856526       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:27:49.856566       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:27:49.856575       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:27:49.856612       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:27:49.904286       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:27:49.904341       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:27:49.904350       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:27:49.904400       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:30:02.351363       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:30:02.351398       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:30:02.351407       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:30:02.351440       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:30:03.719303       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:30:03.719338       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:30:03.719347       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:30:03.719380       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:30:33.316267       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:30:33.316297       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:30:33.316307       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:30:33.316336       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:30:33.330998       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:30:33.331030       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:30:33.331038       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:30:33.331066       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:31:31.688121       1 controller.go:221] sync(): Kind profile: openshift-cluster-node-tuning-operator/new-worker-X.example.com
      I1128 13:31:31.688136       1 controller.go:374] sync(): Profile new-worker-X.example.com
      I1128 13:31:31.688300       1 profilecalculator.go:164] calculateProfile(new-worker-X.example.com)
      I1128 13:31:31.688337       1 controller.go:677] syncProfile(): Profile new-worker-X.example.com not found, creating one [openshift-node]
      I1128 13:31:31.688396       1 request.go:1073] Request Body: {"kind":"Profile","apiVersion":"tuned.openshift.io/v1","metadata":{"name":"new-worker-X.example.com","namespace":"openshift-cluster-node-tuning-operator","creationTimestamp":null,"ownerReferences":[{"apiVersion":"tuned.openshift.io/v1","kind":"Tuned","name":"default","uid":"324f82ad-4475-4b49-ac29-57cb454314e7","controller":true,"blockOwnerDeletion":true}]},"spec":{"config":{"tunedProfile":"openshift-node","debug":false,"tunedConfig":{"reapply_sysctl":null}}},"status":{"bootcmdline":"","tunedProfile":"","conditions":[{"type":"Applied","status":"Unknown","lastTransitionTime":"2023-11-28T13:31:31Z"},{"type":"Degraded","status":"Unknown","lastTransitionTime":"2023-11-28T13:31:31Z"}]}}
      I1128 13:31:31.698807       1 request.go:1073] Response Body: {"apiVersion":"tuned.openshift.io/v1","kind":"Profile","metadata":{"creationTimestamp":"2023-11-28T13:31:31Z","generation":1,"managedFields":[{"apiVersion":"tuned.openshift.io/v1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:ownerReferences":{".":{},"k:{\"uid\":\"324f82ad-4475-4b49-ac29-57cb454314e7\"}":{}}},"f:spec":{".":{},"f:config":{".":{},"f:debug":{},"f:tunedConfig":{},"f:tunedProfile":{}}}},"manager":"cluster-node-tuning-operator","operation":"Update","time":"2023-11-28T13:31:31Z"}],"name":"new-worker-X.example.com","namespace":"openshift-cluster-node-tuning-operator","ownerReferences":[{"apiVersion":"tuned.openshift.io/v1","blockOwnerDeletion":true,"controller":true,"kind":"Tuned","name":"default","uid":"324f82ad-4475-4b49-ac29-57cb454314e7"}],"resourceVersion":"9673729653","uid":"8607cf52-9a00-49d2-baff-8a97c73b809a"},"spec":{"config":{"debug":false,"tunedConfig":{},"tunedProfile":"openshift-node"}}}
      I1128 13:31:31.698915       1 controller.go:687] created profile new-worker-X.example.com [openshift-node]
      I1128 13:31:31.698925       1 controller.go:209] event from workqueue (profile/openshift-cluster-node-tuning-operator/new-worker-X.example.com) successfully processed
      I1128 13:31:31.702309       1 controller.go:1121] add event to workqueue due to *v1.Profile, Namespace=openshift-cluster-node-tuning-operator, Name=new-worker-X.example.com (add)
      I1128 13:31:31.702335       1 controller.go:221] sync(): Kind profile: openshift-cluster-node-tuning-operator/new-worker-X.example.com
      I1128 13:31:31.702358       1 controller.go:374] sync(): Profile new-worker-X.example.com
      I1128 13:31:31.702494       1 profilecalculator.go:164] calculateProfile(new-worker-X.example.com)
      I1128 13:31:31.713444       1 controller.go:752] syncProfile(): updating Profile new-worker-X.example.com [openshift-node]
      I1128 13:31:31.713543       1 request.go:1073] Request Body: {"kind":"Profile","apiVersion":"tuned.openshift.io/v1","metadata":{"name":"new-worker-X.example.com","namespace":"openshift-cluster-node-tuning-operator","uid":"8607cf52-9a00-49d2-baff-8a97c73b809a","resourceVersion":"9673729653","generation":1,"creationTimestamp":"2023-11-28T13:31:31Z","ownerReferences":[{"apiVersion":"tuned.openshift.io/v1","kind":"Tuned","name":"default","uid":"324f82ad-4475-4b49-ac29-57cb454314e7","controller":true,"blockOwnerDeletion":true}],"managedFields":[{"manager":"cluster-node-tuning-operator","operation":"Update","apiVersion":"tuned.openshift.io/v1","time":"2023-11-28T13:31:31Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:ownerReferences":{".":{},"k:{\"uid\":\"324f82ad-4475-4b49-ac29-57cb454314e7\"}":{}}},"f:spec":{".":{},"f:config":{".":{},"f:debug":{},"f:tunedConfig":{},"f:tunedProfile":{}}}}}]},"spec":{"config":{"tunedProfile":"openshift-node","debug":false,"tunedConfig":{"reapply_sysctl":null},"providerName":"aws"}},"status":{"bootcmdline":"","tunedProfile":"","conditions":[{"type":"Applied","status":"Unknown","lastTransitionTime":"2023-11-28T13:31:31Z"},{"type":"Degraded","status":"Unknown","lastTransitionTime":"2023-11-28T13:31:31Z"}]}}
      I1128 13:31:31.713611       1 round_trippers.go:466] curl -v -XPUT  -H "User-Agent: cluster-node-tuning-operator/v0.0.0 (linux/amd64) kubernetes/$Format" -H "Accept: application/json, */*" -H "Authorization: Bearer <masked>" -H "Content-Type: application/json" 'https://172.16.0.1:443/apis/tuned.openshift.io/v1/namespaces/openshift-cluster-node-tuning-operator/profiles/new-worker-X.example.com'
      I1128 13:31:31.720708       1 round_trippers.go:553] PUT https://172.16.0.1:443/apis/tuned.openshift.io/v1/namespaces/openshift-cluster-node-tuning-operator/profiles/new-worker-X.example.com 200 OK in 7 milliseconds
      I1128 13:31:31.720855       1 request.go:1073] Response Body: {"apiVersion":"tuned.openshift.io/v1","kind":"Profile","metadata":{"creationTimestamp":"2023-11-28T13:31:31Z","generation":2,"managedFields":[{"apiVersion":"tuned.openshift.io/v1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:ownerReferences":{".":{},"k:{\"uid\":\"324f82ad-4475-4b49-ac29-57cb454314e7\"}":{}}},"f:spec":{".":{},"f:config":{".":{},"f:debug":{},"f:providerName":{},"f:tunedConfig":{},"f:tunedProfile":{}}}},"manager":"cluster-node-tuning-operator","operation":"Update","time":"2023-11-28T13:31:31Z"}],"name":"new-worker-X.example.com","namespace":"openshift-cluster-node-tuning-operator","ownerReferences":[{"apiVersion":"tuned.openshift.io/v1","blockOwnerDeletion":true,"controller":true,"kind":"Tuned","name":"default","uid":"324f82ad-4475-4b49-ac29-57cb454314e7"}],"resourceVersion":"9673729659","uid":"8607cf52-9a00-49d2-baff-8a97c73b809a"},"spec":{"config":{"debug":false,"providerName":"aws","tunedConfig":{},"tunedProfile":"openshift-node"}}}
      I1128 13:31:31.720946       1 controller.go:757] updated profile new-worker-X.example.com [openshift-node]
      I1128 13:31:31.720955       1 controller.go:209] event from workqueue (profile/openshift-cluster-node-tuning-operator/new-worker-X.example.com) successfully processed
      I1128 13:31:31.721160       1 controller.go:1136] add event to workqueue due to *v1.Profile, Namespace=openshift-cluster-node-tuning-operator, Name=new-worker-X.example.com (update)
      I1128 13:31:31.724833       1 controller.go:221] sync(): Kind profile: openshift-cluster-node-tuning-operator/new-worker-X.example.com
      I1128 13:31:31.724847       1 controller.go:374] sync(): Profile new-worker-X.example.com
      I1128 13:31:31.724971       1 profilecalculator.go:164] calculateProfile(new-worker-X.example.com)
      I1128 13:31:31.726987       1 controller.go:742] syncProfile(): no need to update Profile new-worker-X.example.com
      I1128 13:31:31.726993       1 controller.go:209] event from workqueue (profile/openshift-cluster-node-tuning-operator/new-worker-X.example.com) successfully processed
      I1128 13:31:32.273200       1 controller.go:1136] add event to workqueue due to *v1.Profile, Namespace=openshift-cluster-node-tuning-operator, Name=new-worker-X.example.com (update)
      I1128 13:31:32.273234       1 controller.go:221] sync(): Kind profile: openshift-cluster-node-tuning-operator/new-worker-X.example.com
      I1128 13:31:32.273246       1 controller.go:374] sync(): Profile new-worker-X.example.com
      I1128 13:31:32.273410       1 profilecalculator.go:164] calculateProfile(new-worker-X.example.com)
      I1128 13:31:32.284388       1 controller.go:742] syncProfile(): no need to update Profile new-worker-X.example.com
      I1128 13:31:32.284400       1 controller.go:209] event from workqueue (profile/openshift-cluster-node-tuning-operator/new-worker-X.example.com) successfully processed
      I1128 13:31:38.766803       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:31:38.769582       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:31:38.769588       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:31:38.769617       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      I1128 13:35:39.839137       1 controller.go:1136] add event to workqueue due to *v1.Node, Name=new-worker-X.example.com (update)
      I1128 13:35:39.839174       1 controller.go:221] sync(): Kind node: /new-worker-X.example.com
      I1128 13:35:39.839182       1 controller.go:282] sync(): Node new-worker-X.example.com
      I1128 13:35:39.839215       1 controller.go:209] event from workqueue (node//new-worker-X.example.com) successfully processed
      

      So at 13:05:12 the OpenShift Container Platform 4 - Node called `new-worker-X.example.com` would indeed become available but it still took until 13:31:31 until the tuned profile was created and therefore required settings on the OpenShift Container Platform 4 - Node are being applied.

      Attachments

        Issue Links

          Activity

            People

              jmencak Jiri Mencak
              rhn-support-sreber Simon Reber
              Liquan Cui Liquan Cui
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: