Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-23039

AWS/CI: Local Zones provisioning is failing in us-west-2-las-1b

    XMLWordPrintable

Details

    Description

      Description of problem:

      Installations on AWS with Local Zone subnets or zones are failing on CI when the zone selected is us-west-2-las-1b (new zone in Las Vegas/US).
      
      The CI selects randomly one zone[1], when that specific zone is selected and added to the install-config.yaml[2] the installer tries to discover the instance type offered in the zone[3], but the EC2 API is returning empty values[4] - which means no EC2 available in that zone, consequently the installer will fill the InstanceType in the MachineSet with empty value, leading to fail when MAPI tries to provision the instance[5].
      
      Additionally, the AWS Local Zones feature page[6] does not show the zone us-west-2-las-1b listed.
      
      [1] https://github.com/openshift/release/blob/f36aa8f608a786674c27db163c30f7d88a0f64e9/ci-operator/step-registry/ipi/aws/pre/local-zones/opt-in/ipi-aws-pre-local-zones-opt-in-commands.sh#L15-L16
      [2] https://github.com/openshift/release/blob/f36aa8f608a786674c27db163c30f7d88a0f64e9/ci-operator/step-registry/ipi/conf/aws/ipi-conf-aws-commands.sh#L324-L336
      [3] https://github.com/openshift/installer/blob/2596e5d46c7b8f2c4907c7c6d80beaf768ff9953/pkg/asset/machines/worker.go#L206-L220
      [4] No Instances in us-west-2-las-1b
      ```
      $ aws ec2 describe-instance-type-offerings --region us-west-2 --location-type availability-zone --filters Name=location,Values=us-west-2-las-1b
      {
          "InstanceTypeOfferings": []
      }
      ```
      [5] Machine status
      ```
      status.providerStatus:
          conditions:
          - lastTransitionTime: "2023-11-07T19:32:52Z"
            message: 'error launching instance: Invalid value '''' for InstanceType.'
            reason: MachineCreationFailed
            status: "False"
            type: MachineCreation
      ```
      [6] https://aws.amazon.com/about-aws/global-infrastructure/localzones/features/?nc=sn&loc=2
      
      

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Always

      Steps to Reproduce:

      1. create the install-config, setting edge compute pool with zone us-west-2-las-1b
      2. generate the manifests or install the cluster
      3.
      

      Actual results:

      Empty Instance Type, MAPi fails to provision the cluster, CI job does not progress as it requires to have the number of nodes equals the machines

      Expected results:

      - installer never progress when there are no EC2 available in the zone
      - CI step check if there is available EC2 offerings in the zone, otherwise, select another zone (preventing to lose the job run)

      Additional info:

      [1] Example of job runs:
      - https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/45158/rehearse-45158-pull-ci-openshift-installer-release-4.14-e2e-aws-ovn-shared-vpc-localzones/1721937572780838912
      - https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_installer/7676/pull-ci-openshift-installer-master-e2e-aws-ovn-localzones/1721327505312321536
      - https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.15-e2e-aws-ovn-shared-vpc-localzones/1720954245286465536

      Attachments

        Issue Links

          Activity

            People

              rhn-support-mrbraga Marco Braga
              rhn-support-mrbraga Marco Braga
              Yunfei Jiang Yunfei Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: