BoxGrinder Build
  1. BoxGrinder Build
  2. BGBUILD-172

Boxgrinder EC2 meta appliance doesn't set up glibc correctly

    Details

    • Type: Bug Bug
    • Status: Closed Closed (View Workflow)
    • Priority: Major Major
    • Resolution: Won't Fix Won't Fix
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: Meta appliance
    • Labels:
      None
    • Environment:
      Amazon EC2 32-bit environments, Fedora
    • Similar Issues:
      Show 10 results 

      Description

      There's a severe bug in the chroot environment set up by boxgrinder that affects 32-bit systems running on Amazon EC2, and also under other Xen environments.

      Xen requires a special glibc runtime, named nosegneg in order for applications to access thread-local storage efficiently. This is configured in a file named /etc/ld.so.conf.d/libc6-xen.conf with the following contents:

      hwcap 1 nosegneg
      

      These ld.so config files must also be present in the chroot environment, or the nosegneg version of glibc will not be used. And if those are not used, some applications will be very slow, while others will either crash or go into an infinite loop.

      Here is a bug I reported against GHC that turned out to not be a GHC bug - it was really due to GHC not being run with the nosegneg libraries inside boxgrinder's chroot environment: http://hackage.haskell.org/trac/ghc/ticket/4993

      That problem was causing my attempt to build a VM image with boxgrinder on 32-bit EC2 to hang forever, and it took me ages to figure out why.

      I think there are two parts to fixing this:

      • Copy /etc/ld.so.conf and the files in /etc/ld.so.conf.d into the chroot environment. (Actually, the real fix would be to figure out what creates /etc/ld.so.conf.d/libc6-xen.conf and do the exact same thing, but I've not been able to figure out what does that.)
      • Run ldconfig in the chroot environment.

      Thanks for looking into this. It's a big deal, as it's very hard to understand why an installer gets stuck, and very difficult to work around.

        Issue Links

          Activity

          Hide
          Bryan O'Sullivan
          added a comment -

          By the way, since I believe that the build/appliances/i686/fedora/14/xxx/fedora-plugin/tmp/imgcreate-xxx/install_root directory contains the intended final image, this is actually somewhat more serious than I reported: if the build of an image succeeds without that nosegneg stuff in the built virtual machine image and the image is then run under Xen, we could see crashes or hangs in those VMs at run time, long after the install.

          Show
          Bryan O'Sullivan
          added a comment - By the way, since I believe that the build/appliances/i686/fedora/14/ xxx /fedora-plugin/tmp/imgcreate- xxx /install_root directory contains the intended final image, this is actually somewhat more serious than I reported: if the build of an image succeeds without that nosegneg stuff in the built virtual machine image and the image is then run under Xen, we could see crashes or hangs in those VMs at run time, long after the install.
          Hide
          Bryan O'Sullivan
          added a comment -

          This could easily get confusing.

          If the host where a VM is being built is running under Xen or on EC2, then the chroot in which the target VM image is being built must have the nosegneg magic at least temporarily, so that the programs building the image will run correctly. But if the target is not being run under Xen or on EC2, then once it's built, when it is run, it should not have the nosegneg stuff.

          Show
          Bryan O'Sullivan
          added a comment - This could easily get confusing. If the host where a VM is being built is running under Xen or on EC2, then the chroot in which the target VM image is being built must have the nosegneg magic at least temporarily, so that the programs building the image will run correctly. But if the target is not being run under Xen or on EC2, then once it's built, when it is run, it should not have the nosegneg stuff.
          Hide
          Marek Goldmann
          added a comment -

          Your bug report is related to BGBUILD-109.

          We've seen this issue with Fedora Cloud SIG image, see here: https://bugzilla.redhat.com/show_bug.cgi?id=651861.

          I've added proposed fix for this in EC2 plugin, version 0.0.6 and higher, see here: https://github.com/boxgrinder/boxgrinder-build-plugins/commit/5ad6e61a7251ad55804b4964d7d917ccad428900

          I don't know with which version was built latest meta appliance on EC2. I'm going to push new meta appliance soon, with other requests (createrepo & co).

          Show
          Marek Goldmann
          added a comment - Your bug report is related to BGBUILD-109 . We've seen this issue with Fedora Cloud SIG image, see here: https://bugzilla.redhat.com/show_bug.cgi?id=651861 . I've added proposed fix for this in EC2 plugin, version 0.0.6 and higher, see here: https://github.com/boxgrinder/boxgrinder-build-plugins/commit/5ad6e61a7251ad55804b4964d7d917ccad428900 I don't know with which version was built latest meta appliance on EC2. I'm going to push new meta appliance soon, with other requests (createrepo & co).
          Hide
          Bryan O'Sullivan
          added a comment -

          Thanks, Marek! Impressive response, as ever

          Show
          Bryan O'Sullivan
          added a comment - Thanks, Marek! Impressive response, as ever
          Hide
          Marek Goldmann
          added a comment -

          The glibc hack (hwcap 1 nosegneg) needs to be added to a host where BGBUILD is running in case the issue exists. We cannot really do anything about it. But we enable this fix for appliances built using BG.

          Show
          Marek Goldmann
          added a comment - The glibc hack (hwcap 1 nosegneg) needs to be added to a host where BGBUILD is running in case the issue exists. We cannot really do anything about it. But we enable this fix for appliances built using BG.

            People

            • Assignee:
              Marek Goldmann
              Reporter:
              Bryan O'Sullivan
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: