Uploaded image for project: 'BoxGrinder Build'
  1. BoxGrinder Build
  2. BGBUILD-172

Boxgrinder EC2 meta appliance doesn't set up glibc correctly

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: Meta appliance
    • Labels:
      None
    • Environment:
      Amazon EC2 32-bit environments, Fedora

      Description

      There's a severe bug in the chroot environment set up by boxgrinder that affects 32-bit systems running on Amazon EC2, and also under other Xen environments.

      Xen requires a special glibc runtime, named nosegneg in order for applications to access thread-local storage efficiently. This is configured in a file named /etc/ld.so.conf.d/libc6-xen.conf with the following contents:

      hwcap 1 nosegneg
      

      These ld.so config files must also be present in the chroot environment, or the nosegneg version of glibc will not be used. And if those are not used, some applications will be very slow, while others will either crash or go into an infinite loop.

      Here is a bug I reported against GHC that turned out to not be a GHC bug - it was really due to GHC not being run with the nosegneg libraries inside boxgrinder's chroot environment: http://hackage.haskell.org/trac/ghc/ticket/4993

      That problem was causing my attempt to build a VM image with boxgrinder on 32-bit EC2 to hang forever, and it took me ages to figure out why.

      I think there are two parts to fixing this:

      • Copy /etc/ld.so.conf and the files in /etc/ld.so.conf.d into the chroot environment. (Actually, the real fix would be to figure out what creates /etc/ld.so.conf.d/libc6-xen.conf and do the exact same thing, but I've not been able to figure out what does that.)
      • Run ldconfig in the chroot environment.

      Thanks for looking into this. It's a big deal, as it's very hard to understand why an installer gets stuck, and very difficult to work around.

        Gliffy Diagrams

          Issue Links

            Activity

            Hide
            bos31337 Bryan O'Sullivan added a comment -

            By the way, since I believe that the build/appliances/i686/fedora/14/xxx/fedora-plugin/tmp/imgcreate-xxx/install_root directory contains the intended final image, this is actually somewhat more serious than I reported: if the build of an image succeeds without that nosegneg stuff in the built virtual machine image and the image is then run under Xen, we could see crashes or hangs in those VMs at run time, long after the install.

            Show
            bos31337 Bryan O'Sullivan added a comment - By the way, since I believe that the build/appliances/i686/fedora/14/ xxx /fedora-plugin/tmp/imgcreate- xxx /install_root directory contains the intended final image, this is actually somewhat more serious than I reported: if the build of an image succeeds without that nosegneg stuff in the built virtual machine image and the image is then run under Xen, we could see crashes or hangs in those VMs at run time, long after the install.
            Hide
            bos31337 Bryan O'Sullivan added a comment -

            This could easily get confusing.

            If the host where a VM is being built is running under Xen or on EC2, then the chroot in which the target VM image is being built must have the nosegneg magic at least temporarily, so that the programs building the image will run correctly. But if the target is not being run under Xen or on EC2, then once it's built, when it is run, it should not have the nosegneg stuff.

            Show
            bos31337 Bryan O'Sullivan added a comment - This could easily get confusing. If the host where a VM is being built is running under Xen or on EC2, then the chroot in which the target VM image is being built must have the nosegneg magic at least temporarily, so that the programs building the image will run correctly. But if the target is not being run under Xen or on EC2, then once it's built, when it is run, it should not have the nosegneg stuff.
            Hide
            goldmann Marek Goldmann added a comment -

            Your bug report is related to BGBUILD-109.

            We've seen this issue with Fedora Cloud SIG image, see here: https://bugzilla.redhat.com/show_bug.cgi?id=651861.

            I've added proposed fix for this in EC2 plugin, version 0.0.6 and higher, see here: https://github.com/boxgrinder/boxgrinder-build-plugins/commit/5ad6e61a7251ad55804b4964d7d917ccad428900

            I don't know with which version was built latest meta appliance on EC2. I'm going to push new meta appliance soon, with other requests (createrepo & co).

            Show
            goldmann Marek Goldmann added a comment - Your bug report is related to BGBUILD-109 . We've seen this issue with Fedora Cloud SIG image, see here: https://bugzilla.redhat.com/show_bug.cgi?id=651861 . I've added proposed fix for this in EC2 plugin, version 0.0.6 and higher, see here: https://github.com/boxgrinder/boxgrinder-build-plugins/commit/5ad6e61a7251ad55804b4964d7d917ccad428900 I don't know with which version was built latest meta appliance on EC2. I'm going to push new meta appliance soon, with other requests (createrepo & co).
            Hide
            bos31337 Bryan O'Sullivan added a comment -

            Thanks, Marek! Impressive response, as ever

            Show
            bos31337 Bryan O'Sullivan added a comment - Thanks, Marek! Impressive response, as ever
            Hide
            goldmann Marek Goldmann added a comment -

            The glibc hack (hwcap 1 nosegneg) needs to be added to a host where BGBUILD is running in case the issue exists. We cannot really do anything about it. But we enable this fix for appliances built using BG.

            Show
            goldmann Marek Goldmann added a comment - The glibc hack (hwcap 1 nosegneg) needs to be added to a host where BGBUILD is running in case the issue exists. We cannot really do anything about it. But we enable this fix for appliances built using BG.

              People

              • Assignee:
                goldmann Marek Goldmann
                Reporter:
                bos31337 Bryan O'Sullivan
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development