Uploaded image for project: 'Red Hat CodeReady Studio (devstudio)'
  1. Red Hat CodeReady Studio (devstudio)
  2. JBDS-2254

clean out redundant test runtime caches on in static_build_env

    XMLWordPrintable

    Details

      Description

      Noticed today that we have a lot of duplicate copies of the same runtimes. For example:

      /home/hudson/static_build_env/jbds/download-cache/soa-p-5.3.0.ER2.zip
      /home/hudson/static_build_env/jbds/requirements/download/soa-p-5.3.0.ER2/soa-p-5.3.0.ER2.zip
      /home/hudson/static_build_env/jbds/requirements/download/soap/soa-p-5.3.0.ER2.zip
      

      Doing a full search for dupes from the download-cache folder vs. the requirements/download/ folder, I see:

      $ cd /home/hudson/static_build_env/jbds/download-cache; for d in 
      $(find . -type f -name "*.zip" | sort); do du -sch 
      {/home/hudson/static_build_env/jbds/download-cache/,/home/hudson/static_build_env/jbds/requirements/download/*/}${d};
      done
      
      178M    /home/hudson/static_build_env/jbds/download-cache/./GateIn-3.1.0-FINAL-jbossas.zip
      178M    /home/hudson/static_build_env/jbds/requirements/download/gatein-3.1.0.Final/./GateIn-3.1.0-FINAL-jbossas.zip
      355M    total
      
      8.0K    /home/hudson/static_build_env/jbds/download-cache/./jbds-seamfp-patch.zip
      du: cannot access `/home/hudson/static_build_env/jbds/requirements/download/*/./jbds-seamfp-patch.zip': No such file or directory
      8.0K    total
      
      54M     /home/hudson/static_build_env/jbds/download-cache/./jboss-3.2.8.SP1.zip
      54M     /home/hudson/static_build_env/jbds/requirements/download/jbossas/./jboss-3.2.8.SP1.zip
      107M    total
      
      78M     /home/hudson/static_build_env/jbds/download-cache/./jboss-4.0.5.GA.zip
      78M     /home/hudson/static_build_env/jbds/requirements/download/jbossas/./jboss-4.0.5.GA.zip
      155M    total
      
      105M    /home/hudson/static_build_env/jbds/download-cache/./jboss-5.0.1.GA.zip
      105M    /home/hudson/static_build_env/jbds/requirements/download/jbossas/./jboss-5.0.1.GA.zip
      209M    total
      
      128M    /home/hudson/static_build_env/jbds/download-cache/./jboss-5.1.0.GA.zip
      128M    /home/hudson/static_build_env/jbds/requirements/download/jbossas-5.1.0.GA/./jboss-5.1.0.GA.zip
      128M    /home/hudson/static_build_env/jbds/requirements/download/jbossas/./jboss-5.1.0.GA.zip
      383M    total
      
      270M    /home/hudson/static_build_env/jbds/download-cache/./jboss-eap-4.3.0.GA_CP10.zip
      du: cannot access `/home/hudson/static_build_env/jbds/requirements/download/*/./jboss-eap-4.3.0.GA_CP10.zip': No such file or directory
      270M    total
      
      293M    /home/hudson/static_build_env/jbds/download-cache/./jboss-eap-5.1.2.zip
      du: cannot access `/home/hudson/static_build_env/jbds/requirements/download/*/./jboss-eap-5.1.2.zip': No such file or directory
      293M    total
      
      110M    /home/hudson/static_build_env/jbds/download-cache/./jboss-eap-6.0.0.zip
      110M    /home/hudson/static_build_env/jbds/requirements/download/jbeap-6/./jboss-eap-6.0.0.zip
      220M    total
      
      78M     /home/hudson/static_build_env/jbds/download-cache/./jboss-eap-fp-4.3.0.CP03-FP01.zip
      78M     /home/hudson/static_build_env/jbds/requirements/download/seamfp/./jboss-eap-fp-4.3.0.CP03-FP01.zip
      155M    total
      
      193M    /home/hudson/static_build_env/jbds/download-cache/./jboss-eap-noauth-4.3.0.GA_CP03.zip
      193M    /home/hudson/static_build_env/jbds/requirements/download/jbeap/./jboss-eap-noauth-4.3.0.GA_CP03.zip
      385M    total
      
      293M    /home/hudson/static_build_env/jbds/download-cache/./jboss-eap-noauth-5.1.2.zip
      du: cannot access `/home/hudson/static_build_env/jbds/requirements/download/*/./jboss-eap-noauth-5.1.2.zip': No such file or directory
      293M    total
      
      189M    /home/hudson/static_build_env/jbds/download-cache/./jbossesb-4.11.zip
      189M    /home/hudson/static_build_env/jbds/requirements/download/jbossesb-4.11/./jbossesb-4.11.zip
      377M    total
      
      157M    /home/hudson/static_build_env/jbds/download-cache/./jboss-portal-2.7.2-bundled.zip
      157M    /home/hudson/static_build_env/jbds/requirements/download/JBoss-Portal-2.7.2.GA/./jboss-portal-2.7.2-bundled.zip
      314M    total
      
      26M     /home/hudson/static_build_env/jbds/download-cache/./JBossPortletBridge-2.1.0.FINAL.zip
      26M     /home/hudson/static_build_env/jbds/requirements/download/JBossPortletBridge-2.1.0.FINAL/./JBossPortletBridge-2.1.0.FINAL.zip
      51M     total
      
      93M     /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.0.1.GA.zip
      93M     /home/hudson/static_build_env/jbds/requirements/download/seam-2.0.1.GA/./jboss-seam-2.0.1.GA.zip
      186M    total
      
      89M     /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.0.2.SP1.zip
      89M     /home/hudson/static_build_env/jbds/requirements/download/seam-2.0.2.SP1/./jboss-seam-2.0.2.SP1.zip
      177M    total
      
      89M     /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.0.3.CR1.zip
      89M     /home/hudson/static_build_env/jbds/requirements/download/seam-2.0.3.CR1/./jboss-seam-2.0.3.CR1.zip
      178M    total
      
      107M    /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.1.1.GA.zip
      107M    /home/hudson/static_build_env/jbds/requirements/download/seam-2.1.1.GA/./jboss-seam-2.1.1.GA.zip
      213M    total
      
      137M    /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.1.2.zip
      137M    /home/hudson/static_build_env/jbds/requirements/download/seam-2.1.2.GA/./jboss-seam-2.1.2.zip
      273M    total
      
      141M    /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.2.0.CR1.zip
      141M    /home/hudson/static_build_env/jbds/requirements/download/seam-2.2.0.CR1/./jboss-seam-2.2.0.CR1.zip
      281M    total
      
      113M    /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.2.0.GA.zip
      113M    /home/hudson/static_build_env/jbds/requirements/download/seam-2.2.0.GA/./jboss-seam-2.2.0.GA.zip
      225M    total
      
      108M    /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.2.1.Final.zip
      108M    /home/hudson/static_build_env/jbds/requirements/download/seam-2.2.1.Final/./jboss-seam-2.2.1.Final.zip
      216M    total
      
      110M    /home/hudson/static_build_env/jbds/download-cache/./jboss-seam-2.3.0.ALPHA-distribution.zip
      110M    /home/hudson/static_build_env/jbds/requirements/download/seam-2.3.0/./jboss-seam-2.3.0.ALPHA-distribution.zip
      219M    total
      
      24M     /home/hudson/static_build_env/jbds/download-cache/./jbpm-3.1.4.zip
      24M     /home/hudson/static_build_env/jbds/requirements/download/jbpm-3.1.4/./jbpm-3.1.4.zip
      47M     total
      
      8.2M    /home/hudson/static_build_env/jbds/download-cache/./rhds-seam-gen-patch.zip
      8.2M    /home/hudson/static_build_env/jbds/requirements/download/eapseamgen/./rhds-seam-gen-patch.zip
      17M     total
      
      60M     /home/hudson/static_build_env/jbds/download-cache/./riftsaw-2.3.0.Final.zip
      60M     /home/hudson/static_build_env/jbds/requirements/download/riftsaw-2.3.0.Final/./riftsaw-2.3.0.Final.zip
      120M    total
      
      569M    /home/hudson/static_build_env/jbds/download-cache/./soa-4.3.0.GA_CP05.zip
      569M    /home/hudson/static_build_env/jbds/requirements/download/soa-4.3.0.GA_CP05/./soa-4.3.0.GA_CP05.zip
      569M    /home/hudson/static_build_env/jbds/requirements/download/soap/./soa-4.3.0.GA_CP05.zip
      1.7G    total
      
      593M    /home/hudson/static_build_env/jbds/download-cache/./soa-5.0.1.GA-signed.zip
      593M    /home/hudson/static_build_env/jbds/requirements/download/soa-5.0.1.GA-signed/./soa-5.0.1.GA-signed.zip
      593M    /home/hudson/static_build_env/jbds/requirements/download/soap/./soa-5.0.1.GA-signed.zip
      1.8G    total
      
      635M    /home/hudson/static_build_env/jbds/download-cache/./soa-p-5.3.0.ER2.zip
      635M    /home/hudson/static_build_env/jbds/requirements/download/soa-p-5.3.0.ER2/./soa-p-5.3.0.ER2.zip
      635M    /home/hudson/static_build_env/jbds/requirements/download/soap/./soa-p-5.3.0.ER2.zip
      1.9G    total
      
      629M    /home/hudson/static_build_env/jbds/download-cache/./soa-p-5.3.0.ER5.zip
      du: cannot access `/home/hudson/static_build_env/jbds/requirements/download/*/./soa-p-5.3.0.ER5.zip': No such file or directory
      629M    total
      
      

      Full disk footprint between the two locations:

      $ du -sch /home/hudson/static_build_env/jbds/download-cache /home/hudson/static_build_env/jbds/requirements/download/
      
      5.6G    /home/hudson/static_build_env/jbds/download-cache
      13G     /home/hudson/static_build_env/jbds/requirements/download/
      18G     total
      

      So... seems like there are two problems:

      a) we have a lot of old/obsolete reqs which can be purged
      b) we fetch into two places - I suspect one path (download-cache?) is the new approach mistria wrote for JBDS 6, and the old path is the old Ant-based fetch approach (requirements/download/).

      Spoke with Len and he said we were good to purge these, but I wanted to double-check w/ mistria and dgolovin that wiping out all 18G and letting the builds re-fetch what they need is an acceptable way to clean up this area.

      Also, can we fix JBDS 5 (and 4?) so it uses the new maven-based fetch instead of the old Ant one, in order to avoid the duplicates and disk usage?

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  nickboldt Nick Boldt
                  Reporter:
                  nickboldt Nick Boldt
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  5 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: