Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-5591

Fabric git pushes rejected due to non fast forward

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: jboss-fuse-6.2
    • Fix Version/s: jboss-fuse-6.3
    • Component/s: Fabric8 v1
    • Labels:
      None
    • Steps to Reproduce:
      Hide

      TBD

      Show
      TBD
    • Workaround Description:
      Hide

      Unknown

      Show
      Unknown

      Description

      In an installation where:

      • there are many containers, > 400,
      • alternating live and stage profile versions [see below for more detail]
      • more than one fabric-git version in play
        • 1.2.0.redhat-161
        • 1.2.0.redhat-165

      It is often seen in the logs:

      2016-04-21 10:59:18,877 | INFO | pool-36-thread-1 | io.fabric8.git.internal.DefaultPullPushPolicy | 69 - io.fabric8.fabric-git - 1.2.0.redhat-161 |  -  -  |  -  -  | Performing a pull on remote URL: http://d154.mydomain.org:1234/git/fabric/
      2016-04-21 10:59:26,199 | WARN | pool-36-thread-1 | io.fabric8.git.internal.DefaultPullPushPolicy | 69 - io.fabric8.fabric-git - 1.2.0.redhat-161 |  -  -  |  -  -  | Pull failed because of: org.eclipse.jgit.api.errors.TransportException: Read timed out after 5,000 ms
      2016-04-21 10:59:26,858 | INFO | pool-36-thread-1 | io.fabric8.git.internal.DefaultPullPushPolicy | 69 - io.fabric8.fabric-git - 1.2.0.redhat-161 |  -  -  |  -  -  | Performing a pull on remote URL: http://d154.mydomain.org:1234/git/fabric/
      2016-04-21 10:59:33,782 | WARN | pool-36-thread-1 | io.fabric8.git.internal.DefaultPullPushPolicy | 69 - io.fabric8.fabric-git - 1.2.0.redhat-161 |  -  -  |  -  -  | Pull failed because of: org.eclipse.jgit.api.errors.TransportException: Read timed out after 5,000 ms
      2016-04-21 10:59:46,963 | INFO | pool-36-thread-1 | io.fabric8.git.internal.DefaultPullPushPolicy | 69 - io.fabric8.fabric-git - 1.2.0.redhat-161 |  -  -  |  -  -  | Performing a pull on remote URL: http:/XXXX:1234/git/fabric/
      

      and infrequently seen:

      run-em_p10.log:java.lang.IllegalStateException: Push rejected: [RemoteRefUpdate[remoteName=refs/heads/60.2, REJECTED_NONFASTFORWARD, (null)...bec7f3e3383f43c61d6f7f4ff1029ae2e5fa08a5, srcRef=refs/heads/60.2, message=null], RemoteRefUpdate[
      remoteName=refs/heads/50.2, REJECTED_NONFASTFORWARD, (null)...592df1b0ea7a98e34aa22311e9ed10c67ef7fd4c, srcRef=refs/heads/50.2, message=null]]
      run-em_p12.log:2016-05-10 07:56:38,168 | WARN | qtp1672381131-66692 | io.fabric8.git.internal.DefaultPullPushPolicy | 79 - io.fabric8.fabric-git - 1.2.0.redhat-161 |  -  -  | Rejected push: {}RemoteRefUpdate[remoteName=refs/heads/60.1, REJ
      ECTED_NONFASTFORWARD, (null)...f80d25ef779b93e3ab7f616da372a715e6403364, srcRef=refs/heads/60.1, message=null]
      

      or

      2016-05-13 12:16:15,720 | INFO | Thread-61794 | io.fabric8.git.internal.DefaultPullPushPolicy | 79 - io.fabric8.fabric-git - 1.2.0.redhat-161 |  -  -  | Push result: [accepted=25,rejected=1,error=null]
      

      Where the timeout had previously been increased to 15,000ms with no change in behaviour and the version strategy is to have 2 versions for each of the regions so that regions can be moved forward at their own pace and are not tied to another region. Then it alternates between the versions having 1 be a live version and the other a stage version for the next go-live update:

      i.e.
      10.1 - 10.2
      11.1 - 11.2
      20.1 - 20.2
      30.1 - 30.2
      40.1 - 40.2
      50.1 - 50.2
      60.1 - 60.2
      70.1 - 70.2

      As well rarely it has been seen:

      2016-05-17 14:10:23,912 | WARN | pool-22-thread-1 | io.fabric8.service.ZkDataStoreImpl | 67 - io.fabric8.fabric-core - 1.2.0.redhat-153 |  -  -  |  -  -  | Caught: java.lang.IllegalStateException: Cannot obtain profile read lock in time
      java.lang.IllegalStateException: Cannot obtain profile read lock in time
      	at io.fabric8.api.gravia.IllegalStateAssertion.assertTrue(IllegalStateAssertion.java:55)[66:io.fabric8.fabric-api:1.2.0.redhat-133]
      	at io.fabric8.git.internal.GitDataStoreImpl.aquireReadLock(GitDataStoreImpl.java:434)[69:io.fabric8.fabric-git:1.2.0.redhat-161]
      	at io.fabric8.git.internal.GitDataStoreImpl.getVersionFromCacheRO(GitDataStoreImpl.java:485)[69:io.fabric8.fabric-git:1.2.0.redhat-161]
      	at io.fabric8.git.internal.GitDataStoreImpl.getVersionFromCache(GitDataStoreImpl.java:477)[69:io.fabric8.fabric-git:1.2.0.redhat-161]
      	at io.fabric8.git.internal.GitDataStoreImpl.getVersion(GitDataStoreImpl.java:642)[69:io.fabric8.fabric-git:1.2.0.redhat-161]
      	at io.fabric8.internal.ProfileServiceImpl.getVersion(ProfileServiceImpl.java:126)[67:io.fabric8.fabric-core:1.2.0.redhat-153]
      	at io.fabric8.internal.ContainerImpl.getVersion(ContainerImpl.java:203)[67:io.fabric8.fabric-core:1.2.0.redhat-153]
      	at io.fabric8.internal.ContainerImpl.getContainerProfile(ContainerImpl.java:762)[67:io.fabric8.fabric-core:1.2.0.redhat-153]
      	at io.fabric8.internal.ContainerImpl.getOverlayProfile(ContainerImpl.java:306)[67:io.fabric8.fabric-core:1.2.0.redhat-153]
      

      The end resut of which is that the Fabric Git repository is "reverting" to previous values after updating profiles.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  grgrzybek Grzegorz Grzybek
                  Reporter:
                  roddiekieley Roderick Kieley
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  7 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: