Uploaded image for project: 'Keycloak'
  1. Keycloak
  2. KEYCLOAK-9765

Zero down-time between redeployes and AuthorizationTokenService - Unexpected error while evaluating permissions

    XMLWordPrintable

    Details

    • Type: Enhancement
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Rejected
    • Affects Version/s: 4.8.3.Final
    • Fix Version/s: None
    • Component/s: Authorization Services
    • Labels:
      None
    • Environment:
    • Docs QE Status:
      NEW
    • QE Status:
      NEW

      Description

      We want to have "Zero down time" between redeploys/upgrades of Keycloak. We do this by upgrading first instance, that the second one. (e.g. docker-compose stop 1, up 1, wait for 1, stop 2, up 2).

      Everything works ok from the point of deyploment.
      BUT user UMA authorizations start to fail (token is valid and is not logged out).

      Ones LB Traefik switches to another Keycloak instance (or back to first one). Exception ocures when adapter calls Keyclaok to get UMA ticket:
      {{+++
      petrol-keycloak-2 | 11:05:00,723 ERROR [org.keycloak.authorization.authorization.AuthorizationTokenService] (default task-4) Unexpected error while evaluating permissions: java.lang.RuntimeException: Error while reading attributes from security token.
      petrol-keycloak-2 | at org.keycloak.authorization.common.KeycloakIdentity.<init>(KeycloakIdentity.java:146)
      petrol-keycloak-2 | at org.keycloak.authorization.common.KeycloakIdentity.<init>(KeycloakIdentity.java:69)
      petrol-keycloak-2 | at org.keycloak.authorization.authorization.AuthorizationTokenService.lambda$static$1(AuthorizationTokenService.java:130)
      petrol-keycloak-2 | at org.keycloak.authorization.authorization.AuthorizationTokenService.createEvaluationContext(AuthorizationTokenService.java:378)
      petrol-keycloak-2 | at org.keycloak.authorization.authorization.AuthorizationTokenService.authorize(AuthorizationTokenService.java:159)
      petrol-keycloak-2 | at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.permissionGrant(TokenEndpoint.java:1153)
      petrol-keycloak-2 | at org.keycloak.protocol.oidc.endpoints.TokenEndpoint.processGrantRequest(TokenEndpoint.java:193)
      +++}}

      Stack trace from Jetty Adapter:
      {{+++
      2019-02-19 14:15:31,828 WARN /relationships/candidates {} java.lang.RuntimeException: Failed to enforce policy decisions. at org.keycloak.adapters.AuthenticatedActionsHandler.isAuthorized(AuthenticatedActionsHandler.java:168)
      at org.keycloak.adapters.AuthenticatedActionsHandler.handledRequest(AuthenticatedActionsHandler.java:60)
      at org.keycloak.adapters.jetty.core.AbstractKeycloakJettyAuthenticator.validateRequest(AbstractKeycloakJettyAuthenticator.java:311)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:483)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
      +++}}

      This error is unrecoverable. User must logout and login again. (or wait for token to expire on J2EE aplication)

      THE SOLUTION:

      We have configured infinispan cache with jboss-cli (embed-server) during building the image:
      <distributed-cache name="sessions" owners="2"/>
      <distributed-cache name="authenticationSessions" owners="2"/>
      <distributed-cache name="offlineSessions" owners="2"/>
      <distributed-cache name="clientSessions" owners="2"/>
      <distributed-cache name="offlineClientSessions" owners="2"/>
      <distributed-cache name="loginFailures" owners="2"/>
      <distributed-cache name="actionTokens" owners="2">

      Now caches seems to be replicated on both Keycloak instances. User stays logged-in, resolving UMA authorizations are switch to one instance and back. Everything works very-well.

      QUESTIONS:
      1) Is this a good solution to set owners="2" to distributed-cache? What are drawbacks? Should we use this solution or not?
      2) Sometimes UMA flow still fails, and warning is reported in log. After browser F5 refresh. Everything works again. Why is this happening? It is not a big issue, but is there a way to force trigger cache sync between Keycloak upgrades?
      {{+++
      petrol-keycloak-1 | 07:05:07,678 WARN [org.keycloak.models.sessions.infinispan.changes.InfinispanChangelogBasedTransaction] (default task-11) Not present cache item for key d8213936-ad2d-45df-b14d-322e886e9f34
      petrol-keycloak-1 | 07:05:09,279 WARN [org.keycloak.models.sessions.infinispan.changes.InfinispanChangelogBasedTransaction] (default task-11) Not present cache item for key d8213936-ad2d-45df-b14d-322e886e9f34
      +++}}

      Thanks for doing a great job with Keycloak.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                klemenki Klemen Kisel
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: