IBM Support

PI44539: CWWDR0012E: JAVA.LANG.STACKOVERFLOWERROR ERROR BETWEEN WEBSPHERE SECURITY AND DRS FOR AUTH CACHE INVALIDATION.

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • Servant region log produced an FFDC containing
    
    CWWDR0012E: java.lang.StackOverflowError
    error.java.lang.RuntimeException: java.lang.StackOverflowError
    at com.ibm.ws390.orb.WS390InterceptorManager.
       interceptOutboundRequest(WS390InterceptorManager.java:151)
    at com.ibm.ws390.orb.ClientDelegate.commonInvoke(
        ClientDelegate.java:1012)
    at com.ibm.ws390.orb.ClientDelegate.invoke(
        ClientDelegate.java:892)
    at org.omg.CORBA.portable.ObjectImpl._invoke(
        ObjectImpl.java:484)
    at com.ibm.ws.drs.ws390.proxy.controller.
        _DRSControllerProxy_Stub.updateEntryProp(
        _DRSControllerProxy_St ub.java:379)
    at com.ibm.ws.drs.ws390.DRSDataXferStub.updateEntryProp(
        DRSDataXfer Stub.java:298)
    at com.ibm.ws.cache.drs.DRSNotificationService.updateEntryProp(
        DRSNotificationService.java:783)
    at com.ibm.ws.cache.drs.DRSNotificationService.pubMsg(
        DRSNotificati onService.java:248)
    at com.ibm.ws.cache.drs.DRSNotificationService.
        publishInvalidateEventsToDRS(
        DRSNotificationService.java:581)
    at com.ibm.ws.cache.drs.DRSNotificationService.batchUpdate(
        DRSNotificationService.java:463)
    at com.ibm.ws.cache.drs.DRSRemoteServices.batchUpdate(
        DRSRemoteServices.java:147)
    at com.ibm.ws.cache.BatchUpdateDaemon.wakeUp(
        BatchUpdateDaemon.java :339)
    at com.ibm.ws.cache.BatchUpdateDaemon.invalidateById(
        BatchUpdateDaemon.java:158)
    at com.ibm.ws.cache.BatchUpdateDaemon.invalidateById(
        BatchUpdateDa e mon.java:137)
    at com.ibm.ws.cache.Cache.invalidateById(Cache.java:1399)
    at com.ibm.ws.cache.DCacheBase.invalidateById(
        DCacheBase.java:423)
    at com.ibm.ws.cache.DistributedObjectCacheAdapter.
        common_invalidate(DistributedObjectCacheAdapter.java:1105)
    at com.ibm.ws.cache.DistributedMapImpl.invalidate(
        DistributedMapImpl.java:91)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        invalidateDistributedObject(
        WSCredentialTokenMapper.java:2255)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:443)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:392)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:342)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        checkCushionValidityOfAllTokens(
        WSCredentialTokenMapper.java:3036)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        checkValidityO fAllTokensAndRefresh
        (WSCredentialTokenMapper.java:3086)
    at com.ibm.ISecurityLocalObjectBaseL13Impl.
        CSICredentialsManager.
        getInvocationSubject(CSICredentialsManager.java:128)
    at com.ibm.ISecurityLocalObjectBaseL13Impl.CSIClientRIForCFW.
        send_request(CSIClientRIForCFW.java:194)
    at com.ibm.rmi.pi.InterceptorManager.invokeInterceptor(
        InterceptorManager.java:590)
    at com.ibm.rmi.pi.InterceptorManager.iterateClientInterceptors(
        InterceptorManager.java:404)
    at com.ibm.rmi.pi.InterceptorManager.iterateSendRequest(
        Interceptor Manager.java:658)
    at com.ibm.ws390.orb.WS390InterceptorManager.
        interceptOutboundReque st(WS390InterceptorManager.java:139)
    at com.ibm.ws390.orb.ClientDelegate.commonInvoke(
        ClientDelegate.java:1012)
    at com.ibm.ws390.orb.ClientDelegate.invoke(
        ClientDelegate.java:892)
    at org.omg.CORBA.portable.ObjectImpl._invoke(
        ObjectImpl.java:484)
    at com.ibm.ws.drs.ws390.proxy.controller.
        _DRSControllerProxy_Stub.updateEntryProp(
        _DRSControllerProxy_Stub.java:379)
    at com.ibm.ws.drs.ws390.DRSDataXferStub.updateEntryProp(
        DRSDataXferStub.java:298)
    at com.ibm.ws.cache.drs.DRSNotificationService.updateEntryProp(
        DRSNotificationService.java:783)
    at com.ibm.ws.cache.drs.DRSNotificationService.pubMsg(
        DRSNotificati onService.java:248)
    at com.ibm.ws.cache.drs.DRSNotificationService.
        publishInvalidateEventsToDRS(
        DRSNotificationService.java:581)
    at com.ibm.ws.cache.drs.DRSNotificationService.
        batchUpdate(DRSNotificationService.java:463)
    at com.ibm.ws.cache.drs.DRSRemoteServices.batchUpdate(
        DRSRemoteServices.java:147)
    at com.ibm.ws.cache.BatchUpdateDaemon.wakeUp(
        BatchUpdateDaemon.java:339)
    at com.ibm.ws.cache.BatchUpdateDaemon.invalidateById(
        BatchUpdateDae mon.java:158)
    at com.ibm.ws.cache.BatchUpdateDaemon.invalidateById(
        BatchUpdateDae mon.java:137)
    at com.ibm.ws.cache.Cache.invalidateById(Cache.java:1399)
    at com.ibm.ws.cache.DCacheBase.invalidateById(
        DCacheBase.java:423)
    at com.ibm.ws.cache.DistributedObjectCacheAdapter.
        common_invalidate(DistributedObjectCacheAdapter.java:1105)
    at com.ibm.ws.cache.DistributedMapImpl.invalidate(
        DistributedMapImpl.java:91)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        invalidateDistributedObject(
        WSCredentialTokenMapper.java:2255)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:443)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:392)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:34 2)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        checkCushionValidityOfAllTokens(
        WSCredentialTokenMapper.java:3036)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        checkValidityOfAllTokensAndRefresh(
        WSCredentialTokenMapper.java:3086)
    at com.ibm.ISecurityLocalObjectBaseL13Impl.
        CSICredentialsManager.
        getInvocationSubject(CSICredentialsManager.java:128)
    at com.ibm.ISecurityLocalObjectBaseL13Impl.CSIClientRIForCFW.
        send_request(CSIClientRIForCFW.java:194)
    at com.ibm.rmi.pi.InterceptorManager.invokeInterceptor(
        InterceptorManager.java:590)
    at com.ibm.rmi.pi.InterceptorManager.iterateClientInterceptors(
        InterceptorManager.java:404)
    at com.ibm.rmi.pi.InterceptorManager.iterateSendRequest(
        InterceptorManager.java:658)
    at com.ibm.ws390.orb.WS390InterceptorManager.
        interceptOutboundRequest(WS390InterceptorManager.java:139)
    at com.ibm.ws390.orb.ClientDelegate.commonInvoke(
        ClientDelegate.java:1012)
    at com.ibm.ws390.orb.ClientDelegate.invoke(
        ClientDelegate.java:892)
    at org.omg.CORBA.portable.ObjectImpl._invoke(
        ObjectImpl.java:484)
    at com.ibm.ws.drs.ws390.proxy.controller.
        _DRSControllerProxy_Stub.updateEntryProp(
        _DRSControllerProxy_Stub.java:379)
    at com.ibm.ws.drs.ws390.DRSDataXferStub.updateEntryProp(
        DRSDataXferStub.java:298)
    at com.ibm.ws.cache.drs.DRSNotificationService.updateEntryProp(
        DRSNotificationService.java:783)
    at com.ibm.ws.cache.drs.DRSNotificationService.pubMsg(
        DRSNotificationService.java:248)
    at com.ibm.ws.cache.drs.DRSNotificationService.
        publishInvalidateEventsToDRS(
        DRSNotificationService.java:581)
    at com.ibm.ws.cache.drs.DRSNotificationService.batchUpdate(
        DRSNotificationService.java:463)
    at com.ibm.ws.cache.drs.DRSRemoteServices.batchUpdate(
        DRSRemoteServices.java:147)
    at com.ibm.ws.cache.BatchUpdateDaemon.wakeUp(
        BatchUpdateDaemon.java:339)
    at com.ibm.ws.cache.BatchUpdateDaemon.invalidateById(
        BatchUpdateDae mon.java:158)
    at com.ibm.ws.cache.BatchUpdateDaemon.invalidateById(
        BatchUpdateDae mon.java:137)
    at com.ibm.ws.cache.Cache.invalidateById(Cache.java:1399)
    at com.ibm.ws.cache.DCacheBase.invalidateById(
        DCacheBase.java:423)
    at com.ibm.ws.cache.DistributedObjectCacheAdapter.
        common_invalidate (DistributedObjectCacheAdapter.java:1105)
    at com.ibm.ws.cache.DistributedMapImpl.invalidate(
        DistributedMapImpl.java:91)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        invalidateDistributedObject(
        WSCredentialTokenMapper.java:2255)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:443)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:392)
    at com.ibm.ws.security.auth.AuthCache.removeEntry(
        AuthCache.java:342)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        checkCushionValidityOfAllTokens(
        WSCredentialTokenMapper.java:3036)
    at com.ibm.ws.security.token.WSCredentialTokenMapper.
        checkValidityOfAllTokensAndRefresh(
        WSCredentialTokenMapper.java:3086)
    at com.ibm.ISecurityLocalObjectBaseL13Impl.
        CSICredentialsManager.getInvocationSubject(
        CSICredentialsManager.java:128)
    at com.ibm.ISecurityLocalObjectBaseL13Impl.CSIClientRIForCFW.
        send_request(CSIClientRIForCFW.java:194)
    ...
    In one FFDc, there were 112 recursions through
    com.ibm.ISecurityLocalObjectBaseL13Impl.CSIClientRIForCFW.
        send_request(CSIClientRIForCFW.java:194)
    

Local fix

Problem summary

  • ****************************************************************
    * USERS AFFECTED:  All users of IBM WebSphere Application      *
    *                  Server V8.0 and V*.5                        *
    ****************************************************************
    * PROBLEM DESCRIPTION: StackOverflowError or looping in        *
    *                      CSIv2 outbound processing when          *
    *                      KRBAuthnToken is expired.               *
    ****************************************************************
    * RECOMMENDATION:                                              *
    ****************************************************************
    During CSIv2 outbound request processing, security code can
    find itself in a loop when the KRBAuthnToken is expired.
    Outbound processing will trigger validation of the invocation
    subject. If tokens, in particular the KRBAuthnToken, are
    expired the subject will be refreshed, if possible.
    To refresh the subject we will attempt to validate each token
    and update the expiration timestamp. Then, remove the subject
    from both AuthCache and Dynacache.
    We go through refresh processing and find that the
    KRBAuthnToken is expired and can not be updated because the
    Kerberos token was not delegated. We exit refresh processing,
    but incorrectly indicate the subject was refreshed. This is
    the error.
    We then try to remove the subject from Dynacache. If a DRS
    Replication Domain is configured, we must send an outbound
    request to the replication domain. This sends us back into the
    same loop in CSIv2 outbound processing. We attempt to refresh
    again, we fail to refresh, attempt to cleanup Dynacache, go
    outbound, etc....
    

Problem conclusion

Temporary fix

Comments

APAR Information

  • APAR number

    PI44539

  • Reported component name

    WEBSPHERE FOR Z

  • Reported component ID

    5655I3500

  • Reported release

    800

  • Status

    CLOSED PER

  • PE

    NoPE

  • HIPER

    NoHIPER

  • Special Attention

    NoSpecatt / Xsystem

  • Submitted date

    2015-07-07

  • Closed date

    2015-10-05

  • Last modified date

    2016-02-18

  • APAR is sysrouted FROM one or more of the following:

  • APAR is sysrouted TO one or more of the following:

Fix information

  • Fixed component name

    WEBSPHERE FOR Z

  • Fixed component ID

    5655I3500

Applicable component levels

  • R800 PSY

       UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"800","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
27 April 2022