IBM Support

PI97241: SERVER HANG ON SERVER SHUTDOWN DUE TO DEADLOCK

Fixes are available

8.5.5.15: WebSphere Application Server V8.5.5 Fix Pack 15
9.0.0.11: WebSphere Application Server traditional V9.0 Fix Pack 11
9.0.5.0: WebSphere Application Server traditional Version 9.0.5 Refresh Pack
9.0.5.1: WebSphere Application Server traditional Version 9.0.5 Fix Pack 1
9.0.5.2: WebSphere Application Server traditional Version 9.0.5 Fix Pack 2
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
9.0.5.3: WebSphere Application Server traditional Version 9.0.5 Fix Pack 3
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21

Subscribe

You can track all active APARs for this component.

 

APAR status

  • Closed as program error.

Error description

  • Running WebSphere Application Server Network Deployment version
8.0.0.14, experiencing intermittent, unpredictable,
non-reproducible server hangs on server shutdown.

Javacore file generated with linperf.sh script reveals deadlock:

1LKDEADLOCK Deadlock detected !!! NULL ---------------------
NULL 2LKDEADLOCKTHR Thread "WorkManager.IB_RETAIL_WM : 17"
(0x000000008D29DE00) 3LKDEADLOCKWTR is waiting for:
4LKDEADLOCKMON sys_mon_t:0x00007F65F15B7F88 infl_mon_t:
0x00007F65F15B8000: 4LKDEADLOCKOBJ
java/lang/Object@0x0000000001C10410 3LKDEADLOCKOWN which is
owned by: 2LKDEADLOCKTHR Thread "WorkManager.IB_RETAIL_WM : 3"
(0x000000008CB7D600) 3LKDEADLOCKWTR which is waiting for:
4LKDEADLOCKMON sys_mon_t:0x00007F65ED9B94F8 infl_mon_t:
0x00007F65ED9B9570: 4LKDEADLOCKOBJ
com/ibm/ejs/j2c/J2CGlobalConfigProperties@0x000000001A5EC5E0
3LKDEADLOCKOWN which is owned by: 2LKDEADLOCKTHR Thread
"WorkManager.IB_RETAIL_WM : 17" (0x000000008D29DE00)

while executing these threads:
3XMTHREADBLOCK Blocked on:
com/ibm/ejs/j2c/J2CGlobalConfigProperties@0x000000001A5EC5E0
Owned by: "WorkManager.IB_RETAIL_WM : 17"
(J9VMThread:0x000000008D29DE00,
java/lang/Thread:0x000000001B4A8C88) 3XMHEAPALLOC Heap bytes
allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
callstack: 4XESTACKTRACE at
com/ibm/ejs/j2c/J2CDiagnosticAlertHelperImpl.propertyChange
(J2CDiagnosticAlertHelperImpl.java:969) 4XESTACKTRACE at
com/ibm/ejs/j2c/J2CGlobalConfigProperties.propertyChange
(J2CGlobalConfigProperties.java:2162) 4XESTACKTRACE at
com/ibm/ws/rasdiag/AlertConfig.
notifyListeners(AlertConfig.java:388)

and

3XMTHREADINFO "WorkManager.IB_RETAIL_WM : 17" J9VMThread:
0x000000008D29DE00, j9thread_t:0x00007F65F4064D10,
java/lang/Thread: 0x000000001B4A8C88, state:B, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x13D, isDaemon:true)
3XMTHREADINFO1 (native thread ID:0xB4B3, native priority: 0x5,
native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201)
3XMTHREADINFO2 (native stack address range from:
0x00007F65D6A73000, to:0x00007F65D6AB4000, size:0x41000)
3XMCPUTIME CPU usage total: 4.011094347 secs 3XMTHREADBLOCK
Blocked on: java/lang/Object@0x0000000001C10410 Owned by:
"WorkManager.IB_RETAIL_WM : 3" (J9VMThread: 0x000000008CB7D600,
java/lang/Thread:0x000000001B436F20) 3XMHEAPALLOC Heap bytes
allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
callstack: 4XESTACKTRACE at
com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriver.compl
eteInit (ServerRuleDriver.java:1147) 4XESTACKTRACE at
com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriverMBean.
getRPAAttributeValue(ServerRuleDriverMBean.java:1116)
4XESTACKTRACE at
com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriverMBean.
getRPAAttributeValues(ServerRuleDriverMBean.java:1086)

Local fix

  • n/a

Problem summary

  • ****************************************************************
* USERS AFFECTED:  All users of IBM WebSphere Application      *
*                  Server using performance monitoring and     *
*                  datasources                                 *
****************************************************************
* PROBLEM DESCRIPTION: A intermittent deadlock may occur       *
*                      between the ServerRullDriverMBean and   *
*                      the J2CDiagnosticAlertHelperImpl code.  *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
A java core will have the following threads deadlocked, while
executing these threads:
3XMTHREADBLOCK Blocked on:
com/ibm/ejs/j2c/J2CGlobalConfigProperties@0x000000001A5EC5E0
Owned by: "WorkManager.IB_RETAIL_WM : 17"
(J9VMThread:0x000000008D29DE00,
java/lang/Thread:0x000000001B4A8C88) 3XMHEAPALLOC Heap bytes
allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
callstack: 4XESTACKTRACE at
com/ibm/ejs/j2c/J2CDiagnosticAlertHelperImpl.propertyChange
(J2CDiagnosticAlertHelperImpl.java:969) 4XESTACKTRACE at
com/ibm/ejs/j2c/J2CGlobalConfigProperties.propertyChange
(J2CGlobalConfigProperties.java:2162) 4XESTACKTRACE at
com/ibm/ws/rasdiag/AlertConfig.
notifyListeners(AlertConfig.java:388)
and
3XMTHREADINFO "WorkManager.IB_RETAIL_WM : 17" J9VMThread:
0x000000008D29DE00, j9thread_t:0x00007F65F4064D10,
java/lang/Thread: 0x000000001B4A8C88, state:B, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x13D, isDaemon:true)
3XMTHREADINFO1 (native thread ID:0xB4B3, native priority: 0x5,
native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000201)
3XMTHREADINFO2 (native stack address range from:
0x00007F65D6A73000, to:0x00007F65D6AB4000, size:0x41000)
3XMCPUTIME CPU usage total: 4.011094347 secs 3XMTHREADBLOCK
Blocked on: java/lang/Object@0x0000000001C10410 Owned by:
"WorkManager.IB_RETAIL_WM : 3" (J9VMThread: 0x000000008CB7D600,
java/lang/Thread:0x000000001B436F20) 3XMHEAPALLOC Heap bytes
allocated since last GC cycle=0 (0x0) 3XMTHREADINFO3 Java
callstack: 4XESTACKTRACE at
com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriver.compl
eteInit (ServerRuleDriver.java:1147) 4XESTACKTRACE at
com/ibm/ws/performance/tuning/serverAlert/ServerRuleDriverMBean.
getRPAAttributeValue(ServerRuleDriverMBean.java:1116)

Problem conclusion

Temporary fix

  • 



Comments


    

  • 



APAR Information




    

  • APAR number
    PI97241
    • 

  • Reported component name
    WEBS APP SERV N
    • 

  • Reported component ID
    5724H8800
    • 

  • Reported release
    800
    • 

  • Status
    CLOSED  PER
    • 

  • PE
    NoPE
    • 

  • HIPER
    NoHIPER
    • 

  • Special Attention
    NoSpecatt / Xsystem
    • 

  • Submitted date
    2018-04-25
    • 

  • Closed date
    2018-12-06
    • 

  • Last modified date
    2018-12-06
    • 





    

  • APAR is sysrouted FROM one or more of the following:
    

    • 

  • APAR is sysrouted TO one or more of the following:
    

    • 



Fix information


    

  • Fixed component name
    WEBS APP SERV N
    • 

  • Fixed component ID
    5724H8800
    • 



Applicable component levels


    

  • R800  PSY
       UP
    • 

  • R850  PSY
       UP
    • 

  • R900  PSY
       UP
    • 








      
              
                            

              

              [{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.0","Line of Business":{"code":"LOB45","label":"Automation"}}]
              

                          

          
 
        

      

    

      

        

          

            
Document Information


            

            


                        

            Modified date:
            

            27 April 2022
            

            
                      

        


        

        


      

    

  



    

  

  
    
            

          
Page Feedback