PM91608: DEADLOCK IN COM/IBM/WS/ASYNCHBEANS/SERVICES/WLM/ENCLAVE

Fixes are available

APAR status

Closed as program error.

Error description

When running Websphere Application Server zOS, the
com/ibm/ws/asynchbeans/services/wlm/EnclaveManager goes into a
deadlock with two threads causing the server to hang and/or
slowdown.
The Servant could timeout and abend with an ABENDEC3 RC=0413000x
.
Looking at a javacore dump from the servant region you may see
the following deadlock contention.
.
1LKDEADLOCK    Deadlock detected !!!
NULL           ---------------------
NULL
2LKDEADLOCKTHR  Thread "WebSphere WLM Dispatch Thread
                  t=008b3620" (0x0000000040F03A00)
3LKDEADLOCKWTR    is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x00000049A1DE4738
               infl_mon_t: 0x00000049A1DE47B0:
4LKDEADLOCKOBJ      com/ibm/ws/asynchbeans/services/wlm/
               Enclave@0x000000009E60F370
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "java.lang.ThreadGroup&#65517;name=xx.zzzzzzzzzz
                :xxxxxxxx.0,
3LKDEADLOCKWTR    which is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x00000049A1DE46B0
                infl_mon_t: 0x00000049A1DE4728:
4LKDEADLOCKOBJ      java/util/HashMap@0x0000000082724078
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "WebSphere WLM Dispatch Thread
        t=008b3620" (0x0000000040F03A00)

Local fix

Problem summary

****************************************************************
* USERS AFFECTED:  All users of IBM WebSphere Application      *
*                  Server V8.5                                 *
****************************************************************
* PROBLEM DESCRIPTION: Deadlock between asynchbeans            *
*                      registerEnclave and deregisterEnclave   *
*                      threads                                 *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
Under extreme load due to timing issues a deadlock can occur
between the following two thread stacks involving asynchbeans
EnclaveManager.
WorkManagerImpl.schedule():
"WebSphere WLM Dispatch Thread t=008b6528"
J9VMThread:0x00000000400C3000, j9thread_t:0x0000004808FC3260,
java/lang/Thread:0x00000000992C5F38, state:B, prio=5
(native thread ID:0x4899B100, native priority:0x5,
native policy:UNKNOWN)
Blocked on:
com/ibm/ws/asynchbeans/services/wlm/Enclave@0x000000009D74F590
Owned by: "java.lang.ThreadGroup[name=wm.XXXName:
XXXX04.0,maxpri=10]" (J9VMThread:0x0000000041030C00,
java/lang/Thread:0x000000009D6F17E8)
Java callstack:
at
com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.registerEncla
ve(EnclaveManager.java:214)
(entered lock:
java/util/HashMap@0x000000008273FD30, entry count: 1)
at
com/ibm/ws/asynchbeans/services/wlm/ClassificationService.peek(C
lassificationService.java:248)
at
com/ibm/ws/asynchbeans/J2EEServiceManager.peek(J2EEServiceManage
r.java:361(Compiled Code))
at
com/ibm/ws/asynchbeans/WorkManagerImpl.create(WorkManagerImpl.ja
va:842)
at
com/ibm/ws/asynchbeans/WLMWorkManagerImpl.create(WLMWorkManagerI
mpl.java:180)
at
com/ibm/ws/asynchbeans/WorkManagerImpl.schedule(WorkManagerImpl.
java:921)
Work.run():
"java.lang.ThreadGroup[name=wm.XXXX: XXXX4.0,maxpri=10]"
J9VMThread:0x0000000041030C00, j9thread_t:0x0000004A6FA48560,
java/lang/Thread:0x000000009D6F17E8, state:B, prio=5
(native thread ID:0x444EA400, native priority:0x5,
native policy:UNKNOWN)
Blocked on: java/util/HashMap@0x000000008273FD30 Owned by:
"WebSphere WLM Dispatch Thread t=008b6528"
(J9VMThread:0x00000000400C3000,
java/lang/Thread:0x00000000992C5F38)
Java callstack:
at
com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.deregisterEnc
lave(EnclaveManager.java:247)
at
com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.leaveEnclave(
EnclaveManager.java:326)
at
com/ibm/ws/asynchbeans/services/wlm/EnclaveManager.unpinEnclave(
EnclaveManager.java:346)
(entered lock:
com/ibm/ws/asynchbeans/services/wlm/Enclave@0x000000009D74F590,
entry count: 1)
at
com/ibm/ws/asynchbeans/services/wlm/ClassificationService.push(C
lassificationService.java:282)
at
com/ibm/ws/asynchbeans/J2EEServiceManager.pushContext(J2EEServic
eManager.java:484(Compiled Code))
at
com/ibm/ws/asynchbeans/J2EEContext.push(J2EEContext.java:448)
at
com/ibm/ws/asynchbeans/J2EEContext.run(J2EEContext.java:1066)
at
com/ibm/ws/asynchbeans/WorkWithExecutionContextImpl.go(WorkWithE
xecutionContextImpl.java:199)
at
com/ibm/ws/asynchbeans/CJWorkItemImpl.run(CJWorkItemImpl.java:23
6(Compiled Code))
at
com/ibm/ws/asynchbeans/WLMCJWorkItemImpl.run(WLMCJWorkItemImpl.j
ava:69)

Problem conclusion

The WebSphere asynchbeans component has been corrected to
handle the deadlock by using proper synchronization blocks.

APAR PM91608 is currently targeted for inclusion in Fix Pack
8.5.5.2 of WebSphere Application Server V8.5.

Please refer to the Recommended Updates page for delivery
information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

In addition, please refer to URL:
http://www.ibm.com/support/docview.wss?rs=404&uid=swg27006970
for Fix Pack PTF information.

Temporary fix

Comments

APAR Information

APAR number
PM91608
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2013-06-21
Closed date
2013-09-06
Last modified date
2013-09-06

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500

Applicable component levels

R850 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"850","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
28 April 2022

Tips

PM91608: DEADLOCK IN COM/IBM/WS/ASYNCHBEANS/SERVICES/WLM/ENCLAVE

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R850 PSY

Document Information

Share your feedback

Need support?