PI85490: DEADLOCK CAUSED BY WSLOGMANAGER AND SIB TRACE CODE

Fixes are available

APAR status

Closed as program error.

Error description

When setting trace at runtime, a deadlock might happen with the
following thread stack in the java dump file, indicating a lock
contention when WsLogManager tries to set trace at runtime
while SIB debug code is locking the code.
1LKDEADLOCK    Deadlock detected !!!
NULL           ---------------------
NULL
2LKDEADLOCKTHR  Thread "Default : 8" (0x0000000052C04700)
3LKDEADLOCKWTR    is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x00000100198EA4F8 infl_mon_t:
0x00000100198EA540:
4LKDEADLOCKOBJ
com/ibm/ws/wmqcsi/trace/TraceImpl@0x00000000414F1090
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "Thread-161" (0x0000000052C7E000)
3LKDEADLOCKWTR    which is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x000001001963C3D8 infl_mon_t:
0x000001001963C420:
4LKDEADLOCKOBJ
com/ibm/ws/bootstrap/WsLogManager@0x0000000040DDDB50
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "Default : 8" (0x0000000052C04700)

3XMTHREADINFO      "Default : 8" J9VMThread:0x0000000052C04700,
j9thread_t:0x000001001830FAA0,
java/lang/Thread:0x00000000438BA838,
state:B, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x9E,
isDaemon:true)
3XMTHREADINFO1            (native thread ID:0x5470091, native
priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread
flags:0x00000201)
3XMCPUTIME               CPU usage total: 1.024461000 secs,
user:
0.638929000 secs, system: 0.385532000 secs
3XMTHREADBLOCK     Blocked on:
com/ibm/ws/wmqcsi/trace/TraceImpl@0x00000000414F1090 Owned by:
"Thread-161" (J9VMThread:0x0000000052C7E000,
java/lang/Thread:0x00000000438906E0)
3XMHEAPALLOC             Heap bytes allocated since last GC
cycle=0
(0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at
com/ibm/msg/client/commonservices/trace/Trace.setOn(Trace.java:5
15)
5XESTACKTRACE                   (entered lock:
com/ibm/msg/client/commonservices/trace/Trace@0x0000000040D58A20
, entry
count: 1)
4XESTACKTRACE                at
com/ibm/ws/wmqcsi/trace/TraceImpl$1.traceStateChanged(TraceImpl.
java:232
)
4XESTACKTRACE                at
com/ibm/ejs/ras/TraceComponent.applyTraceEnableEvent(TraceCompon
ent.java
:163(Compiled Code))
4XESTACKTRACE                at
com/ibm/ejs/ras/Group.applyTraceEnableEvent(Group.java:112(Compi
led
Code))
4XESTACKTRACE                at
com/ibm/ejs/ras/ComponentManager.fireTraceEnableEvent(ComponentM
anager.j
ava:786)
4XESTACKTRACE                at
com/ibm/ejs/ras/ComponentManager.processLoggingSpecString(Compon
entManag
er.java:1125)
4XESTACKTRACE                at
com/ibm/ejs/ras/ComponentManager.processTraceString(ComponentMan
ager.jav
a:380)

3XMTHREADINFO      "Thread-161" J9VMThread:0x0000000052C7E000,
j9thread_t:0x000001001556CEA0,
java/lang/Thread:0x00000000438906E0,
state:B, prio=5
3XMJAVALTHREAD            (java/lang/Thread getId:0x10D,
isDaemon:true)
3XMTHREADINFO1            (native thread ID:0x6EE00A1, native
priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread
flags:0x00000201)
3XMCPUTIME               CPU usage total: 24.078727000 secs,
user:
19.058190000 secs, system: 5.020537000 secs
3XMTHREADBLOCK     Blocked on:
com/ibm/ws/bootstrap/WsLogManager@0x0000000040DDDB50 Owned by:
"Default
: 8" (J9VMThread:0x0000000052C04700,
java/lang/Thread:0x00000000438BA838)
3XMHEAPALLOC             Heap bytes allocated since last GC
cycle=0
(0x0)
3XMTHREADINFO3           Java callstack:
4XESTACKTRACE                at
java/util/logging/LogManager.initializeGlobalHandlers(LogManager
.java:14
07(Compiled Code))
5XESTACKTRACE                   (entered lock:
com/ibm/ws/bootstrap/WsLogManager@0x0000000040DDDB50, entry
count: 1)
4XESTACKTRACE                at
java/util/logging/LogManager.access$1500(LogManager.java:160(Com
piled
Code))
4XESTACKTRACE                at
java/util/logging/LogManager$RootLogger.accessCheckedHandlers(Lo
gManager
.java:1506(Compiled Code))
4XESTACKTRACE                at
java/util/logging/Logger.getHandlers(Logger.java:1362(Compiled
Code))
4XESTACKTRACE                at
java/util/logging/Logger.log(Logger.java:624(Compiled Code))
4XESTACKTRACE                at
com/ibm/ejs/ras/Tr.logToJSR47Logger(Tr.java:1720(Compiled
Code))
4XESTACKTRACE                at
com/ibm/ejs/ras/Tr.fireEvent(Tr.java:1675(Compiled Code))
4XESTACKTRACE                at
com/ibm/ejs/ras/Tr.fireTraceEvent(Tr.java:1594(Compiled Code))
4XESTACKTRACE                at
com/ibm/ejs/ras/Tr.debug(Tr.java:651(Compiled Code))

See Liberty issue:
https://github.ibm.com/was-liberty/WS-CD-Open/issues/7279
Se Liberty pull request:
https://github.ibm.com/was-liberty/WS-CD-Open/pull/7280

Local fix

```
N/A
```

Problem summary

****************************************************************
* USERS AFFECTED:  All users of IBM WebSphere Application      *
*                  Server who enable MQ Resource adapter       *
*                  tracing.                                    *
*                  Also fixed in Liberty 17.0.0.3              *
****************************************************************
* PROBLEM DESCRIPTION: Deadlock between WslogManager and Sib   *
*                      MQ                                      *
*                      client trace.                           *
****************************************************************
* RECOMMENDATION:  This is hard to reproduce and only rarely   *
*                  occurs,                                     *
*                  if necessary apply the fix.                 *
****************************************************************
A dump will show a server thread locked at
com/ibm/msg/client/commonservices/trace/Trace.setOn(Trace.java
:5
15)

Problem conclusion

The fix for this APAR is currently targeted for inclusion in
fix pack 7.0.0.45, 8.0.0.15, 8.5.5.13, and 9.0.0.6.

Please refer to the Recommended Updates page for delivery
information:
http://www.ibm.com/support/docview.wss?
rs=180&uid=swg27004980

Temporary fix

Comments

APAR Information

APAR number
PI85490
Reported component name
WEBS APP SERV N
Reported component ID
5724H8800
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2017-08-03
Closed date
2017-09-26
Last modified date
2017-10-10

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
WEBS APP SERV N
Fixed component ID
5724H8800

Applicable component levels

R700 PSY
UP
R800 PSY
UP
R850 PSY
UP
R900 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"8.5","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
04 May 2022

Tips

PI85490: DEADLOCK CAUSED BY WSLOGMANAGER AND SIB TRACE CODE

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R700 PSY

R800 PSY

R850 PSY

R900 PSY

Document Information

Share your feedback

Need support?