PK69110: THE SERVANT ABENDS CAUSED BY HUNG THREADS IN JDBC DRIVER.

Fixes are available

APAR status

Closed as program error.

Error description

The servant abends caused by hung threads in JDBC Driver.
After 5 minutes of seeing the hang, the control region (CR)
kills the servant.

The method com.ibm.ws.rsadapter.spi.WSManagedConnectionFactor
Impl$Equals.checkPrivateCredentials() in J2C component is
causing a deadlock.  This method checkPrivateCredentials() is
expected to be invoked in multi threaded environment.

A javacore taken at the time of the deadlock will show the
following:

1LKDEADLOCK    Deadlock detected !!!
NULL           ---------------------
NULL
2LKDEADLOCKTHR  Thread "ORB.thread.pool : 0"
(0x00000000412BCC00)
3LKDEADLOCKWTR    is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x00000000826641D0 infl_mon_t:
0x0000000082664240:
4LKDEADLOCKOBJ
java/util/Collections$SynchronizedSet@00000000031930E0/000000000
31930EC:
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "ORB.thread.pool : 4"
(0x0000000042945800)
3LKDEADLOCKWTR    which is waiting for:
4LKDEADLOCKMON      sys_mon_t:0x00000200121AC420 infl_mon_t:
0x00000200121AC490:
4LKDEADLOCKOBJ
java/util/Collections$SynchronizedSet@000000000329CFC8/000000000
329CFD4:
3LKDEADLOCKOWN    which is owned by:
2LKDEADLOCKTHR  Thread "ORB.thread.pool : 0"
(0x00000000412BCC00)


The stack traces of the two threads involved in the deadlock
show this:

"ORB.thread.pool : 0" TID:0x00000000412BCC00,
j9thread_t:0x00000200081FD4E0, state:B, prio=5
       (native thread ID:0x5B85, native priority:0x5, native
policy:UNKNOWN)
    at
java/util/Collections$SynchronizedCollection.size(Collections.ja
va:478(Compiled Code))
    at java/util/AbstractSet.equals(AbstractSet.java:51(Compiled
Code))
    at
java/util/Collections$SynchronizedSet.equals(Collections.java:83
1(Compiled Code))
    at
com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.c
heckPrivateCredentials(WSManagedConnectionFactoryImpl.java:3227(
Compiled Code))
    at
com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.r
un(WSManagedConnectionFactoryImpl.java:3187(Compiled Code))
    at
com/ibm/ws/security/util/AccessController.doPrivileged(AccessCon
troller.java:63(Compiled Code))
    at
com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl.matchMan
agedConnections(WSManagedConnectionFactoryImpl.java:2087(Compile
d Code))
    at
com/ibm/ejs/j2c/FreePool.getMCWrapperFromMatch(FreePool.java:179
8(Compiled Code))
    at
com/ibm/ejs/j2c/FreePool.getFreeConnection(FreePool.java:873(Com
piled Code))
    at
com/ibm/ejs/j2c/PoolManager.reserve(PoolManager.java:2153(Compil
ed Code))
...


"ORB.thread.pool : 4" TID:0x0000000042945800,
j9thread_t:0x00000200098C5CE0, state:B, prio=5
       (native thread ID:0x18ED, native priority:0x5, native
policy:UNKNOWN)
    at
java/util/Collections$SynchronizedCollection.size(Collections.ja
va:478(Compiled Code))
    at java/util/AbstractSet.equals(AbstractSet.java:51(Compiled
Code))
    at
java/util/Collections$SynchronizedSet.equals(Collections.java:83
1(Compiled Code))
    at
com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.c
heckPrivateCredentials(WSManagedConnectionFactoryImpl.java:3227(
Compiled Code))
    at
com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl$Equals.r
un(WSManagedConnectionFactoryImpl.java:3187(Compiled Code))
    at
com/ibm/ws/security/util/AccessController.doPrivileged(AccessCon
troller.java:63(Compiled Code))
    at
com/ibm/ws/rsadapter/spi/WSManagedConnectionFactoryImpl.matchMan
agedConnections(WSManagedConnectionFactoryImpl.java:2087(Compile
d Code))
    at
com/ibm/ejs/j2c/FreePool.getMCWrapperFromMatch(FreePool.java:179
8(Compiled Code))
    at
com/ibm/ejs/j2c/FreePool.getFreeConnection(FreePool.java:873(Com
piled Code))
    at
com/ibm/ejs/j2c/PoolManager.reserve(PoolManager.java:2153(Compil
ed Code))
...

Local fix

Problem summary

****************************************************************
* USERS AFFECTED:  WebSphere Application Server V6.1.0         *
*                  for z/OS & distributed platforms JCA        *
*                  and JDBC                                    *
****************************************************************
* PROBLEM DESCRIPTION: Selectable fix for servant abends       *
*                      caused by hung threads in WebSphere     *
*                      Application Server adapter component.   *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
Method com.ibm.ws.rsadapter.spi.WSManagedConnectionFactor
Impl$Equals.checkPrivateCredentials() or
checkPublicCrentials() in adapter component is
causing a deadlock.  If a customer has java cores that contain
this deadlock they need to enable this fix.  To enable
the fix, set custom property "useRRASetEquals" of type
java.lang.String to true on the datasource.

Problem conclusion

The JDK's implementation of Subject.equals() is
synchronized. This fix implements an unsynchronized equals.
Only enable this fix if the java cores contain a deadlock in
checkPrivateCredentials() or checkPublicCredentials().

If a customer has java cores that contain this deadlock they
need to enable this fix.  To enable the fix, set custom
property "useRRASetEquals" of type java.lang.String to true on
the datasource.

APAR PK69110 is currently targeted for inclusion in Service
Level (Fix Pack) 6.1.0.23 of WebSphere Application Server V6.1
for z/OS.

Please refer to URL:
//www.ibm.com/support/docview.wss?rs=404&uid=swg27006970
for Fix Pack availability.

Temporary fix

Comments

APAR Information

APAR number
PK69110
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
610
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2008-07-16
Closed date
2008-10-29
Last modified date
2012-05-29

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

PK75897

Fix information

Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500

Applicable component levels

R610 PSY UK44403
UP09/03/21 P F903

Fix is available

Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"6.1","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
29 December 2021

Tips

PK69110: THE SERVANT ABENDS CAUSED BY HUNG THREADS IN JDBC DRIVER.

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R610 PSY UK44403

Fix is available

Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.

Document Information

Share your feedback

Need support?