PI79261: Deadlock with persistent EJB timers for Singleton beans

Fixes are available

APAR status

Closed as program error.

Error description

If a persistent EJB timer for a Singleton EJB expires at the
same time that a local or remote method is being called on
that Singleton and the method attempts to access EJB timers
by calling a method on the EJB TimerService, then it is
possible the two threads deadlock.  This occurs because the
thread that is running the persistent EJB timer will obtain
a lock on the timer database and then a lock on the
Singleton bean, whereas the thread running the local or
remote method will first obtain the lock on the Singleton
bean and then the lock on the timer database.

When this occurs, the stack of the EJB timer thread will
contain the following:

com.ibm.ejs.container.SingletonBeanO.preInvoke(SingletonBean
O.java:526)
com.ibm.ejs.container.EJSContainer.preInvokeActivate(EJSCont
ainer.java:3264)
com.ibm.ejs.container.EJSContainer.EjbPreInvoke(EJSContainer
.java:2576)
com.ibm.ejs.container.TimedObjectWrapper.invokeCallback(Time
dObjectWrapper.java:84)
com.ibm.ejs.container.PersistentTimerTaskHandler.run(Persist
entTimerTaskHandler.java:290)
com.ibm.ws.concurrent.persistent.internal.InvokerTask.run(In
vokerTask.java:426)

And the stack of the local or remote method call will look
something like :

com.ibm.ws.concurrent.persistent.internal.PersistentExecutor
Impl.remove(PersistentExecutorImpl.java:1243)
com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.EJBPe
rsistentTimerRuntimeImpl.remove(EJBPersistentTimerRuntimeImp
l.java:352)
com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.Persi
stentTimerImpl.cancel(PersistentTimerImpl.java:114)
ejb.AutoTimer.cancel(AutoTimer.java:29)
ejb.EJSLocalNSGAutoTimer_0535901b.cancel(EJSLocalNSGAutoTime
r_0535901b.java)

Local fix

Problem summary

****************************************************************
* USERS AFFECTED:  All users of IBM WebSphere Application      *
*                  Server Liberty - Enterprise Java Beans      *
*                  (EJB)                                       *
****************************************************************
* PROBLEM DESCRIPTION: Deadlock with persistent EJB timers for *
*                      Singleton beans                         *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
If a persistent EJB timer for a Singleton EJB expires at the
same time that a local or remote method is being called on that
Singleton and the method attempts to access EJB timers by
calling a method on the EJB TimerService, then it is possible
the two threads deadlock.  This occurs because the thread that
is running the persistent EJB timer will obtain a lock on the
timer database and then a lock on the Singleton bean, whereas
the thread running the local or remote method will first obtain
the lock on the Singleton bean and then the lock on the timer
database.

When this occurs, the stack of the EJB timer thread will contain
the following:

com.ibm.ejs.container.SingletonBeanO.preInvoke(SingletonBeanO.ja
va:526)
com.ibm.ejs.container.EJSContainer.preInvokeActivate(EJSContaine
r.java:3264)
com.ibm.ejs.container.EJSContainer.EjbPreInvoke(EJSContainer.jav
a:2576)
com.ibm.ejs.container.TimedObjectWrapper.invokeCallback(TimedObj
ectWrapper.java:84)
com.ibm.ejs.container.PersistentTimerTaskHandler.run(PersistentT
imerTaskHandler.java:290)
com.ibm.ws.concurrent.persistent.internal.InvokerTask.run(Invoke
rTask.java:426)

And the stack of the local or remote method call will look
something like :

com.ibm.ws.concurrent.persistent.internal.PersistentExecutorImpl
.remove(PersistentExecutorImpl.java:1243)
com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.EJBPersis
tentTimerRuntimeImpl.remove(EJBPersistentTimerRuntimeImpl.java:3
52)
com.ibm.ws.ejbcontainer.timer.persistent.osgi.internal.Persisten
tTimerImpl.cancel(PersistentTimerImpl.java:114)
ejb.AutoTimer.cancel(AutoTimer.java:29)
ejb.EJSLocalNSGAutoTimer_0535901b.cancel(EJSLocalNSGAutoTimer_05
35901b.java)

Problem conclusion

The deadlock scenario has been resolved by changing the order
that locks are obtained for the timer expiration.  When an EJB
timer for a singleton bean expires, the lock on the Singleton
bean will now be acquired before the lock on the timer database.
Since the order the locks are obtained is now the same as when a
local or remote method is called, the deadlock can no longer
occur.

The fix for this APAR is currently targeted for inclusion in fix
pack 17.0.0.2.  Please refer to the Recommended Updates page for
delivery information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

Temporary fix

Comments

APAR Information

APAR number
PI79261
Reported component name
WAS LIBERTY COR
Reported component ID
5725L2900
Reported release
CD0
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2017-04-03
Closed date
2017-04-12
Last modified date
2017-04-12

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
WAS LIBERTY COR
Fixed component ID
5725L2900

Applicable component levels

RCD0 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSD28V","label":"WebSphere Application Server Liberty Core"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"CD0","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
03 May 2022

Tips

PI79261: Deadlock with persistent EJB timers for Singleton beans

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

RCD0 PSY

Document Information

Share your feedback

Need support?