PI49810: Application fails to start after rolling update due to hung MessageReferenceHandler thread

Fixes are available

APAR status

Closed as program error.

Error description

The application fails to start during a rolling update.  The
output from deployment:

 10:15:41 [NORMAL] ->Entered function: startApplication()
10:15:41 [NORMAL] Thu Jul 17 07:17:89 2014
10:15:41 [NORMAL] Starting <appName>...
10:15:43 [NORMAL] ADMA5071I: Distribution status check started
for application <appName>
...
10:15:44 [NORMAL] ADMA5011I: The cleanup of the temp directory
for application <appName> is complete.
10:15:44 [NORMAL] Distribution status check completed for
application <appName>. result=false
10:15:44  <appName> is not ready to be started yet.
...
10:46:36 [NORMAL] (<jclass
com.ibm.ws.scripting.ScriptingException at -1507096197>,
com.ibm.ws.scripting.ScriptingException:
javax.management.MBeanException: Exception thrown in
RequiredModelMBean while trying to invoke operation
startApplication, <traceback object at 169029709>)

The MessageReferenceHandler is hung in the Control Region
Adjunct:

"WebSphere non-WLM Dispatch Thread t=00881be0"
J9VMThread:0x0000000044ABE700, j9thread_t:0x0000004809F68F80,
java/lang/Thread:0x00000000839253C8, state:B, prio=5
 (java/lang/Thread getId:0xBD, isDaemon:true)
 (native thread ID:0x4CC0A800, native priority:0x5, native
policy:UNKNOWN)
Blocked on: com/ibm/ws/wmqra/zdispatcher/
WASAdjunctMessageReferenceHandler@0 x00000000828219B8 Owned by:
"Jmqi AsyncConsume Thread."
(J9VMThread:0x0000000044B31300,
java/lang/Thread:0x00000000830DD708)
Heap bytes allocated since last GC cycle=0 (0x0)
Java callstack: at
com/ibm/mq/jms/MQMessageReferenceHandler.endDeliver
(MQMessageReferenceHandler.java:100)

When the endpoint for an MQ Activation Specification is being
shutdown, the MessageReferenceHandler thread is not freed in
the Control Region Adjunct.

Local fix

```
N/A
```

Problem summary

****************************************************************
* USERS AFFECTED:  WebSphere Application Server users of WMQ   *
*                  Activation Specifications.                  *
****************************************************************
* PROBLEM DESCRIPTION: Deadlock in the Control Region          *
*                      Adjunct process when rolling out        *
*                      updates to an application that uses     *
*                      WMQ activation specs.                   *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
Automatically rolling out updates to a high availability
application that uses a WMQ activation specification when using
the option to pause/resume the application server may result
in a deadlock in the control region adjunct process causing the
application restart to fail.

Problem conclusion

The messaging component was modified so that the throttling
mechanism for WMQ actication specifications running in the
control region adjunct process did not cause the deadlock.
This change in function is dependant on configuring the
inclusion of WMQ activation specifications among the listeners
being paused or resumed as part of automatically rolling out
updates to a high availability application provided in APAR
PI43141, enabled by the WebSphere environment variable
pause_listeners_include_cra.

APAR PI49810 is currently targeted for inclusion in Fix Packs
8.5.5.10 and 9.0.0.1 of WebSphere Application Server.

Please refer to the Recommended Updates page for delivery
information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

In addition, please refer to URL:
http://www.ibm.com/support/docview.wss?rs=404&uid=swg27006970
for Fix Pack PTF information.

Temporary fix

Comments

APAR Information

APAR number
PI49810
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt / Xsystem
Submitted date
2015-10-01
Closed date
2016-07-11
Last modified date
2016-07-11

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500

Applicable component levels

R850 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"850","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
04 May 2022

Tips

PI49810: Application fails to start after rolling update due to hung MessageReferenceHandler thread

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R850 PSY

Document Information

Share your feedback

Need support?