Fixes are available
8.5.5.4: WebSphere Application Server V8.5.5 Fix Pack 4
8.5.5.5: WebSphere Application Server V8.5.5 Fix Pack 5
8.5.5.6: WebSphere Application Server V8.5.5 Fix Pack 6
8.5.5.7: WebSphere Application Server V8.5.5 Fix Pack 7
8.5.5.8: WebSphere Application Server V8.5.5 Fix Pack 8
8.5.5.9: WebSphere Application Server V8.5.5 Fix Pack 9
8.5.5.10: WebSphere Application Server V8.5.5 Fix Pack 10
8.5.5.11: WebSphere Application Server V8.5.5 Fix Pack 11
8.5.5.12: WebSphere Application Server V8.5.5 Fix Pack 12
8.5.5.13: WebSphere Application Server V8.5.5 Fix Pack 13
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14
8.5.5.15: WebSphere Application Server V8.5.5 Fix Pack 15
8.5.5.14: WebSphere Application Server V8.5.5 Fix Pack 14
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21
APAR status
Closed as program error.
Error description
Running with one Servant Region with the autopause listeners option (control_region_dreg_on_no_srs=1), if a timeout happens causing the Servant Region to be abended, the new Servant Region that starts may not completely initialize. The Control Region needed to be cancelled for the server to come back up. The Control Region experiences a deadlock scenario between two threads, the stack and state of the threads is as follows: Thread A -------- name: WebSphere t=00ac13b0 Thread object: java/lang/Thread @ 0x820011c0 Daemon: true ID: 105 (0x69) Priority: 5 Thread.State: BLOCKED JVMTI state: ALIVE BLOCKED_ON_MONITOR_ENTER waiting to enter: com/ibm/ejs/jms/listener/WS390MessageReferenceHandler@0x85051680 owner name: "Jmqi AsyncConsume Thread." owner id: 0x4128c700 Java stack frames: com/ibm/ejs/jms/listener/WS390MessageReferenceHandler.endDeliver () source: WS390MessageReferenceHandler.java:<data unavailable> com/ibm/msg/client/jms/internal/JmsConnectionBrowserImpl.stop() source: JmsConnectionBrowserImpl.java:190 com/ibm/msg/client/jms/internal/JmsConnectionBrowserImpl.close(b oolean) source: JmsConnectionBrowserImpl.java:248 com/ibm/msg/client/jms/internal/JmsConnectionBrowserImpl.close() source: JmsConnectionBrowserImpl.java:206 com/ibm/mq/jms/MQConnectionBrowser.close() source: MQConnectionBrowser.java:84 com/ibm/ejs/jms/listener/WS390ConnectionBrowserWrapper.internalS top() source: WS390ConnectionBrowserWrapper.java:158 com/ibm/ejs/jms/listener/WS390ConnectionBrowserWrapper.close() source: WS390ConnectionBrowserWrapper.java:209 com/ibm/ejs/jms/listener/WS390ConnectionBrowserManager.remove(St ring) source: WS390ConnectionBrowserManager.java:425 com/ibm/ejs/jms/listener/WS390ConnectionBrowserManager.removeBro wsers(String) source: WS390ConnectionBrowserManager.java:520 sun/reflect/NativeMethodAccessorImpl.invoke0(reflect.Method, Object, Object[]) (Native Method) sun/reflect/NativeMethodAccessorImpl.invoke(Object, Object[]) source: NativeMethodAccessorImpl.java:60 sun/reflect/DelegatingMethodAccessorImpl.invoke(Object, Object[]) source: DelegatingMethodAccessorImpl.java:37 java/lang/reflect/Method.invoke(Object, Object[]) source: Method.java:611 com/ibm/ejs/jms/listener/ListenerPortMBeanProxy.stop() source: ListenerPortMBeanProxy.java:365 sun/reflect/NativeMethodAccessorImpl.invoke0(reflect.Method, Object, Object[]) (Native Method) ... Thread B -------- name: Jmqi AsyncConsume Thread. Thread object: java/lang/Thread @ 0x850122d0 Daemon: true ID: 449 (0x1c1) Priority: 5 Thread.State: WAITING JVMTI state: ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT waiting to be notified on: com/ibm/ejs/jms/listener/MessageReferenceListenerPort@0x83683d20 owner name: <unowned> Java stack frames: java/lang/Object.wait(long, int) (Native Method) java/lang/Object.wait() source: Object.java:167 com/ibm/ejs/jms/listener/MessageReferenceListenerPort.checkHighT hreshold() source: MessageReferenceListenerPort.java:343 com/ibm/ejs/jms/listener/WS390MessageReferenceHandler.handleMess ageReference(com.ibm.mq.jms.MessageReference) source: WS390MessageReferenceHandler.java:100 com/ibm/mq/jms/MQMessageReferenceHandler.handleMessageReference( com.ibm.msg.client.jms.JmsMessageReference) source: MQMessageReferenceHandler.java:85 com/ibm/msg/client/jms/internal/JmsConnectionImpl$JmsProviderMes sageRefHandler.handleMessageReference(com.ibm.msg.client.provide r.ProviderMessageReference) source: JmsConnectionImpl.java:1491 com/ibm/msg/client/wmq/internal/WMQConnectionBrowser$WMQConnecti onBrowserShadow.consumer(com.ibm.mq.jmqi.handles.Hconn, com.ibm.mq.jmqi.MQMD, com.ibm.mq.jmqi.MQGMO, java.nio.ByteBuffer, com.ibm.mq.jmqi.MQCBC) source: WMQConnectionBrowser.java:863 com/ibm/mq/jmqi/local/internal/LocalProxyConsumer.jmqiConsumerMe thod(byte[], byte[], byte[], byte[]) source: LocalProxyConsumer.java:222
Local fix
stop the Control Region and restart the server
Problem summary
**************************************************************** * USERS AFFECTED: All users of IBM WebSphere Application * * Server V8.5 who have the autopause * * listeners option set * * (control_region_dreg_on_no_srs=1) * **************************************************************** * PROBLEM DESCRIPTION: Following an abend in the Servant * * region, it fails to fully * * initialize when restarted due to a * * deadlock in the control region. * **************************************************************** * RECOMMENDATION: * **************************************************************** The new Servant region becomes stuck initialization waiting for an async request to complete in the Control region. The async request is unable to complete because it requires a lock that is currently being held by a thread that is attempting to stop a listener port. This is a deadlock scenario, the causing threads and the top of their stacks can be seen in the error description.
Problem conclusion
The code has been altered to ensure that this deadlock will not occurr. APAR PI19787 is currently targeted for inclusion in Fix Pack 8.5.5.4 of WebSphere Application Server V8.5. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980 In addition, please refer to URL: http://www.ibm.com/support/docview.wss?rs=404&uid=swg27006970 for Fix Pack PTF information.
Temporary fix
Comments
APAR Information
APAR number
PI19787
Reported component name
WEBSPHERE FOR Z
Reported component ID
5655I3500
Reported release
850
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2014-06-11
Closed date
2014-09-12
Last modified date
2014-09-12
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
Fix information
Fixed component name
WEBSPHERE FOR Z
Fixed component ID
5655I3500
Applicable component levels
R850 PSY
UP
[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SS7K4U","label":"WebSphere Application Server for z\/OS"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"850","Line of Business":{"code":"LOB45","label":"Automation"}}]
Document Information
Modified date:
27 April 2022