PH05716: MIXED TRANSACTION OUTCOME POSSIBLE IF LOGS STORED IN A DATABASE ARE RECLAIMED DURING TIMING WINDOW USING

9.0.5.0: WebSphere Application Server traditional Version 9.0.5 Refresh Pack
9.0.5.1: WebSphere Application Server traditional Version 9.0.5 Fix Pack 1
9.0.5.2: WebSphere Application Server traditional Version 9.0.5 Fix Pack 2
8.5.5.17: WebSphere Application Server V8.5.5 Fix Pack 17
9.0.5.3: WebSphere Application Server traditional Version 9.0.5 Fix Pack 3
8.5.5.20: WebSphere Application Server V8.5.5.20
8.5.5.18: WebSphere Application Server V8.5.5 Fix Pack 18
8.5.5.19: WebSphere Application Server V8.5.5 Fix Pack 19
8.5.5.16: WebSphere Application Server V8.5.5 Fix Pack 16
8.5.5.21: WebSphere Application Server V8.5.5.21

APAR status

• Closed as program error.

Error description

• An application server that propagates transactions to a
  backend application server running in a cluster with
  transactional high availability enabled and the transaction
  logs stored in a relational database receives repeated
  failures to commit transactions when invoking the commit
  method on the backend server's subordinate transaction
  coordinator.
  
  The front end application server receives the exception
  org.omg.CORBA.OBJECT_NOT_EXIST with minor code 0.
  
  The callstack from the backend server is of the form:
  
  >> SERVER (id=abababab, host=somehost) TRACE START:
  >>    org.omg.CORBA.OBJECT_NOT_EXIST:   vmcid: 0x0  minor
  code: 0  completed: No
  >>  at
  com.ibm.ws.Transaction.JTS.JTSServantManager.keyToObject(JTSServ
  antManager.java:283)
  >>  at com.ibm.ejs.oa.EJSOAImpl.keyToObject(EJSOAImpl.java:553)
  >>  at
  com.ibm.ejs.oa.EJSRootOAImpl.keyToObject(EJSRootOAImpl.java:277)
  >>  at
  com.ibm.rmi.corba.ObjectManager.lookupServant(ObjectManager.java
  :104)
  >>  at
  com.ibm.CORBA.iiop.ServerDelegate.getServantForRequest(ServerDel
  egate.java:394)
  >>  at
  com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:4
  61)
  >>  at com.ibm.rmi.iiop.ORB.process(ORB.java:616)
  ...
  
  The front end application server will report a
  HeuristicMixedException to the application.
  
  Although the transaction committed, the subordinate branch in
  the backend server rolled back and a mixed transactional
  outcome occurred.
  
  Investigation shows that:
  1 - The backend server had previously started to recover a
  peer server's transaction logs (stored in a database) and
  generated a CWRLS0013I message indicating this.
  2 - The peer had later reclaimed its transaction logs
  3 - The backend server had not generated a CWRLS0014I message
  to indicate that it had finished processing the peer's
  transaction logs.
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED:  WebSphere Application Server users of       *
  *                  transactional high availability             *
  ****************************************************************
  * PROBLEM DESCRIPTION: Transactions repeatedly fail to commit  *
  *                      with OBJECT_NOT_EXIST minor code 0.     *
  *                      Transaction outcome is mixed.           *
  ****************************************************************
  * RECOMMENDATION:                                              *
  ****************************************************************
  A timing window existed when using transactional high
  availability with the transaction service logs stored in a
  relational database whereby an application server performing
  peer recovery would not correctly clear up state when handing
  back the logs.
  This resulted in WLM misrouting requests for the second phase
  of the transaction 2PC protocol (ie commit) to the
  application server that were intended for the peer application
  server, the prepare phase having been routed correctly.
  More importantly, it also resulted in the incorrect processing
  of those requests resulting in a
  org.omg.CORBA.OBJECT_NOT_EXIST Exception with minor code 0
  being thrown.  This Exception and minor code indicates that the
  target transaction does not exist (as opposed to indicating
  that the transaction was not located in the target runtime).
  This results in a javax.transaction.HeuristicMixedException in
  the front end application server and the transaction is
  forgotten.  Later when the subordinate transaction, which is
  in-doubt, attempts to determine the transaction outcome via
  replay_completion it is told that the transaction does not
  exist and therefore, in accordance with the standard 2PC
  presumed abort optimization, the subordinate transaction
  branch is rolled back.
  

Problem conclusion

• The synchronization of the transaction service code was
  modified so that the timing window did not occur and any state
  associated with peer recovery, including WLM routing for the
  transaction service endpoints, was correctly cleaned up.  The
  code that processed inbound protocol requests was also
  modified to perform additional checks which would result in a
  org.omg.CORBA.TRANSIENT being thrown in certain circumstances.
  
  
  The fix for this APAR is currently targeted for inclusion in
  fix packs 8.5.5.16 and 9.0.5.0.  Please refer to the
  Recommended Updates page for delivery information:
  http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WEBS APP SERV N

• Fixed component ID

  5724H8800

Applicable component levels

• R850 PSY

     UP

• R900 PSY

     UP