PI76281: CANCEL OF CICS HANGS. RO TCB WAITING FOR TRACE LOCK

Obtain the fix for this APAR.

APAR status

• Closed as program error.

Error description

• A CICS region hung after being cancelled. The Cancel hung
  because the RO TCB is waiting for the Trace lock but the owner
  of the lock, the QR TCB, has already been detached.
  .
  An IPCS SUMMARY FORMAT command shows that the RO TCB is holding
  up the cancel. The RO TCB is in an SVC1 WAIT out of DFHTRPT.
  The QR TCB has already terminated.
  .
  At the time of the Cancel request, the QR TCB was writing out a
  trace entry and was the owner of the trace lock. Because of the
  Cancel, the QR TCB gets interrupted with the 33E abend for
  DETACH processing. The CICS ESTAE does not handle the trace
  lock, and does not give control to DFHTRPT's recovery routine.
  So the trace lock is not released.
  .
  After the QR TCB has detached, its mother task, the RO TCB
  starts its terminate processing. The Kernel stacks for the RO's
  KTCB task look like this:
  .
  KE_NUM @STACK   LEN  TYPE ADDRESS  LINK REG OFFSET ERR NAME
  .
  0003   29B06040 0130 Bot  2996E318 A99707F0 0024D8     DFHKETCB
  0003   29B06170 0630 Dom  2996E318 A997023A 001F22     DFHKETCB
  0003   29B07740 0990 Dom  29A92368 A9A92582 00021A     DFHLDDM
  0003   29B080D0 0340 Dom  2994EAA8 2992CE5C 000000     DFHTRPT
  .
  Module DFHKETCB calls DFHLDDM for TERMINATE_DOMAIN. DFHLDDM
  calls DFHTRPT to trace entry to the Loader domain. But DFHTRPT
  put the RO TCB into a wait to wait for the trace lock but since
  the owner is the QR TCB who is already gone, the RO TCB remains
  in a wait. And so the Cancel of the CICS job hangs.
  .
  Additional symptoms / search keywords:
  KIXREVxxx  KTCB_UNTRUSTED_KTCB untrusted QEL LCB TRA_LOCK_BLOCK
  hang hangs
  

Local fix

Problem summary

• ****************************************************************
  * USERS AFFECTED: All CICS users.                              *
  ****************************************************************
  * PROBLEM DESCRIPTION: CICS system hangs during termination    *
  *                      after a CANCEL was issued against       *
  *                      the job.                                *
  ****************************************************************
  A CANCEL request was issued against a CICS job from the JES
  console. The CICS system starts to shut down with a S222 abend.
  All of the TCBs within the CICS system are abended (code 33E).
  .
  In the reported problem, DFHTRAP was active so any TCB writing
  a trace entry must first obtain the trace table lock. This is
  a CICS Kernel lock, held at the TCB level. The CICS QR TCB
  happened to be writing a trace entry, and hold this
  lock, at the point when it received the 33E abend.
  .
  The QR TCB was detached, but was still the owner of the
  trace lock. Later on during the CICS region shutdown,
  the RO TCB needed to obtain the trace lock, in order to write
  a trace entry. This caused the shutdown to hang.
  

Problem conclusion

• The CICS ESTAE DFHKESTX has been updated to ensure
  that the trace lock is released when a TCB shuts down.
  DFHKEDUF is also updated to format the untrusted
  kernel lock chain.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

  UI45009 UI45010 UI45011

Modules/Macros

• DFHKEDUF DFHKESTX
  

Fix information

• Fixed component name

  CICS TS Z/OS V5

• Fixed component ID

  5655Y0400

Applicable component levels

• R000 PSY UI45010

     UP17/02/25 P F702

• R800 PSY UI45011

     UP17/02/25 P F702

• R900 PSY UI45009

     UP17/02/25 P F702

Fix is available

• Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.