A fix is available
APAR status
Closed as program error.
Error description
When executing Dynamic Program Links (DPLs) which cause WLM Locked affinities to be created, some affinities may remain active after the unit of work (UOW) for which the affinity was created has ended. This can eventually lead to CMAS termination due to the WLM dataspace being over allocated with orphaned affinity elements.
Local fix
Problem summary
**************************************************************** * USERS AFFECTED: All CICSPlex SM V4R2M0 Users * **************************************************************** * PROBLEM DESCRIPTION: Various errors may result in CPSM WLM * * affinities not being removed properly. * * This can eventually lead to CMAS * * termination due to the WLM dataspace * * being over allocated with orphaned * * affinity elements. * * * * * * - When executing transactions or * * Dynamic Program links (DPLs) which * * are routed through CPSM WLM and * * cause affinities with a lifetime of * * Signon to be created, the affinities * * will remain active after the user * * for which the affinities were * * created signs off. * * * * * * - When executing Dynamic Program Links * * (DPLs) which are routed through CPSM * * WLM and cause Locked affinities to * * be created, affinities may remain * * active after the units of work * * (UOWs) for which the affinities were * * created have ended. * * * * These orphaned affinities may occur * * in situations which may include, but * * are not limited to: * * * * - when the DPL is issued through a * * pre CICS 670 routing region * * (TOR). * * * * - when many Locked affinities are * * generated within a short time * * period (less than 1 second) and * * the CICS Unit-of-Work identifier * * embedded in the Locked affinity * * key contains the hex characters * * '4E', '5C' or '6F'. * * * * - when the routed transaction that * * executes the remote program (or * * one of its subordinates) abends, * * causing a "Route Abend" * * notification back in the routing * * region (TOR). * * * * - when the routed transaction that * * is to execute the remote program * * cannot be started at the selected * * target region (AOR), causing a * * "Route Error" notification back * * in the routing region (TOR). * * * * In addition to orphaned Locked * * affinities occurring, the CICS trace * * dataset in the TOR may contain * * CICSplex SM exception trace entries, * * issued by modules WDTR and WDIN: * * * * Method Caller TPID Debug text * * ------ ------ ---- ---------- * * WDIN WDTR 26 FTABLOCI * * WDIN WDTR 33 WFMGXCLR * * WDTR XLOP 6 WDININIT * **************************************************************** * RECOMMENDATION: After applying the PTF that resolves this * * APAR, all CMASes and MASes defined as CPSM * * WLM routing regions must be recycled to pick * * up the new code. The restarts need not be * * performed at the same time, however until a * * CMAS and its connected routing regions are * * restarted with the updated code, the problem * * may continue to occur. * * * * It is not necessary to restart any AORs or * * CPSM Workloads to correct the problem. * * However, if any workloads that report * * orphaned affinities are not restarted, then * * those orphaned affinities should be manually * * removed using the CICSplex SM WUI "Active * * workload transaction group affinities" view * * to FORCE remove them. * **************************************************************** - When a user signs off while having active affinities with a lifetime of Signon, method EYU0WMTA (WMTA) is called to remove the affinities. A logic error in WMTA results in the affinities not being removed. - Locked affinities were introduced in CTS 4.2 to ensure that consecutive Dynamic Program Link requests within the same CICS unit of work return to the same target region for execution. - Locked affinities require that the TOR be running CICS 670. A logic error in method EYU0WTCL (WTCL), allows a Locked affinity to be created for a TOR running a release of CICS prior to 670. Since the CICS region itself does not support Locked affinities, the affinity is invalidly built with null values for the CICS UOWs, and the affinity is not removed when the unit of work ends. - Historically, CPSM Affinities have been based on character key data (User Ids, terminal Ids, etc.). However, Locked affinities are keyed on CICS UOW ids, which are hexadecimal values. This is not in itself a functional problem, until a TOR needs to scan the active affinities list for any affinities associated with its completing unit of work. CPSM cache list searches are predicated on character data, and have a built in capability to detect the generic search characters '+', '*' and '?'. These are represented by X'4E', X'5C' and X'6F', respectively, in hexadecimal. A CICS Unit of Work ID may contain these hexadecimal characters. A new WUOW list of active Locked affinities is anchored in the WLM TOR descriptor. This list is scanned when a UOW completes to find the Locked affinity which must be terminated. If the UOW key contains one of these characters, then the WUOW scan may remove more elements from the WUOW list than is actually required. Whilst the completing UOW will have its Locked affinity terminated, the removal of additional WUOW elements means that when those UOWs complete, the TOR will have no cross-reference to locate their Locked affinities. Thus, orphaned Locked affinity elements are displayed in the WLMATAFF list. These Locked orphans may subsequently be removed using the FORCE command against the affinity element. - When a route request that causes a Locked affinity to be created abends, a Route Abend (RTABND) is returned to the CPSM WLM agent in the TOR, at which point the newly created affinity must be removed. RTABND processing in the TOR is insufficient for dealing with the removal of Locked affinities. CPSM exception traces are also issued because the TOR UOW completion call was not locating the WUOW element properly. - When an AOR fails to launch a DPL transaction successfully, then the TOR is called with a Route Error (RTERR) function code to indicate that WLM should select a different AOR to route to. If a Locked affinity was created for the failed route, the TOR must remove the affinity and create a new one for the new AOR the request will be routed to. This is not being done by RTERR, resulting in the original affinity not being released, and a failure to build the new affinity for the re-selected AOR. CPSM exception traces were also issued because the TOR UOW completion call was unable to locate the expected WUOW element.
Problem conclusion
- Method EYU0WMTA (WMTA) has been updated to ensure that any active affinities with a lifetime of Signon are removed when the user associated with the affinities signs off. - The following updates have been made to address the errors with Locked affinities described above: - Method EYU0WTCL (WTCL) has been changed to check the CICS release of the TOR when a DPL route should result in a Locked affinity being created. If the CICS release is 670, the affinity is created. If the CICS release is prior to 670, the affinity is not created. - Method EYU0WFMG (WFMG) is the general purpose WLM Affinity Services processor for the creation and removal of affinity elements associated with workloads. It is executable in both CMAS and TOR environments. The Locked affinity removal process has been modified to search UOW elements based upon a new WUOW alternate index list, rather than the primary key list. This will insulate the code from inappropriately removing list elements unrelated to the current request due to the existence of X'4E', X'5C' or X'6F' characters in the UOW Id fields in the WUOW key. A modification was also made to ensure that the correct transaction group name is applied to the affinity processing subroutine. - Method EYU0WNWK (WNWK) is executed in a MAS as part of the TOR initialisation process of a workload. It is driven to register the workload components required within the TOR itself. It has been modified to build the new WUOW alternate index list through a call to XCLX. - Method EYU0XCLX (XCLX) is called to build an alternate index against a previously defined search list. The validation process of the Alternate index key location and length has been rewritten to allow alternate index keys to overlay the search list primary key. - Method EYU0WDI8 (WDI8) is called to normalise the CICS 670 dynamic routing commarea (DFHDYPDS) into a CICS release independent workarea (EYURWTDA) for subsequent dynamic route selection processing by WLM. When WDI8 is called to process a Route Abend function, then the DYRUOWAF flag is set to CALL_NO to instruct CICS that a call back is no longer required at Unit-of-work completion. - Method EYU0WDIN (WDIN) is the Dynamic Routing request initialisation processor. It has been modified to take the necessary corrective actions to the Route Abend and Route Error processing routines concerned with Locked affinities. In addition, the End-of-UOW process has been updated to issue a Level 31 soft error trace if the terminating UOW does not have an expected WUOW element - probably because Route Abend processing has already removed and processed it. The affinity deletion process within WDIN has been upgraded to ensure that any outstanding WUOW elements are removed (from the list anchored in the WTOR block) after the affinity service method (WFMG) has been called to remove the Locked affinity associated with the current WLM transaction group. If the key of the element removed from the list does not match the WUOW key of the current item, then a new exception trace is issued. - Method EYU0WDAF (WDAF) is the WLM Service module for processing create-affinity and delete-affinity requests for the default internal WRAM program embedded into the DTR mainline program (EYU0WDTR). The Delete_affinity process starts by checking the affinity state flag in the TDA and returning an error to the WDTR mainline if it is not active. This test has been modified to allow an inactive state flag to be an acceptable condition for an End-of-UOW service call from CICS, in which case various WRAM commarea fields are nullified, and a normal return is made to the DTR mainline program.
Temporary fix
FIX AVAILABLE BY PTF ONLY
Comments
APAR Information
APAR number
PM51685
Reported component name
CICS TS Z/OS V4
Reported component ID
5655S9700
Reported release
70M
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2011-11-07
Closed date
2012-02-17
Last modified date
2012-03-01
APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:
UK76286
Modules/Macros
CJA0WNWK CJB0WNWK CJC0WNWK EYU0UCLX EYU0WDAF EYU0WDIN EYU0WDI8 EYU0WFMG EYU0WMTA EYU0WMTJ EYU0WNWK EYU0WTCL EYU0XCLX
Fix information
Fixed component name
CICS TS Z/OS V4
Fixed component ID
5655S9700
Applicable component levels
R70M PSY UK76286
UP12/02/18 P F202
Fix is available
Select the PTF appropriate for your component level. You will be required to sign in. Distribution on physical media is not available in all countries.
[{"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSGMGV","label":"CICS Transaction Server"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"4.2","Edition":"","Line of Business":{"code":"LOB35","label":"Mainframe SW"}},{"Business Unit":{"code":"BU054","label":"Systems w\/TPS"},"Product":{"code":"SG19M","label":"APARs - z\/OS environment"},"Component":"","ARM Category":[],"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"4.2","Edition":"","Line of Business":{"code":"","label":""}}]
Document Information
Modified date:
01 March 2012