PK92941: IF A FILESTORE GROWS PAST 2GB IT MIGHT BECOME CORRUPT, RESULTING LOST MESSAGES OR AN INABILITY TO RESTART

Fixes are available

APAR status

Closed as program error.

Error description

If the filestore for a WebSphere Application Server default
messaging provider messaging engine grows past 2GB, corruption
might occur resulting in lost messages.
The symptoms are unpredictable, due to the nature of the
corruption that occurs.

The issue might affect the TemporaryStore of the messaging
engine, causing nonpersistent messaging operations to fail
until the messaging engine is next restarted.

The issue might affect the PermanentStore of the messaging
engine, causing loss of persistent messages and possibly
preventing the messaging engine from restarting.

The most likely symptom is an FFDC as follows:
Source =
  com.ibm.ws.sib.processor.impl.store.items.MessageItem.restore
probeid = 1:447:1.210.1.10
The type of exception in the FFDC might vary, including:
java.lang.ArrayIndexOutOfBoundsException
java.nio.BufferUnderflowException

An FFDC as follows might also be seen, due to the same cause:
Source = com.ibm.ws.sib.msgstore.persistence.objectManager.
  PersistableImpl.getPersistedData
probeid = 1:167:1.16
com.ibm.ws.objectManager.PermanentIOException:
  CWSOM1004E: ObjectManager caught
  IOException=java.io.StreamCorruptedException:
  invalid stream header

Secondary symptoms might include the following log entries:
CWSIS1500E: The dispatcher cannot accept work.
Problem The dispatcher cannot accept more work due to an
earlier data store error.
CWSIK0027E: The destination with name {0} is corrupt
Problem The destination with the given name is corrupt and
cannot be used.

Local fix

A restart clears the issue until it happens again which may
happen in a matter of hours or may take several days before
the problem happens again.

Problem summary

****************************************************************
* USERS AFFECTED:  Users of the default messaging provider for *
*                  IBM WebSphere Application Server            *
*                  Version 7.0 or                              *
*                  Version 6.1 fix pack 6.1.0.25               *
****************************************************************
* PROBLEM DESCRIPTION: The filestore of a messaging engine     *
*                      might become corrupt if it grows        *
*                      beyond 2GB in size                      *
****************************************************************
* RECOMMENDATION:  By default a messaging engine is configured *
*                  with a maximum temporary store size of 500MB*
*                  and maximum permanent store size of 500MB.  *
*                  Messaging engines with this configuration   *
*                  cannot encounter this issue.                *
*                                                              *
*                  All users of the affected versions who      *
*                  have configured a file store with a         *
*                  maximum temporary store size, or permanent  *
*                  store size, of 2048 MB or greater should    *
*                  apply a fix for this APAR.                  *
****************************************************************
The code that that manages the allocation of space within the
file store contains an unsafe cast from a 64bit long to a
32bit int, inside a comparison routine.

In certain circumstances, this causes the data structures used
to maintain knowledge of the allocated and free space inside
the file store to become corrupt. The corruption only occurs
when there are two identically sized areas unallocated space
in the file store, that are separated in position by more than
2147483647 bytes.

If corruption of these data structures occurs, the same area
of storage might be allocated more than once, resulting in
corruption of data written to the file store.
This data corruption might result in a number of symptoms,
including exceptions thrown to applications, loss of persistent
messages, and an inability to restart a messaging engine.

In WebSphere Application Server Version 6.1, the issue only
occurs if APAR PK79872 is applied as part of fix pack 6.1.0.25.
This is because the fix for PK79872 in Version 6.1 included a
backport of the comparison routine containing the unsafe
cast from Version 7.0.

Problem conclusion

The fix for this APAR corrects the comparison routine.
This prevents the corruption from occurring.

If a file store has previously been corrupted as a result of
this APAR, and the messaging engine cannot start, it is
necessary to delete the Log, PermanentStore and TemporaryStore
files to allow the messaging engine to restart.
All persistent messages and state contained in the file store
are lost if these files are deleted.

The fix for this APAR is currently targeted for inclusion in
fix pack 6.1.0.27 and fix pack 7.0.0.7.
Refer to the Recommended Updates page for delivery information:
http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980

Temporary fix

Comments

APAR Information

APAR number
PK92941
Reported component name
PLAT MSG COM
Reported component ID
620600101
Reported release
200
Status
CLOSED PER
PE
NoPE
HIPER
NoHIPER
Special Attention
NoSpecatt
Submitted date
2009-08-03
Closed date
2009-08-26
Last modified date
2009-10-06

APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name
PLAT MSG COM
Fixed component ID
620600101

Applicable component levels

R200 PSY
UP

[{"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSEQTP","label":"WebSphere Application Server"},"Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"6.1","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

Modified date:
28 December 2021

Tips

PK92941: IF A FILESTORE GROWS PAST 2GB IT MIGHT BECOME CORRUPT, RESULTING LOST MESSAGES OR AN INABILITY TO RESTART

Fixes are available

Subscribe

APAR status

Closed as program error.

Error description

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

APAR Information

APAR number

Reported component name

Reported component ID

Reported release

Status

PE

HIPER

Special Attention

Submitted date

Closed date

Last modified date

APAR is sysrouted FROM one or more of the following:

APAR is sysrouted TO one or more of the following:

Fix information

Fixed component name

Fixed component ID

Applicable component levels

R200 PSY

Document Information

Share your feedback

Need support?