Document ID: 300765
http://support.veritas.com/docs/300765
E-Mail this document to a colleague
http://support.veritas.com/docs/300765
E-Mail this document to a colleague
Intermittent failures and possible system crashes are known to occur when using Storage Foundation for UNIX, (SF/HA) Veritas Cluster Server, (VCS) Storage Foundation for Oracle RAC, (SFRAC) and Storage Foundation Cluster File Server (SFCFS) with certain AIX platforms.
Details:
Problem: System Crashes/Hangs in one of the drivers of the cluster solutions
System crash/hang will occur with any version of clustering products based on 4.0 or 5.0 releases. The Service Pack releases of AIX with the patch levels specified in the following table are the levels of AIX which result in the crash.
If your system is upgraded to or installed at the specified levels or above, you are impacted. You must follow the instructions provided below to recover. The specific IBM AIX APAR triggering this LLT panic has the abstract: Bring DLPI driver "to SPEC" applies to AIX - enhance DLPI driver to comply with specifications or normal streams processing. This APAR is installed as:
Problem and fix description
Changes to the DLPI driver with the latest TLs on AIX 5.3 and 6.1 result in upcalls to LLT in the interrupt context instead of the process context. Some LLT locks are not interrupt safe which caused panics. Some upcalls panic in GAB where it does an xmalloc in interrupt context.
All LLT locks on AIX are now interrupt safe. Out-of order lock releases are handled correctly with the new interrupt safe locks. Link notifications are provided for the client in the thread or process context instead of the interrupt context. This will ensure that no changes are required for LLT's clients like GAB/LMX thus abstracting all the changes to just one driver.
Impacted releases and corresponding LLT patches
The corresponding LLT patches are found at Patch Central below and under the Related Documents section.
4.0 MP3 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2106
4.0 MP4 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2107
5.0 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2104
5.0 MP1 and 5.0 MP1 Update 1 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2103
Recommended procedure for an existing setup with panics
- Install and configure the appropriate patch for your MP
Recommended procedure for an existing setup without panics
- Install and configure the patch for the appropriate release
(follow directions in the README)
- Upgrade to the latest AIX release when ready.
Recommended procedure for a fresh install
- Use installonly to install the base release
- Upgrade to the latest MP
- Install the required llt patch, then reboot the system.
- Use the configure option to configure the cluster-server.
NOTE# The removal of IBM temporary patch. (Workaround)
Once the LLT patch is installed as above, the workaround patch can be uninstalled, if it was installed.
To uninstall the workaround patch:
For AIX 6.1:
# emgr -rL 61TL00dlpi
For AIX 5.3 TL6:
# emgr -rL 53TL06dlpi
For AIX 5.3 TL7:
# emgr -rL 53TL07dlpi
Problem: System Crashes/Hangs in one of the drivers of the cluster solutions
System crash/hang will occur with any version of clustering products based on 4.0 or 5.0 releases. The Service Pack releases of AIX with the patch levels specified in the following table are the levels of AIX which result in the crash.
| O/S Level | Storage Foundation | Veritas Cluster Server | Storage Foundation High Availability | Storage Foundation Cluster File Server | Storage Foundation Oracle RAC |
|---|---|---|---|---|---|
| AIX 5.2 TL10 SP5 | NO | YES | YES | YES | YES |
| AIX 5.3 TL6 SP3 | NO | YES | YES | YES | YES |
| AIX 5.3 TL7 SP2 | NO | YES | YES | YES | YES |
| AIX 6.1 Base SP3 | NO | YES | YES | YES | YES |
If your system is upgraded to or installed at the specified levels or above, you are impacted. You must follow the instructions provided below to recover. The specific IBM AIX APAR triggering this LLT panic has the abstract: Bring DLPI driver "to SPEC" applies to AIX - enhance DLPI driver to comply with specifications or normal streams processing. This APAR is installed as:
| AIX OS Level | AIX APAR |
|---|---|
| 5200-10 | IZ19838 |
| 5300-06 | IZ05430 |
| 5300-07 | IZ11726 |
| 6100-00 | IZ13304 |
Problem and fix description
Changes to the DLPI driver with the latest TLs on AIX 5.3 and 6.1 result in upcalls to LLT in the interrupt context instead of the process context. Some LLT locks are not interrupt safe which caused panics. Some upcalls panic in GAB where it does an xmalloc in interrupt context.
All LLT locks on AIX are now interrupt safe. Out-of order lock releases are handled correctly with the new interrupt safe locks. Link notifications are provided for the client in the thread or process context instead of the interrupt context. This will ensure that no changes are required for LLT's clients like GAB/LMX thus abstracting all the changes to just one driver.
Impacted releases and corresponding LLT patches
| Release | Patch |
|---|---|
| 4.0MP3 | 4.0MP3+e1274390_llt_b |
| 4.0MP4 | 4.0MP4+e1274390_llt |
| 5.0 | 5.0+e1274390_llt |
| 5.0MP1 | 5.0MP1_5.0MP1EXT+e1274390_llt |
| 5.0MP1-AIX6.1 | 5.0MP1_5.0MP1EXT+e1274390_llt |
The corresponding LLT patches are found at Patch Central below and under the Related Documents section.
4.0 MP3 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2106
4.0 MP4 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2107
5.0 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2104
5.0 MP1 and 5.0 MP1 Update 1 https://vias.symantec.com/labs/vpcs/vpcs/patchinfo/2103
Recommended procedure for an existing setup with panics
- Install and configure the appropriate patch for your MP
Recommended procedure for an existing setup without panics
- Install and configure the patch for the appropriate release
(follow directions in the README)
- Upgrade to the latest AIX release when ready.
Recommended procedure for a fresh install
- Use installonly to install the base release
- Upgrade to the latest MP
- Install the required llt patch, then reboot the system.
- Use the configure option to configure the cluster-server.
NOTE# The removal of IBM temporary patch. (Workaround)
Once the LLT patch is installed as above, the workaround patch can be uninstalled, if it was installed.
To uninstall the workaround patch:
For AIX 6.1:
# emgr -rL 61TL00dlpi
For AIX 5.3 TL6:
# emgr -rL 53TL06dlpi
For AIX 5.3 TL7:
# emgr -rL 53TL07dlpi
Related Documents:
http://support.veritas.com/docs/300577
http://support.veritas.com/docs/324907
http://support.veritas.com/docs/324908
http://support.veritas.com/docs/324926
http://support.veritas.com/docs/324927
http://support.veritas.com/docs/300577
http://support.veritas.com/docs/324907
http://support.veritas.com/docs/324908
http://support.veritas.com/docs/324926
http://support.veritas.com/docs/324927
Products Applied:
Cluster Server for UNIX 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Storage Foundation Cluster File System 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Storage Foundation for Oracle Real Application Clusters 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Storage Foundation for UNIX/Linux 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Cluster Server for UNIX 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Storage Foundation Cluster File System 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Storage Foundation for Oracle Real Application Clusters 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Storage Foundation for UNIX/Linux 4.0 MP3 (AIX), 4.0 MP4 (AIX), 5.0 (AIX), 5.0 MP1 (AIX), 5.0 MP1 RU1 (AIX)
Last Updated: May 13 2009 01:40 AM GMT
Expires on: 05-13-2010
Subscribe to receive critical updates about this document
Expires on: 05-13-2010
Subscribe to receive critical updates about this document
Subjects:
Cluster Server for UNIX
Application: Documentation, Informational, Installation, Patches
Publishing Status: Documentation, Techalert
Storage Foundation Cluster File System
Application: Information, Installation, Patches
Publishing Status: Documentation, Techalert
Storage Foundation for Oracle Real Application Clusters
Application: Documentation, Informational, Installation, Patches
Publishing Status: Documentation, Techalert
Storage Foundation for UNIX/Linux
Application: Documentation, Informational, Installation, Patches
Publishing Status: Documentation, Techalert
Cluster Server for UNIX
Application: Documentation, Informational, Installation, Patches
Publishing Status: Documentation, Techalert
Storage Foundation Cluster File System
Application: Information, Installation, Patches
Publishing Status: Documentation, Techalert
Storage Foundation for Oracle Real Application Clusters
Application: Documentation, Informational, Installation, Patches
Publishing Status: Documentation, Techalert
Storage Foundation for UNIX/Linux
Application: Documentation, Informational, Installation, Patches
Publishing Status: Documentation, Techalert
Languages:
English (US)
English (US)
Operating Systems:
AIX
AIX
5.3, 6.1
Symantec World Headquarters:
20330 Stevens Creek Blvd. Cupertino, CA 95014
World Wide Web: http://www.symantec.com,
Tech Support Web: http://entsupport.symantec.com,
E-Mail Support: http://seer.entsupport.symantec.com/email_forms,
FTP: ftp://ftp.entsupport.symantec.com or http://ftp.entsupport.symantec.com
20330 Stevens Creek Blvd. Cupertino, CA 95014
World Wide Web: http://www.symantec.com,
Tech Support Web: http://entsupport.symantec.com,
E-Mail Support: http://seer.entsupport.symantec.com/email_forms,
FTP: ftp://ftp.entsupport.symantec.com or http://ftp.entsupport.symantec.com
THE INFORMATION PROVIDED IN THE SYMANTEC SOFTWARE KNOWLEDGE BASE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. SYMANTEC SOFTWARE DISCLAIMS ALL WARRANTIES, EITHER EXPRESS OR IMPLIED, INCLUDING THE WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT SHALL SYMANTEC SOFTWARE OR ITS SUPPLIERS BE LIABLE FOR ANY DAMAGES WHATSOEVER INCLUDING DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, LOSS OF BUSINESS PROFITS OR SPECIAL DAMAGES,EVEN IF SYMANTEC SOFTWARE OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. SOME STATES DO NOT ALLOW THE EXCLUSION OR LIMITATION OF LIABILITY FOR CONSEQUENTIAL OR INCIDENTAL DAMAGES SO THE FOREGOING LIMITATION MAY NOT APPLY.