在share lvm中添加LV，HA发生切换

环境 AIX 5306,
       HA5.4.1，
       存储：DS4800,V7000,
       HA共享VG：testvg（化名），
       其中的PV：hdisk3,hdisk8  (DS4800)
                      hdisk7,hdisk9 (V7000)
       DS4800和V7000以LV镜像的方式保护VG

问题描述：在NODE1节点的HA的share lvm中添加lv，testvg在此节点是激活的，提示fault，大概的报错意思是没有一份好的VGDA COPY，VG没有激活，之后发现HA发生切换。
             发生切换后在备机检查这个testvg，刚刚建立的lv是存在的，建好了。
             将资源组重新在node1节点启动，也可以看到这个新建的lv,再次通过share lvm的菜单增加lv大小，出现同样的报错,ha再次切换，再次在备机上检查这个testvg，lv大小已经修改成功; 而且发现testvg中原有属于oracle属组的lv（/dev/lvname），在此备节点被改成了root system属组，导致切换到备机后oracle不能启动。
              在NODE1节点上errpt发现很多磁盘报错，有看到VG被关闭的信息，猜想是由于VG被关闭HA才发生切换，但是没想明白为什么VG会被关闭，testvg中的quorum为1,是关闭状态的。检查了两个存储没有任何的报错，链路是正常的。

              想问问大家有没有遇到过类似的情况，会是什么原因造成的呢？


            ERRPT日志如下：
IDENTIFIER TIMESTAMP  T C RESOURCE_NAME  DESCRIPTION
AFA89905   1207172912 I O grpsvcs        Group Services daemon started
97419D60   1207172912 I O topsvcs        Topology Services daemon started
6D19271E   1207172912 I O topsvcs        Topology Services daemon stopped
28854E81   1207172912 I O grpsvcs        Group Services daemon stopped
99FA80C7   1207172912 U S haemd          SOFTWARE
AA8AB241   1207172912 T O OPERATOR       OPERATOR NOTIFICATION
BC3BE5A3   1207172912 P S SRC            SOFTWARE PROGRAM ERROR
C1348779   1207171912 I O SYSJ2          LOG I/O ERROR
C1348779   1207171912 I O SYSJ2          LOG I/O ERROR
C1348779   1207171912 I O SYSJ2          LOG I/O ERROR
C1348779   1207171912 I O SYSJ2          LOG I/O ERROR
B6DB68E0   1207171912 I O SYSJ2          FILE SYSTEM RECOVERY REQUIRED
78ABDDEB   1207171912 I O SYSJ2          META-DATA I/O ERRORR
CAD234BE   1207171912 U H LVDD           QUORUM LOST, VOLUME GROUP CLOSING
F7DDA124   1207171912 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207171912 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207171912 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207171912 P H hdisk8         DISK OPERATION ERROR
F7DDA124   1207171912 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207171912 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
F7DDA124   1207171912 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207171912 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207171912 P H LVDD           I/O ERROR DETECTED BY LVM
E86653C3   1207171912 P H LVDD           I/O ERROR DETECTED BY LVM
DA1477B1   1207171912 P H hdisk5         REQUESTED OPERATION CANNOT BE PERFORMED
DA1477B1   1207171912 P H hdisk3         REQUESTED OPERATION CANNOT BE PERFORMED
F7DDA124   1207171912 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207171912 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207171912 P H LVDD           I/O ERROR DETECTED BY LVM
E86653C3   1207171912 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207171912 P H hdisk9         DISK OPERATION ERROR
B6267342   1207171912 P H hdisk8         DISK OPERATION ERROR
E86653C3   1207171912 P H LVDD           I/O ERROR DETECTED BY LVM
DA1477B1   1207171912 P H hdisk3         REQUESTED OPERATION CANNOT BE PERFORMED
AFA89905   1207153912 I O grpsvcs        Group Services daemon started
97419D60   1207153912 I O topsvcs        Topology Services daemon started
6D19271E   1207153912 I O topsvcs        Topology Services daemon stopped
28854E81   1207153912 I O grpsvcs        Group Services daemon stopped
99FA80C7   1207153912 U S haemd          SOFTWARE
AA8AB241   1207153912 T O OPERATOR       OPERATOR NOTIFICATION
BC3BE5A3   1207153912 P S SRC            SOFTWARE PROGRAM ERROR
C1348779   1207153712 I O SYSJ2          LOG I/O ERROR
C1348779   1207153712 I O SYSJ2          LOG I/O ERROR
C1348779   1207153712 I O SYSJ2          LOG I/O ERROR
C1348779   1207153712 I O SYSJ2          LOG I/O ERROR
B6DB68E0   1207153712 I O SYSJ2          FILE SYSTEM RECOVERY REQUIRED
78ABDDEB   1207153712 I O SYSJ2          META-DATA I/O ERRORR
C1348779   1207153712 I O SYSJ2          LOG I/O ERROR
CAD234BE   1207153712 U H LVDD           QUORUM LOST, VOLUME GROUP CLOSING
F7DDA124   1207153712 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207153712 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207153712 P H hdisk8         DISK OPERATION ERROR
F7DDA124   1207153712 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207153712 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
F7DDA124   1207153712 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207153712 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
DA1477B1   1207153712 P H hdisk5         REQUESTED OPERATION CANNOT BE PERFORMED
DA1477B1   1207153712 P H hdisk3         REQUESTED OPERATION CANNOT BE PERFORMED
EAA3D429   1207153712 U S LVDD           PHYSICAL PARTITION MARKED STALE
F7DDA124   1207153712 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207153712 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207153712 P H hdisk9         DISK OPERATION ERROR
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207153712 P H hdisk8         DISK OPERATION ERROR
EAA3D429   1207153712 U S LVDD           PHYSICAL PARTITION MARKED STALE
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207153712 P H hdisk8         DISK OPERATION ERROR
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
E86653C3   1207153712 P H LVDD           I/O ERROR DETECTED BY LVM
DA1477B1   1207153712 P H hdisk3         REQUESTED OPERATION CANNOT BE PERFORMED
DA1477B1   1207153712 P H hdisk3         REQUESTED OPERATION CANNOT BE PERFORMED
AFA89905   1207152912 I O grpsvcs        Group Services daemon started
97419D60   1207152912 I O topsvcs        Topology Services daemon started
6D19271E   1207152812 I O topsvcs        Topology Services daemon stopped
28854E81   1207152812 I O grpsvcs        Group Services daemon stopped
99FA80C7   1207152812 U S haemd          SOFTWARE
AA8AB241   1207152812 T O OPERATOR       OPERATOR NOTIFICATION
BC3BE5A3   1207152812 P S SRC            SOFTWARE PROGRAM ERROR
C1348779   1207152312 I O SYSJ2          LOG I/O ERROR
C1348779   1207152312 I O SYSJ2          LOG I/O ERROR
C1348779   1207152312 I O SYSJ2          LOG I/O ERROR
C1348779   1207152312 I O SYSJ2          LOG I/O ERROR
B6DB68E0   1207152312 I O SYSJ2          FILE SYSTEM RECOVERY REQUIRED
78ABDDEB   1207152312 I O SYSJ2          META-DATA I/O ERRORR
C1348779   1207152312 I O SYSJ2          LOG I/O ERROR
F7DDA124   1207152312 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207152312 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207152312 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207152312 P H hdisk9         DISK OPERATION ERROR
CAD234BE   1207152312 U H LVDD           QUORUM LOST, VOLUME GROUP CLOSING
F7DDA124   1207152312 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207152312 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207152312 P H LVDD           I/O ERROR DETECTED BY LVM
B6267342   1207152312 P H hdisk8         DISK OPERATION ERROR
41BF2110   1207152312 U H LVDD           MIRROR WRITE CACHE WRITE FAILED
B6267342   1207152312 P H hdisk9         DISK OPERATION ERROR
F7DDA124   1207152312 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
52715FA5   1207152312 U H LVDD           FAILED TO WRITE VOLUME GROUP STATUS AREA
E86653C3   1207152312 P H LVDD           I/O ERROR DETECTED BY LVM
DA1477B1   1207152312 P H hdisk5         REQUESTED OPERATION CANNOT BE PERFORMED
F7DDA124   1207152312 U H LVDD           PHYSICAL VOLUME DECLARED MISSING
41BF2110   1207152312 U H LVDD           MIRROR WRITE CACHE WRITE FAILED
DA1477B1   1207152312 P H hdisk3         REQUESTED OPERATION CANNOT BE PERFORMED



     详细日志：

LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 17:19:25 BEIST 2012
Sequence Number: 3334
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3D60
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 17:19:25 BEIST 2012
Sequence Number: 3333
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0005
BLOCK NUMBER
0000 0008
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 17:19:25 BEIST 2012
Sequence Number: 3332
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3D60
---------------------------------------------------------------------------
LABEL:          J2_FSCK_REQUIRED
IDENTIFIER:     B6DB68E0
Date/Time:       Fri Dec  7 17:19:25 BEIST 2012
Sequence Number: 3331
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
FILE SYSTEM RECOVERY REQUIRED
Probable Causes
INVALID FILE SYSTEM CONTROL DATA DETECTED
Recommended Actions
PERFORM FULL FILE SYSTEM RECOVERY USING FSCK UTILITY
OBTAIN DUMP
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Detail Data
ERROR CODE
0000 0005
JFS2 MAJOR/MINOR DEVICE NUMBER
0033 001C
CALLER
0027 9FF4
CALLER
0027 7844
CALLER
0029 2F5C
---------------------------------------------------------------------------
LABEL:          J2_METADATA_EIO
IDENTIFIER:     78ABDDEB
Date/Time:       Fri Dec  7 17:19:25 BEIST 2012
Sequence Number: 3330
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
META-DATA I/O ERRORR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 MAJOR/MINOR DEVICE NUMBER
0033 001C
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 8004
BLOCK NUMBER
0000 19D8
FILE SYSTEM DEVICE AND MOUNT POINT
not available
---------------------------------------------------------------------------
LABEL:          LVM_SA_QUORCLOSE
IDENTIFIER:     CAD234BE
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3329
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
QUORUM LOST, VOLUME GROUP CLOSING
Probable Causes
PHYSICAL VOLUME UNAVAILABLE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0033 0000 0000
QUORUM COUNT
           3
ACTIVE COUNT
           0
SENSE DATA
0000 0000 0000 056B 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3328
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0002
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3327
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
                    16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3326
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3325
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk8         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L2000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           0
SENSE DATA
0A00 2E00 0000 0080 0001 0000 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0003 1B4F 0000 0640 0000 0007 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3324
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0006
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3323
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0006
ERROR CODE AS DEFINED IN sys/errno.h
                    16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3322
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0004
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3321
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0004
ERROR CODE AS DEFINED IN sys/errno.h
                     0
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3320
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0006
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3319
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0004
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3318
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk5         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L3000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2E00 0000 0080 0001 0004 0000 0000 0000 0000 0000 6A90 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0FF4 0000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3317
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk3         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L1000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2E00 0000 0080 0001 0004 0000 0000 0000 0000 000A 3BBC 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 1064 C000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3316
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0003
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3315
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0003
ERROR CODE AS DEFINED IN sys/errno.h
                     0
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3314
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0003
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3313
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
               2901784
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0019
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                  8192
NUMBER OF BLOCKS
                  8192
I/O TYPE
USER DATA     
SENSE DATA
0000 0000 0000 1623 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3312
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk9         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L3000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           2
SENSE DATA
0A00 2E00 0000 0080 0001 0000 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0004 628F 0003 E040 0000 0007 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3311
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk8         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L2000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           0
SENSE DATA
0A00 2800 002C 4718 0000 1000 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0003 1B4F 0000 0640 0000 0007 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3310
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0004
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
             155469592
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0019
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                  8192
NUMBER OF BLOCKS
                  8192
I/O TYPE
USER DATA     
SENSE DATA
0000 0000 0004 A223 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 17:19:20 BEIST 2012
Sequence Number: 3309
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk3         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L1000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2800 0944 4718 0000 1004 0000 0000 0000 0000 000A 3BBC 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 1064 C000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          GS_START_ST
IDENTIFIER:     AFA89905
Date/Time:       Fri Dec  7 15:39:59 BEIST 2012
Sequence Number: 3308
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   grpsvcs         
Description
Group Services daemon started
Probable Causes
Daemon started during system startup
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Check that Group Services daemon is running
Detail Data
DETECTING MODULE
RSCT,pgsd.C,1.62,585                          
ERROR ID
63Y7ej0DpNkE/GbH102.e.1...................
REFERENCE CODE
                                          
DIAGNOSTIC EXPLANATION
HAGS daemon started by SRC. Log file is /var/ha/log/grpsvcs_2_54.fmfagrp.
---------------------------------------------------------------------------
LABEL:          TS_START_ST
IDENTIFIER:     97419D60
Date/Time:       Fri Dec  7 15:39:57 BEIST 2012
Sequence Number: 3307
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   topsvcs         
Description
Topology Services daemon started
Probable Causes
Daemon started during system start-up
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Confirm that this is desirable
Detail Data
DETECTING MODULE
rsct,bootstrp.C,1.204,4416                    
ERROR ID
6UpNEL0BpNkE/MKS.02.e.1...................
REFERENCE CODE
                                          
Topology Services daemon started by:
SRC
Topology Services daemon log file location
/var/ha/log/topsvcs.07.153957.fmfagrp.en_US
Topology Services daemon run directory
/var/ha/run/topsvcs.fmfagrp/
---------------------------------------------------------------------------
LABEL:          TS_STOP_ST
IDENTIFIER:     6D19271E
Date/Time:       Fri Dec  7 15:39:19 BEIST 2012
Sequence Number: 3306
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   topsvcs         
Description
Topology Services daemon stopped
Probable Causes
Daemon stopped by SRC
Daemon stopped by signal
User Causes
Daemon stopped by user
Recommended Actions
Confirm that this is desirable
Detail Data
DETECTING MODULE
rsct,comm.C,1.145,625                        
ERROR ID
6SQG4h/boNkE/1aE.02.e.1...................
REFERENCE CODE
6UpNEL0yeNkE/IVe002.e.1...................
Topology Services daemon stopped by:
Signal SIGTERM
---------------------------------------------------------------------------
LABEL:          GS_STOP_ST
IDENTIFIER:     28854E81
Date/Time:       Fri Dec  7 15:39:17 BEIST 2012
Sequence Number: 3305
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   grpsvcs         
Description
Group Services daemon stopped
Probable Causes
Daemon stopped by SRC
Daemon stopped by signal
User Causes
Daemon stopped manually by user
Recommended Actions
Check that Group Services daemon is stopped
Detail Data
DETECTING MODULE
RSCT,SRCSocket.C,1.63,457                     
ERROR ID
6/uIVc.ZoNkE/dfc/02.e.1...................
REFERENCE CODE
                                          
DIAGNOSTIC EXPLANATION
Received signal[SIGTERM]. Converted to normal stop
---------------------------------------------------------------------------
LABEL:          HA001_TR
IDENTIFIER:     99FA80C7
Date/Time:       Fri Dec  7 15:39:17 BEIST 2012
Sequence Number: 3304
Machine Id:      000000
Node Id:         node1
Class:           S
Type:            UNKN
Resource Name:   haemd           
Description
SOFTWARE
Probable Causes
SUBSYSTEM
Failure Causes
SUBSYSTEM
Recommended Actions
NONE
Detail Data
DETECTING MODULE
LPP=PSSP,Fn=emd.c,SID=1.47,L#=478,                                             
DIAGNOSTIC EXPLANATION
haemd: Received termination signal SIGTERM, program is exiting.
---------------------------------------------------------------------------
LABEL:          OPMSG
IDENTIFIER:     AA8AB241
Date/Time:       Fri Dec  7 15:39:13 BEIST 2012
Sequence Number: 3303
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            TEMP
Resource Name:   OPERATOR        
Description
OPERATOR NOTIFICATION
User Causes
ERRLOGGER COMMAND
Recommended Actions
REVIEW DETAILED DATA
Detail Data
MESSAGE FROM ERRLOGGER COMMAND
clexit.rc : Normal termination of clstrmgrES. Restart now.
---------------------------------------------------------------------------
LABEL:          SRC_SVKO
IDENTIFIER:     BC3BE5A3
Date/Time:       Fri Dec  7 15:39:13 BEIST 2012
Sequence Number: 3302
Machine Id:      000000
Node Id:         node1
Class:           S
Type:            PERM
Resource Name:   SRC            
Description
SOFTWARE PROGRAM ERROR
Probable Causes
APPLICATION PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
MANUALLY RESTART SUBSYSTEM IF NEEDED
Detail Data
SYMPTOM CODE
         256
SOFTWARE ERROR CODE
       -9017
ERROR CODE
           0
DETECTING MODULE
[url=mailto:]'srchevn.c'@line:'350'[/url]
FAILING MODULE
clstrmgrES
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:37:46 BEIST 2012
Sequence Number: 3301
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0005
BLOCK NUMBER
0000 0008
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:37:46 BEIST 2012
Sequence Number: 3300
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3C68
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:37:46 BEIST 2012
Sequence Number: 3299
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0005
BLOCK NUMBER
0000 0008
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:37:46 BEIST 2012
Sequence Number: 3298
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3C68
---------------------------------------------------------------------------
LABEL:          J2_FSCK_REQUIRED
IDENTIFIER:     B6DB68E0
Date/Time:       Fri Dec  7 15:37:43 BEIST 2012
Sequence Number: 3297
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
FILE SYSTEM RECOVERY REQUIRED
Probable Causes
INVALID FILE SYSTEM CONTROL DATA DETECTED
Recommended Actions
PERFORM FULL FILE SYSTEM RECOVERY USING FSCK UTILITY
OBTAIN DUMP
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Detail Data
ERROR CODE
0000 0005
JFS2 MAJOR/MINOR DEVICE NUMBER
0033 001C
CALLER
0027 F69C
CALLER
0028 37AC
CALLER
002B D710
---------------------------------------------------------------------------
LABEL:          J2_METADATA_EIO
IDENTIFIER:     78ABDDEB
Date/Time:       Fri Dec  7 15:37:43 BEIST 2012
Sequence Number: 3296
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
META-DATA I/O ERRORR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 MAJOR/MINOR DEVICE NUMBER
0033 001C
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 8004
BLOCK NUMBER
0000 0078
FILE SYSTEM DEVICE AND MOUNT POINT
not available
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:37:43 BEIST 2012
Sequence Number: 3295
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3C68
---------------------------------------------------------------------------
LABEL:          LVM_SA_QUORCLOSE
IDENTIFIER:     CAD234BE
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3294
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
QUORUM LOST, VOLUME GROUP CLOSING
Probable Causes
PHYSICAL VOLUME UNAVAILABLE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0033 0000 0000
QUORUM COUNT
           3
ACTIVE COUNT
           0
SENSE DATA
0000 0000 0000 056B 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3293
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0002
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3292
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
                    16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3291
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3290
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk8         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L2000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           0
SENSE DATA
0A00 2E00 0000 0080 0001 0000 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0003 1835 0001 2CC0 0000 0005 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3289
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0006
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3288
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0006
ERROR CODE AS DEFINED IN sys/errno.h
                    16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3287
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0004
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3286
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0004
ERROR CODE AS DEFINED IN sys/errno.h
                     0
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3285
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0006
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3284
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0004
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3283
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk5         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L3000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2E00 0000 0080 0001 0004 0000 0000 0000 0000 0000 6A90 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0FF4 0000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3282
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk3         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L1000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2E00 0000 0080 0001 0004 0000 0000 0000 0000 000A 29EA 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 1064 C000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_STALEPP
IDENTIFIER:     EAA3D429
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3281
Machine Id:      000000
Node Id:         node1
Class:           S
Type:            UNKN
Resource Name:   LVDD            
Description
PHYSICAL PARTITION MARKED STALE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
PHYSICAL PARTITION NUMBER (DECIMAL)
                    37
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0012
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3280
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0003
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3279
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0003
ERROR CODE AS DEFINED IN sys/errno.h
                     0
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3278
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0003
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3277
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk9         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L3000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           2
SENSE DATA
0A00 2E00 0000 0080 0001 0000 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0004 5FA4 000A 6700 0000 0005 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3276
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
               4737522
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0012
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                   512
NUMBER OF BLOCKS
                   512
I/O TYPE
USER DATA     
SENSE DATA
0000 0000 0000 2424 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3275
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk8         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L2000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           0
SENSE DATA
0A00 2A00 0048 49F2 0000 0100 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0003 1835 0001 2CC0 0000 0005 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_SA_STALEPP
IDENTIFIER:     EAA3D429
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3274
Machine Id:      000000
Node Id:         node1
Class:           S
Type:            UNKN
Resource Name:   LVDD            
Description
PHYSICAL PARTITION MARKED STALE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
PHYSICAL PARTITION NUMBER (DECIMAL)
                    39
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0011
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3273
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
               4999666
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0011
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                   512
NUMBER OF BLOCKS
                   512
I/O TYPE
USER DATA     
SENSE DATA
0000 0000 0000 2624 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3272
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk8         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L2000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           0
SENSE DATA
0A00 2A00 004C 49F2 0000 0100 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0003 1835 0001 2CC0 0000 0005 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3271
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0004
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
             153635314
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0012
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                   512
NUMBER OF BLOCKS
                   512
I/O TYPE
USER DATA     
SENSE DATA
0000 0000 0004 9424 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3270
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0004
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
             153373170
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0011
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                   512
NUMBER OF BLOCKS
                   512
I/O TYPE
USER DATA     
SENSE DATA
0000 0000 0004 9224 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3269
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk3         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L1000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2A00 0928 49F2 0000 0104 0000 0000 0000 0000 000A 29EA 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 1064 C000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 15:37:42 BEIST 2012
Sequence Number: 3268
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk3         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L1000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2A00 0924 49F2 0000 0104 0000 0000 0000 0000 000A 29EA 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 1064 C000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          GS_START_ST
IDENTIFIER:     AFA89905
Date/Time:       Fri Dec  7 15:29:06 BEIST 2012
Sequence Number: 3267
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   grpsvcs         
Description
Group Services daemon started
Probable Causes
Daemon started during system startup
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Check that Group Services daemon is running
Detail Data
DETECTING MODULE
RSCT,pgsd.C,1.62,585                          
ERROR ID
63Y7ej00fNkE/Dfn.02.e.1...................
REFERENCE CODE
                                          
DIAGNOSTIC EXPLANATION
HAGS daemon started by SRC. Log file is /var/ha/log/grpsvcs_2_53.fmfagrp.
---------------------------------------------------------------------------
LABEL:          TS_START_ST
IDENTIFIER:     97419D60
Date/Time:       Fri Dec  7 15:29:02 BEIST 2012
Sequence Number: 3266
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   topsvcs         
Description
Topology Services daemon started
Probable Causes
Daemon started during system start-up
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Confirm that this is desirable
Detail Data
DETECTING MODULE
rsct,bootstrp.C,1.204,4416                    
ERROR ID
6UpNEL0yeNkE/IVe002.e.1...................
REFERENCE CODE
                                          
Topology Services daemon started by:
SRC
Topology Services daemon log file location
/var/ha/log/topsvcs.07.152902.fmfagrp.en_US
Topology Services daemon run directory
/var/ha/run/topsvcs.fmfagrp/
---------------------------------------------------------------------------
LABEL:          TS_STOP_ST
IDENTIFIER:     6D19271E
Date/Time:       Fri Dec  7 15:28:23 BEIST 2012
Sequence Number: 3265
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   topsvcs         
Description
Topology Services daemon stopped
Probable Causes
Daemon stopped by SRC
Daemon stopped by signal
User Causes
Daemon stopped by user
Recommended Actions
Confirm that this is desirable
Detail Data
DETECTING MODULE
rsct,comm.C,1.145,625                        
ERROR ID
6SQG4h/LeNkE/IUv.02.e.1...................
REFERENCE CODE
6UpNEL0eWifE/ajW102.e.1...................
Topology Services daemon stopped by:
Signal SIGTERM
---------------------------------------------------------------------------
LABEL:          GS_STOP_ST
IDENTIFIER:     28854E81
Date/Time:       Fri Dec  7 15:28:21 BEIST 2012
Sequence Number: 3264
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   grpsvcs         
Description
Group Services daemon stopped
Probable Causes
Daemon stopped by SRC
Daemon stopped by signal
User Causes
Daemon stopped manually by user
Recommended Actions
Check that Group Services daemon is stopped
Detail Data
DETECTING MODULE
RSCT,SRCSocket.C,1.63,457                     
ERROR ID
6/uIVc.JeNkE/At4002.e.1...................
REFERENCE CODE
                                          
DIAGNOSTIC EXPLANATION
Received signal[SIGTERM]. Converted to normal stop
---------------------------------------------------------------------------
LABEL:          HA001_TR
IDENTIFIER:     99FA80C7
Date/Time:       Fri Dec  7 15:28:21 BEIST 2012
Sequence Number: 3263
Machine Id:      000000
Node Id:         node1
Class:           S
Type:            UNKN
Resource Name:   haemd           
Description
SOFTWARE
Probable Causes
SUBSYSTEM
Failure Causes
SUBSYSTEM
Recommended Actions
NONE
Detail Data
DETECTING MODULE
LPP=PSSP,Fn=emd.c,SID=1.47,L#=478,                                             
DIAGNOSTIC EXPLANATION
haemd: Received termination signal SIGTERM, program is exiting.
---------------------------------------------------------------------------
LABEL:          OPMSG
IDENTIFIER:     AA8AB241
Date/Time:       Fri Dec  7 15:28:19 BEIST 2012
Sequence Number: 3262
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            TEMP
Resource Name:   OPERATOR        
Description
OPERATOR NOTIFICATION
User Causes
ERRLOGGER COMMAND
Recommended Actions
REVIEW DETAILED DATA
Detail Data
MESSAGE FROM ERRLOGGER COMMAND
clexit.rc : Normal termination of clstrmgrES. Restart now.
---------------------------------------------------------------------------
LABEL:          SRC_SVKO
IDENTIFIER:     BC3BE5A3
Date/Time:       Fri Dec  7 15:28:18 BEIST 2012
Sequence Number: 3261
Machine Id:      000000
Node Id:         node1
Class:           S
Type:            PERM
Resource Name:   SRC            
Description
SOFTWARE PROGRAM ERROR
Probable Causes
APPLICATION PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
MANUALLY RESTART SUBSYSTEM IF NEEDED
Detail Data
SYMPTOM CODE
         256
SOFTWARE ERROR CODE
       -9017
ERROR CODE
           0
DETECTING MODULE
[url=mailto:]'srchevn.c'@line:'350'[/url]
FAILING MODULE
clstrmgrES
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:23:58 BEIST 2012
Sequence Number: 3260
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0005
BLOCK NUMBER
0000 0008
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:23:58 BEIST 2012
Sequence Number: 3259
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3C50
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:23:58 BEIST 2012
Sequence Number: 3258
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0005
BLOCK NUMBER
0000 0008
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:23:52 BEIST 2012
Sequence Number: 3257
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3C50
---------------------------------------------------------------------------
LABEL:          J2_FSCK_REQUIRED
IDENTIFIER:     B6DB68E0
Date/Time:       Fri Dec  7 15:23:51 BEIST 2012
Sequence Number: 3256
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
FILE SYSTEM RECOVERY REQUIRED
Probable Causes
INVALID FILE SYSTEM CONTROL DATA DETECTED
Recommended Actions
PERFORM FULL FILE SYSTEM RECOVERY USING FSCK UTILITY
OBTAIN DUMP
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Detail Data
ERROR CODE
0000 0005
JFS2 MAJOR/MINOR DEVICE NUMBER
0033 001C
CALLER
0027 F69C
CALLER
0028 37AC
CALLER
002B D710
---------------------------------------------------------------------------
LABEL:          J2_METADATA_EIO
IDENTIFIER:     78ABDDEB
Date/Time:       Fri Dec  7 15:23:51 BEIST 2012
Sequence Number: 3255
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
META-DATA I/O ERRORR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 MAJOR/MINOR DEVICE NUMBER
0033 001C
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 8004
BLOCK NUMBER
0000 0078
FILE SYSTEM DEVICE AND MOUNT POINT
not available
---------------------------------------------------------------------------
LABEL:          J2_LOG_EIO
IDENTIFIER:     C1348779
Date/Time:       Fri Dec  7 15:23:51 BEIST 2012
Sequence Number: 3254
Machine Id:      000000
Node Id:         node1
Class:           O
Type:            INFO
Resource Name:   SYSJ2           
Description
LOG I/O ERROR
Probable Causes
ADAPTER HARDWARE OR MICROCODE
DISK DRIVE HARDWARE OR MICROCODE
SOFTWARE DEVICE DRIVER
STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED
Recommended Actions
CHECK CABLES AND THEIR CONNECTIONS
INSTALL LATEST ADAPTER AND DRIVE MICROCODE
INSTALL LATEST STORAGE DEVICE DRIVERS
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
JFS2 LOG MAJOR/MINOR DEVICE NUMBER
0033 001D
ERROR CODE
0000 0006
BUF STRUCTURE B_FLAGS
000C 0404
BLOCK NUMBER
0000 3C50
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3253
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0003
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3252
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0003
ERROR CODE AS DEFINED IN sys/errno.h
                    16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3251
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0003
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3250
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk9         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L3000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           2
SENSE DATA
0A00 2E00 0000 0080 0001 0000 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0004 5F93 0009 C940 0000 0003 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_SA_QUORCLOSE
IDENTIFIER:     CAD234BE
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3249
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
QUORUM LOST, VOLUME GROUP CLOSING
Probable Causes
PHYSICAL VOLUME UNAVAILABLE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0033 0000 0000
QUORUM COUNT
           3
ACTIVE COUNT
           0
SENSE DATA
0000 0000 0000 056B 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3248
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0002
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3247
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
                    16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3246
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0031 0000 0002
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A C717
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3245
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk8         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L2000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           0
SENSE DATA
0A00 2E00 0000 0080 0001 0000 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0003 182C 000D 4500 0000 0003 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_MWCWFAIL
IDENTIFIER:     41BF2110
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3244
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
MIRROR WRITE CACHE WRITE FAILED
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0031 0000 0003
BLOCK NUMBER
                     2
ERROR CODE AS DEFINED IN sys/errno.h
          16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF B27A 9B7D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          SC_DISK_ERR2
IDENTIFIER:     B6267342
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3243
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk9         
Resource Class:  disk
Resource Type:   mpioosdisk
Location:        U7879.001.DQDWZXR-P1-C6-T1-W50050768022076AF-L3000000000000
VPD:            
        Manufacturer................IBM     
        Machine Type and Model......2145            
        ROS Level and ID............30303030
        Serial Number...............2076
        Device Specific.(Z0)........0000063268181002
        Device Specific.(Z1)........
        Device Specific.(Z2)........
        Device Specific.(Z3)........
Description
DISK OPERATION ERROR
Probable Causes
DASD DEVICE
Failure Causes
DISK DRIVE
DISK DRIVE ELECTRONICS
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
PATH ID
           2
SENSE DATA
0A00 2A00 0000 0002 0000 0100 0000 0000 0000 0000 0000 0000 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0004 5F93 0009 C940 0000 0003 0000 0000 0000 0000 0000 0083 0000
0000 003D 0017
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3242
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0006
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_WRTERR
IDENTIFIER:     52715FA5
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3241
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
FAILED TO WRITE VOLUME GROUP STATUS AREA
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0006
ERROR CODE AS DEFINED IN sys/errno.h
                     0
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_IO_FAIL
IDENTIFIER:     E86653C3
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3240
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
I/O ERROR DETECTED BY LVM
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Recommended Actions
RUN DIAGNOSTICS AGAINST THE FAILING DEVICE
Detail Data
PHYSICAL VOLUME DEVICE MAJOR/MINOR
8000 0029 0000 0006
ERROR CODE AS DEFINED IN sys/errno.h
          16
BLOCK NUMBER
                   128
LOGICAL VOLUME DEVICE MAJOR/MINOR
8000 0033 0000 0000
PHYSICAL BUFFER TRANSACTION TIME
                     0
RESIDUAL COUNT
                131072
NUMBER OF BLOCKS
                131072
I/O TYPE
LVM META DATA
SENSE DATA
0000 0000 0000 0000 00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 9699 A74D
0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3239
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk5         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L3000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2E00 0000 0080 0001 0004 0000 0000 0000 0000 0000 6A90 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0FF4 0000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_SA_PVMISS
IDENTIFIER:     F7DDA124
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3238
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
PHYSICAL VOLUME DECLARED MISSING
Probable Causes
POWER, DRIVE, ADAPTER, OR CABLE FAILURE
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0004
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          LVM_MWCWFAIL
IDENTIFIER:     41BF2110
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3237
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            UNKN
Resource Name:   LVDD            
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
MIRROR WRITE CACHE WRITE FAILED
Detail Data
MAJOR/MINOR DEVICE NUMBER
8000 0029 0000 0004
BLOCK NUMBER
                     2
ERROR CODE AS DEFINED IN sys/errno.h
          16
SENSE DATA
00C9 41AF 0000 4C00 0000 0114 7339 F3A0 00C9 41AF 6759 A8E2 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:          FCP_ARRAY_ERR18
IDENTIFIER:     DA1477B1
Date/Time:       Fri Dec  7 15:23:43 BEIST 2012
Sequence Number: 3236
Machine Id:      000000
Node Id:         node1
Class:           H
Type:            PERM
Resource Name:   hdisk3         
Resource Class:  disk
Resource Type:   array
Location:        U7879.001.DQDWZXR-P1-C6-T1-W204500A0B8292B4C-L1000000000000
Description
REQUESTED OPERATION CANNOT BE PERFORMED
Probable Causes
SOFTWARE PROGRAM
Failure Causes
DEVICE LOCKED BY ANOTHER USER
Recommended Actions
UNLOCK RESOURCE AND RETRY
Detail Data
SENSE DATA
0A00 2A00 0000 0002 0000 0104 0000 0000 0000 0000 000A 1929 0118 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 1064 C000 F205 3402 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000
---------------------------------------------------------------------------收起