aix 5.3 RAC HA 一个接点宕机

两台P6-570 服务器 HACMP RAC oracle 双机

节点一上的卷组 非激活状态 。收集了错误日志 ，不知道问题出在哪里。朋友们帮忙分析下 。




---------------------------------------------------------------------------
LABEL:  OPMSG
IDENTIFIER: AA8AB241
Date/Time:       Thu Jun 17 11:02:06 2010
Sequence Number: 1165
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   progname
Description
OPERATOR NOTIFICATION
User Causes
ERRLOGGER COMMAND
Recommended Actions
REVIEW DETAILED DATA
Detail Data
MESSAGE FROM ERRLOGGER COMMAND
SrcCntrld: daemon not started via SRC +++
---------------------------------------------------------------------------
LABEL:  OPMSG
IDENTIFIER: AA8AB241
Date/Time:       Thu Jun 17 10:48:35 2010
Sequence Number: 1164
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   progname
Description
OPERATOR NOTIFICATION
User Causes
ERRLOGGER COMMAND
Recommended Actions
REVIEW DETAILED DATA
Detail Data
MESSAGE FROM ERRLOGGER COMMAND
SrcCntrld: daemon not started via SRC +++
---------------------------------------------------------------------------
LABEL:  TS_LOC_DOWN_ST
IDENTIFIER: 173C787F
Date/Time:       Thu Jun 17 10:31:47 2010
Sequence Number: 1163
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            INFO
WPAR:            Global
Resource Name:   topsvcs
Description
Possible malfunction on local adapter
Probable Causes
Local adapter mal-functioned
Local adapter lost connection to network
Local adapter mis-configured
Failure Causes
Local adapter mal-functioned
Local adapter lost connection to network
Local adapter mis-configured
Recommended Actions
Verify adapter configuration
Verify network connectivity
Detail Data
DETECTING MODULE
rsct,threephs.C,1.206,5242                    
ERROR ID
6zV5DL.XzX4A/7fP1/2.e.1...................
REFERENCE CODE
                                          
Adapter interface name
en8
Adapter offset
           1
Adapter IP address
192.168.1.1
---------------------------------------------------------------------------
LABEL:  GS_START_ST
IDENTIFIER: AFA89905
Date/Time:       Thu Jun 17 10:25:17 2010
Sequence Number: 1162
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   grpsvcs
Description
Group Services daemon started
Probable Causes
Daemon started during system startup
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Check that Group Services daemon is running
Detail Data
DETECTING MODULE
RSCT,pgsd.C,1.62.1.8,606                     
ERROR ID
63Y7ej0RtX4A/bSl0/2.e.1...................
REFERENCE CODE
                                          
DIAGNOSTIC EXPLANATION
HAGS daemon started by SRC. Log file is /var/ha/log/grpsvcs_trace_1_46.
---------------------------------------------------------------------------
LABEL:  TS_START_ST
IDENTIFIER: 97419D60
Date/Time:       Thu Jun 17 10:25:13 2010
Sequence Number: 1161
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   topsvcs
Description
Topology Services daemon started
Probable Causes
Daemon started during system start-up
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Confirm that this is desirable
Detail Data
DETECTING MODULE
rsct,bootstrp.C,1.213,4440                    
ERROR ID
6UpNEL0NtX4A/8fP1/2.e.1...................
REFERENCE CODE
                                          
Topology Services daemon started by:
SRC
Topology Services daemon log file location
/var/ha/log/topsvcs.17.102513.ibm_cls.en_US
Topology Services daemon run directory
/var/ha/run/topsvcs.ibm_cls/
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Thu Jun 17 09:27:58 2010
Sequence Number: 1160
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                 74104
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  OPMSG
IDENTIFIER: AA8AB241
Date/Time:       Thu Jun 17 09:25:52 2010
Sequence Number: 1159
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   progname
Description
OPERATOR NOTIFICATION
User Causes
ERRLOGGER COMMAND
Recommended Actions
REVIEW DETAILED DATA
Detail Data
MESSAGE FROM ERRLOGGER COMMAND
SrcCntrld: daemon not started via SRC +++
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1158
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                188626
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1157
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                 78210
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1156
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                188624
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
Duplicates
Number of duplicates
           2
Time of first duplicate
Sun Jun 13 13:53:22 2010
Time of last duplicate
Sun Jun 13 13:53:22 2010
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1155
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                188624
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1154
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                188624
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1153
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                164286
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1152
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                135596
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
Duplicates
Number of duplicates
           2
Time of first duplicate
Sun Jun 13 13:53:22 2010
Time of last duplicate
Sun Jun 13 13:53:22 2010
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1151
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                135596
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Sun Jun 13 13:53:22 2010
Sequence Number: 1150
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                135596
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  RMCD_INFO_0_ST
IDENTIFIER: A6DF45AA
Date/Time:       Sun Jun 13 13:53:21 2010
Sequence Number: 1149
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   RMCdaemon
Description
The daemon is started.
Probable Causes
The Resource Monitoring and Control daemon has been started.
User Causes
The startsrc -s ctrmc command has been executed or
the rmcctrl -s command has been executed.
Recommended Actions
Confirm that the daemon should be started.
Detail Data
DETECTING MODULE
RSCT,rmcd.c,1.56,213                          
ERROR ID
6eKora0VYG3A/6et//2.e.1...................
REFERENCE CODE
                                          
---------------------------------------------------------------------------
LABEL:  TUNE_RESTRICTED
IDENTIFIER: D221BD55
Date/Time:       Sun Jun 13 13:53:09 2010
Sequence Number: 1148
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   perftune
Description
RESTRICTED TUNABLES MODIFIED AT REBOOT
Probable Causes
SYSTEM TUNING
User Causes
TUNABLE PARAMETER OF TYPE RESTRICTED HAS BEEN MODIFIED
Recommended Actions
REVIEW TUNABLE LISTS IN DETAILED DATA
Detail Data
LIST OF TUNABLE COMMANDS CONTROLLING MODIFIED RESTRICTED TUNABLES AT REBOOT, SEE FILE /etc/tunables/lastboot.log
vmo, no
---------------------------------------------------------------------------
LABEL:  DUMP_STATS
IDENTIFIER: 67145A39
Date/Time:       Sun Jun 13 13:52:37 2010
Sequence Number: 1146
Machine Id:      000000000000
Node Id:         localhost
Class:           S
Type:            UNKN
WPAR:            Global
Resource Name:   SYSDUMP
Description
SYSTEM DUMP
Probable Causes
UNEXPECTED SYSTEM HALT
User Causes
SYSTEM DUMP REQUESTED BY USER
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Failure Causes
UNEXPECTED SYSTEM HALT
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
DUMP DEVICE
/dev/lg_dumplv
DUMP SIZE
             323300864
TIME
Sun Jun 13 13:43:19 2010
DUMP TYPE (1 = PRIMARY, 2 = SECONDARY)
           1
DUMP STATUS
           0
ERROR CODE
           0
DUMP INTEGRITY
Compressed dump - Run dmpfmt with -c flag on dump after uncompressing.
FILE NAME
PROCESSOR ID
           4
---------------------------------------------------------------------------
LABEL:  FIRMWARE_EVENT
IDENTIFIER: A6D1BD62
Date/Time:       Sun Jun 13 13:51:02 2010
Sequence Number: 1145
Machine Id:      000000000000
Node Id:         localhost
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   unspecified
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Firmware Event
Detail Data
Event Information
0644 00E3 0000 0088 A600 8E00 0000 0000 0000 0000 4942 4D00 5048 0030 0100 A206
2010 0613 1851 0200 0000 0000 0000 0000 4C00 0004 0000 0000 0000 0000 0000 0000
0000 0083 0000 0000 5548 0018 0100 A206 8303 1080 0000 0000 0000 0000 0000 0000
4D54 001C 0100 A206 3931 3137 2D4D 4D41 3635 3936 4130 3200 0000 0000 4C52 0014
0100 A206 4000 0000 0000 0001 9C00 D000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  MINIDUMP_LOG
IDENTIFIER: F48137AC
Date/Time:       Sun Jun 13 13:51:02 2010
Sequence Number: 1144
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            UNKN
WPAR:            Global
Resource Name:   minidump
Description
COMPRESSED MINIMAL DUMP
Probable Causes
System dumped. Minimal Dump collected in Non-Volatile Memory.
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
Minidump Data:
4D32 0D6C 24EC 0058 0033 0037 00E4 01DB 0000 0000 1345 2E00 0000 0000 90E4 67C8
0000 0003 4000 0004 4C15 2867 21C6 06CA 0165 6E64 0074 6100 0550 190A 318C C4E7
BB80 5251 0001 0000 6563 DDBA 0108 0040 0180 054C 2A98 3806 A0A1 C387 0E09 0090
9800 8003 1844 5AC0 C0A1 1106 8017 64CA D879 C1E6 CC17 3275 DAC0 6163 0765 6345
A4C0 9001 2040 806D 4C2A 9838 06A0 A09D 8600 8240 9C48 7162 2A12 A5F8 4DC9 3205
8A94 2743 2A56 3440 B100 1012 1011 345C 61EE 059A 376D CABC 7823 27CC 1836 32E1
C879 43A6 CE18 3A2F 62C0 7021 C305 8C17 63E4 CC79 C1E6 CD99 176E 7896 8981 74CE
1C32 4869 9611 C9B5 ABD7 AF60 C38A 1D4B B66C D937 63AC 9271 2126 8D1B B322 EDB4
8133 E60D 9C3C 206C 0819 A200 6E43 1020 D0DC F962 464E 9D2F 69C8 E800 E1E4 4510
0580 057F 1903 E770 E2C5 8D1F 9785 4307 8D9C 3261 C87C 5903 4208 0C05 3F7E 28D0
C7AD A0C8 0210 2174 4C42 10E2 08B4 6AD9 BA9D B802 22C9 8623 387B 062D 7ACD C414
1061 0165 6300 2404 0040 90E0 9067 AC02 2C18 F2A4 0914 0001 4610 2848 5083 2910
F00A BEA0 B706 0C45 01C2 B400 A208 3182 0492 0427 16FC C7D2 1FC9 8128 21C6 0C60
2066 0492 4894 10C9 4870 A31D 0023 2942 9011 0F80 0104 1802 9189 0940 C047 04CA
82F6 94A7 0216 456F AD7C BDBB 9A75 6B41 7847 602C 2D88 552B 57B3 1403 982C 8AC0
0194 4C9A 4090 842B 1725 4CA6 28D9 0248 A002 25CB 7F2E D3A2 1410 C008 5092 4329
428B 50AB 2E41 1128 8420 A208 84E2 0601 76DA 7C61 F3E6 CD9A 3A70 BE90 91B3 4206
0D94 0944 9008 50A0 0000 9E14 C9A8 C44B 5041 AC56 007A C88B 5307 5DAA 6F41 3150
0016 B389 260A 246D E3D6 CD1B DDAA 1741 1148 8187 A232 CA44 C96F E7DE DD9B 9592
E83B E09C 7016 13BB DDEC CBB9 A363 352D 2884 F027 9E91 6C32 871D FAED CD59 F18B
3E1D 450F DA24 C571 1F73 BDB5 F281 7BF0 4533 E06E 76A0 51C3 2814 7500 4202 13E9
C6C6 01F9 5CC1 8B7B 5D80 02C0 4405 1C10 D16C 0008 A11A 20B0 6DE4 8663 4D85 2495
41D6 40F3 2200 90C8 C054 5104 5400 0804 7940 9122 3D3F C5C4 6241 4302 3853 5147
2565 5541 0140 E084 270F 1949 5002 5212 7457 6D72 2CF3 5100 4020 8092 088D A0E4
5A95 5716 3442 9852 9659 1003 6CB9 058D 2430 CCF4 5296 3326 0620 1122 10F1 625D
1600 E08A 5314 0D11 0D22 7F15 CA4F 6673 8856 061B 65D0 5106 1E2B C420 449C 2425
C10B 0488 2ACA A8A3 911A 3104 4A49 00C3 CD1C 68B4 6186 1C65 9471 C71C 910E 711A
494F 3042 CA1A 8FA6 4107 A435 1841 2945 11D0 0307 AD78 D8BA 020E 9F92 C4C0 35EC
A431 C71C 699C B142 0D32 144B 9101 69B0 C0EC 199A A661 4719 A4C9 9003 0E28 1940
0536 85FE B50F 6BAE C156 5001 8016 5481 02C2 9034 4120 9BE0 5747 3A18 1411 1408
1B40 1093 0259 52B4 4800 02B4 BB6E 99EF 0623 2FBD F6A6 A3C1 0FFB F6FB 6FC0 050D
5CB0 9828 BD4B 1C45 F3D6 4B60 3A1B FC11 B1BF 2801 AC65 C504 1B4C 5001 6816 7403
070D 2CEC 3177 E974 F0CA C831 093A 19C7 0C13 B80E 241A 4447 B28D 3DC4 8389 22C2
B891 0187 1E4E 34C4 0031 15A0 175F 04B5 09C5 9BBB 1284 B5D5 5893 B435 006D 4569
57C0 4FFF F886 BA00 982C 6472 1403 1840 9656 33A5 76D9 1AD1 1387 7528 C156 1704
7A13 8440 DF00 6090 E2A1 AF11 F955 DD41 3E2D 276A 4782 2D62 0048 00C0 D601 1004
022E 92C0 6C50 0F21 921B 8554 204B 32F5 F4E4 0140 CE54 023F FE84 B9E6 9C4F 5EF9
E59E 2B59 E594 008C 6E54 E69B 778E 00E5 9677 9E24 E84C A949 5BEA B43D FDC0 5F81
E145 FCEA B9CF EE3C 958A D3D6 240A 0640 1313 064C 12F0 0A13 1C30 E9A4 25D6 A384
BD95 4ED2 6209 4A08 F86B 8088 1251 8404 0944 6054 F78A 1FB9 48D2 061A 7033 638D
42E2 A8E3 0439 8007 F1D6 8692 2239 0F22 4842 8A1D 88C1 1495 D186 4A64 2209 07C0
D087 2D75 0925 0DA8 034A 1830 3BE1 3940 8369 620A 9BC0 E616 6318 416C 8249 0B05
EB64 A33B C16F 4F7E F393 CA74 562E 961C AA0D 8922 C3A2 1AF5 A848 4D0A 5497 CA94
0E37 5586 4E49 AB20 A11A 55A9 4E95 AA55 B5EA 5514 89D5 AC6A 75AB 67E9 0A25 BDFA
1515 8775 4482 1C2B 59CB 6AD6 B3A2 15AE 6A5D 2B5B DBEA D6B7 C235 AE1A FEE3 5CAD
291C 81D0 7103 E010 845F 24BB 5FFE 8C24 002A 30E3 8024 7140 30CE 602F 74C8 E340
0441 01C9 C644 1001 0023 0591 A388 2009 3947 7C3C A120 E933 2053 2659 487E A041
6859 2B08 FEF4 4712 FE25 8724 982B CA00 A0E6 0C16 848E 209C FC18 0046 1114 14D4
A54C 3A0B E420 1B46 076F 088D 2044 9387 1A8A F10A 76FC 8269 1FAA 1DD4 C434 B5BE
E8AE 8427 2489 09A1 60B5 694A F384 719B 0905 9569 36B4 4DB0 8205 94E0 368D 1400
0A66 1325 DF54 2649 A2D0 8C3F C604 707C 331C D800 2738 8D10 0E70 85EB 0990 0CB2
CC97 C404 822F 215D 0026 E33A 40D8 8779 94F0 9D02 1988 17DB EDAE 743B 43CD F26E
A7B9 8416 F4A0 B15B E0F3 9489 50DD 51CE A00A C580 4617 57A5 018A 6E00 C76B 0980
266A 00DC 2514 9052 825E 3F9B 0404 1680 6B30 129C 841B 485A 9005 E022 2604 A092
BF40 7100 4380 23A5 8502 0700 0EC0 3E12 A986 0428 9A1F 8B40 2292 E4D8 8010 FB6B
615E 3EA4 A30A E403 0626 2589 0135 D9C1 0426 E5A6 38AD 1240 C989 1837 28A8 2084
B960 20F3 11AE 0EC6 E401 740D 614C 4618 3667 5401 123C 8588 5B59 6824 22C0 8F45
75F1 D29F 4842 C31A DE30 873B 7414 A424 154A 0058 0A53 38D4 140F 8D08 2A51 91CA
54A8 5215 AB62 E02A 9448 1158 B6C2 D515 4992 45D4 D281 8B1B 4496 B2AE 35C6 2E1A
C58C CD42 23B7 4AB3 4692 888B 5C35 8463 BAC2 D54F 8264 C007 3340 4C1C AED1 301E
1CF3 8E12 4309 04DC 2AB0 9485 4B65 C7D5 2A41 20B0 DC86 F900 62D0 CDA3 50A8 8BB2
8BF9 B64C C78D 8172 99FB B11F 8802 67D2 252F 412C E6C0 9A90 E4B8 A1E4 2E7B 6916
043C C097 B183 5AAF BDD8 F103 6AFC 1200 4433 5A2C 9491 06A8 6EB7 43C9 8C1E 49A4
06B6 BD38 D36A 7E05 2C45 328C E1BF 9284 C324 4461 5AA8 4B37 7D9E ADAD 6F15 2B8A
---------------------------------------------------------------------------
LABEL:  CORE_DUMP
IDENTIFIER: A924A5FC
Date/Time:       Sun Jun 13 13:51:02 2010
Sequence Number: 1143
Machine Id:      000000000000
Node Id:         localhost
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSPROC
Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED
Probable Causes
SOFTWARE PROGRAM
User Causes
USER GENERATED SIGNAL
Recommended Actions
CORRECT THEN RETRY
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
RERUN THE APPLICATION PROGRAM
IF PROBLEM PERSISTS THEN DO THE FOLLOWING
CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
                344100
FILE SYSTEM SERIAL NUMBER
           8
INODE NUMBER
       11238
CORE FILE NAME
/home/oracle/product/10.2.0/crs/log/node1/cssd/core
PROGRAM NAME
ocssd.bin
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
vmpcopy 6BC
PROCESSOR ID
  hw_fru_id: N/A
  hw_cpu_id: N/A
ADDITIONAL INFORMATION
pthread_k B0
??
Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/ocssd.bin SIG/6 FLDS/pthread_k VALU/b0
---------------------------------------------------------------------------
LABEL:  ERRLOG_ON
IDENTIFIER: 9DBCFDEE
Date/Time:       Sun Jun 13 13:53:06 2010
Sequence Number: 1142
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   errdemon
Description
ERROR LOGGING TURNED ON
Probable Causes
ERRDEMON STARTED AUTOMATICALLY
User Causes
/USR/LIB/ERRDEMON COMMAND
Recommended Actions
NONE
---------------------------------------------------------------------------
LABEL:  CORE_DUMP
IDENTIFIER: A924A5FC
Date/Time:       Sun Jun 13 13:43:19 2010
Sequence Number: 1141
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSPROC
Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED
Probable Causes
SOFTWARE PROGRAM
User Causes
USER GENERATED SIGNAL
Recommended Actions
CORRECT THEN RETRY
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
RERUN THE APPLICATION PROGRAM
IF PROBLEM PERSISTS THEN DO THE FOLLOWING
CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
                344100
FILE SYSTEM SERIAL NUMBER
           8
INODE NUMBER
       11238
CORE FILE NAME
/home/oracle/product/10.2.0/crs/log/node1/cssd/core
PROGRAM NAME
ocssd.bin
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
vmpcopy 6BC
PROCESSOR ID
  hw_fru_id: N/A
  hw_cpu_id: N/A
ADDITIONAL INFORMATION
pthread_k B0
??
Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/ocssd.bin SIG/6 FLDS/pthread_k VALU/b0
---------------------------------------------------------------------------
LABEL:  GS_START_ST
IDENTIFIER: AFA89905
Date/Time:       Sun Jun 13 13:10:37 2010
Sequence Number: 1140
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   grpsvcs
Description
Group Services daemon started
Probable Causes
Daemon started during system startup
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Check that Group Services daemon is running
Detail Data
DETECTING MODULE
RSCT,pgsd.C,1.62.1.8,606                     
ERROR ID
63Y7ej0RwF3A/YGg./2.e.1...................
REFERENCE CODE
                                          
DIAGNOSTIC EXPLANATION
HAGS daemon started by SRC. Log file is /var/ha/log/grpsvcs_trace_1_45.
---------------------------------------------------------------------------
LABEL:  TS_START_ST
IDENTIFIER: 97419D60
Date/Time:       Sun Jun 13 13:10:32 2010
Sequence Number: 1139
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   topsvcs
Description
Topology Services daemon started
Probable Causes
Daemon started during system start-up
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Confirm that this is desirable
Detail Data
DETECTING MODULE
rsct,bootstrp.C,1.213,4440                    
ERROR ID
6UpNEL0MwF3A/zL3./2.e.1...................
REFERENCE CODE
                                          
Topology Services daemon started by:
SRC
Topology Services daemon log file location
/var/ha/log/topsvcs.13.131031.ibm_cls.en_US
Topology Services daemon run directory
/var/ha/run/topsvcs.ibm_cls/
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1138
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172488
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1137
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                192736
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1136
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172486
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
Duplicates
Number of duplicates
           2
Time of first duplicate
Fri Jun  4 17:39:08 2010
Time of last duplicate
Fri Jun  4 17:39:08 2010
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1135
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172486
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1134
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172486
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1133
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                176312
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1132
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                131246
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
Duplicates
Number of duplicates
           2
Time of first duplicate
Fri Jun  4 17:39:08 2010
Time of last duplicate
Fri Jun  4 17:39:08 2010
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1131
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                131246
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1130
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                131246
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  RMCD_INFO_0_ST
IDENTIFIER: A6DF45AA
Date/Time:       Fri Jun  4 17:39:08 2010
Sequence Number: 1129
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   RMCdaemon
Description
The daemon is started.
Probable Causes
The Resource Monitoring and Control daemon has been started.
User Causes
The startsrc -s ctrmc command has been executed or
the rmcctrl -s command has been executed.
Recommended Actions
Confirm that the daemon should be started.
Detail Data
DETECTING MODULE
RSCT,rmcd.c,1.56,213                          
ERROR ID
6eKora0A0M0A/o4Q//2.e.1...................
REFERENCE CODE
                                          
---------------------------------------------------------------------------
LABEL:  TUNE_RESTRICTED
IDENTIFIER: D221BD55
Date/Time:       Fri Jun  4 17:38:56 2010
Sequence Number: 1128
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   perftune
Description
RESTRICTED TUNABLES MODIFIED AT REBOOT
Probable Causes
SYSTEM TUNING
User Causes
TUNABLE PARAMETER OF TYPE RESTRICTED HAS BEEN MODIFIED
Recommended Actions
REVIEW TUNABLE LISTS IN DETAILED DATA
Detail Data
LIST OF TUNABLE COMMANDS CONTROLLING MODIFIED RESTRICTED TUNABLES AT REBOOT, SEE FILE /etc/tunables/lastboot.log
vmo, no
---------------------------------------------------------------------------
LABEL:  DUMP_STATS
IDENTIFIER: 67145A39
Date/Time:       Fri Jun  4 17:38:24 2010
Sequence Number: 1126
Machine Id:      000000000000
Node Id:         localhost
Class:           S
Type:            UNKN
WPAR:            Global
Resource Name:   SYSDUMP
Description
SYSTEM DUMP
Probable Causes
UNEXPECTED SYSTEM HALT
User Causes
SYSTEM DUMP REQUESTED BY USER
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Failure Causes
UNEXPECTED SYSTEM HALT
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
DUMP DEVICE
/dev/lg_dumplv
DUMP SIZE
             283123712
TIME
Fri Jun  4 17:27:20 2010
DUMP TYPE (1 = PRIMARY, 2 = SECONDARY)
           1
DUMP STATUS
           0
ERROR CODE
           0
DUMP INTEGRITY
Compressed dump - Run dmpfmt with -c flag on dump after uncompressing.
FILE NAME
PROCESSOR ID
           0
---------------------------------------------------------------------------
LABEL:  FIRMWARE_EVENT
IDENTIFIER: A6D1BD62
Date/Time:       Fri Jun  4 17:36:49 2010
Sequence Number: 1125
Machine Id:      000000000000
Node Id:         localhost
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   unspecified
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Firmware Event
Detail Data
Event Information
0644 00E3 0000 0088 A600 8E00 0000 0000 0000 0000 4942 4D00 5048 0030 0100 A206
2010 0604 2236 4900 0000 0000 0000 0000 4C00 0004 0000 0000 0000 0000 0000 0000
0000 0083 0000 0000 5548 0018 0100 A206 8303 1080 0000 0000 0000 0000 0000 0000
4D54 001C 0100 A206 3931 3137 2D4D 4D41 3635 3936 4130 3200 0000 0000 4C52 0014
0100 A206 4000 0000 0000 0001 9C00 D000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  MINIDUMP_LOG
IDENTIFIER: F48137AC
Date/Time:       Fri Jun  4 17:36:49 2010
Sequence Number: 1124
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            UNKN
WPAR:            Global
Resource Name:   minidump
Description
COMPRESSED MINIMAL DUMP
Probable Causes
System dumped. Minimal Dump collected in Non-Volatile Memory.
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
Minidump Data:
4D32 0B9B 1E2B 0044 0032 0037 00E4 01DB 0000 0000 10E0 2000 0000 0000 8BB7 0217
0000 0003 4000 0000 4C09 7FD1 2B40 E18F 0165 6E64 0074 6100 0548 190A 318C C5C7
3DC0 0152 5100 0000 6563 DDBA 0108 0040 0180 054C 12F4 4106 A0A1 C387 0E09 384C
00C0 010C 222D 60E0 C808 03C0 0B32 65EC BC60 73E6 0B99 3A6D E0B0 B103 6563 45A4
C090 0120 4080 6D4C 12F4 3906 A0A0 9D86 0082 409C 4871 622A 12A5 F84D C932 058A
9427 432A 5634 4071 0008 7F10 1134 5C61 EE05 9A37 6DCA BC78 2327 CC18 3632 E1C8
7943 A6CE 183A 2F62 C070 21C3 058C 1763 E4CC 79C1 E6CD 9917 6E78 9689 8174 CE1C
3248 6996 11C9 B5AB D7AF 60C3 8A1D 4BB6 6CD9 3763 AC92 7121 268D 1BB3 22ED B481
33E6 0D9C 3C20 6C08 19A2 006E 4310 20D0 DCF9 6246 4E9D 2F69 C8E8 00E1 E445 1005
8005 7F19 03E7 70E2 C58D 1F97 8543 078D 9C32 61C8 7C59 0342 080C 053F 7E28 D0C7
ADA0 C802 1021 744C 4210 E208 B46A D9BA 9DB8 0222 C986 2338 7B06 2D7A CDC4 1410
6101 6563 0024 0400 4090 E090 67AC 022C 18F2 A409 1400 0146 1028 4850 8329 10F0
0ABE A0B7 060C 4501 C2B4 00A2 0831 0204 9204 2716 FCC7 D21F C981 2821 C60C 6020
6604 9248 9410 C948 70A3 1D00 2329 4290 110F 40CD 1438 18C4 0420 E023 0225 417B
CA53 018B A2B7 56BE DE59 C5AA B520 BC23 30C8 6CCD 3AB6 6BC1 0026 8B22 7000 2553
2694 6E97 C25C 8A52 2D80 042A 50B2 FCE7 92E2 DC82 0202 1801 4A72 2845 6811 6A81
A028 0285 1044 1481 50DC 20C0 4E9B 2F6C DEBC 5953 07CE 1732 7256 C8A0 8132 8108
1201 0A14 00C0 9322 1995 7409 2E00 730D 000F 479A 2009 4283 2C28 060A C062 36D1
4481 E4EC DAB7 730B 4AA3 2E28 0229 F050 4846 99C8 386D DBB8 75AB B1E5 7C07 9C13
CE62 5647 F917 C071 ECCA D77C 090A C1FB 8967 249B CC61 671D 7976 416B 0439 878E
A247 6C92 71D4 879E 6E6B BCC2 9E7B D108 985C 2407 94F2 0445 1D80 90C0 44B7 55B2
012C 2F1C C15E 17A0 0030 5100 0744 041B 0022 1041 0220 AD6D E4C6 6280 8514 1541
1B84 40C1 8B00 4022 034A 0A30 5114 0115 0052 C01E 1869 44CF 4F31 B148 9291 FFD1
1540 5135 9900 0D1F 283D 20C4 7449 DE55 2541 E50D B088 181F 0500 0402 51D6 83E3
9532 A104 8198 5596 5710 036A B1E5 CD19 489E F5D2 9634 1AF6 1F11 7852 0900 8B60
BAD2 1445 4344 83C8 5E84 F263 D91C 9F95 C146 1974 9481 C70A 3108 0103 4A49 F002
C1A1 892E DA28 A446 0C41 2930 DCCC 8146 1B66 C851 4619 77CC 01E9 10A4 91F4 0423
A4AC E168 1A74 3C5A 8311 9392 1401 3D70 C88A 07AD 2BE0 E029 490C 5CC3 4E1A 73CC
91C6 192B D420 C3B0 1419 9006 0BCA 9E91 691A 7694 119A 0C39 E080 9201 5460 43E8
5EFB A4B6 5A6B 0529 F067 4136 3021 4B61 F0D4 30E0 2023 BC11 1408 1B9C 8492 965C
16B4 4800 02AC 9B6E 79ED C602 AFBC C941 3208 09B9 DC9B 6F4C FC52 F46F C063 92D4
6E55 42C5 3B6F 09C7 38AC 2F49 11FB 0BB0 C004 2950 0149 1930 53CC C1F3 9EF0 80C7
3105 0A59 C608 6747 480B F938 A7EF 8D3C 1012 800F E290 7248 5028 6CB0 D844 430C
1053 0176 E145 509B 50BC 19A7 D450 4B4D 12D5 00AC F5D0 4C5B 0290 748A F4BC 812E
0021 1F09 72D7 4906 B025 D44B 459C 3449 5134 C3CC 52AD B108 41DD 0421 8037 0018
A468 286B 0519 0936 916F 9317 5302 33A9 7500 41FC 00A0 3804 013C E0B8 51C0 C820
0807 9337 F964 6C49 2BCE F852 090C 9E79 E597 4F8E C001 904B CE24 004E 4249 26E2
9D53 6E39 E68F 479E 39EB 9BCF 4426 E03D 0DC9 F900 0FEC D517 5DA2 3349 3AE6 BBBF
EEB5 D2C9 A3B4 3841 6FF8 E24F AE51 8E1B CDC7 019C 74C9 5213 0910 A244 7417 0403
0215 40F2 918B 6741 600C 241F 7F34 EE5E 3381 7912 2803 2003 66F3 24AD 0E41 7180
1435 C040 1608 0124 5667 025D B80E 7F08 4CA0 02E9 42A8 FF64 6F15 92C0 46FA 8CE1
0838 808E 2212 1800 1D96 4223 1B91 040B 37F0 4841 44C0 9499 1144 8430 12C0 17E0
F006 36B0 6105 D42B 0808 5158 101C 50C0 5CBC 7B1A E5FC 07C0 1020 C138 B0F8 C680
76F3 0BA2 C5E9 3F0B 08E2 10D5 2092 BC41 404F 0411 0030 E801 0AC4 1524 831B A448
1285 9830 4140 A217 EC81 40FB 6418 C224 0980 0ABC C81A 00E6 57BF FB21 5189 5D8C
8422 88A6 AFD5 0006 1829 F8A1 16E1 789F 4B1C A220 0880 C011 E9B2 C521 6EE2 654E
A4DE 6D28 8103 2570 6309 1BEA D044 06F0 BD11 0981 0427 02DB 8ACE D744 4059 031A
1DBC 514C 76D4 2305 E403 0AA2 2B92 2A15 B824 A300 E004 7570 6341 A404 C5D8 5831
4D24 A140 24D0 1445 2FC9 9220 05D0 2345 EC88 CB6F 09D3 814B 6153 D6D8 228E 0C40
4377 7ED9 659D 4419 1B3C 11E1 457C 0280 9F48 22B3 F7FD C350 6D40 1419 14C5 2847
414A 5294 B214 A6C6 A9A9 3270 0A5A 0549 02A8 4445 2A53 A14A 5531 6015 4A5E 15AB
59D5 AA59 B842 C9AE 7AE5 CF60 C193 20C5 3A56 B296 D5AC 677D 6B5A D5BA 56B6 B6D5
AD6F 85CB 9BE5 524D 0E0B 5200 92D5 C007 5220 8904 A050 8801 11E2 09EF 8019 4A74
C94B 004C 8C64 0028 4079 3E1A 0591 92D4 A451 4880 4A73 B94B 898D 0C25 C42C C847
B776 C59B 268C 1053 F8C1 4E29 C252 9F52 8C24 C124 090D D071 CC80 18D5 6654 B017
41F0 3546 8308 CAA6 254D 5821 ECE0 0A9D 1184 678D 38C6 2FF2 E10C 1810 CD68 1E5A
DED2 9A96 17D3 31D3 9924 6926 34A0 A6D7 BC3A 936D 33E9 E9D7 8424 369E B634 7086
4DD3 2E01 BB52 C132 EF4A 7BBB 1B62 B3B6 B7BE 69E4 6F7B CBA1 4F0C F258 BF1C 2E71
595B 5C00 8253 BB12 8CCE 7258 B81D 2C7F C9CD 0178 6EB4 A02B 9EEC 0491 DAD2 AA36
96BB 839D 6B67 5BDB D042 CEB4 AB5B AD5C 7697 CAA5 140E B242 FA89 F150 BB40 BAE8
B6B9 CF03 C01B 7E31 8967 C624 7884 1A06 F5B2 0780 EDD1 8500 95A4 0826 AFA9 4924
81A4 9305 D940 0650 11CA 9810 8094 8068 402E E051 5CB3 A164 90BB 6BA5 0110 5000
4CE0 6229 3075 2E99 CA43 0053 75E9 4B28 9140 4D5F 32E0 984C 60C1 694B 26D4 A081
05EB 32D8 2FA6 9A66 924C 744D 8A64 739B 8012 9437 C129 4E72 36EA 5191 8A21 412A
---------------------------------------------------------------------------
LABEL:  CORE_DUMP
IDENTIFIER: A924A5FC
Date/Time:       Fri Jun  4 17:36:49 2010
Sequence Number: 1123
Machine Id:      000000000000
Node Id:         localhost
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSPROC
Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED
Probable Causes
SOFTWARE PROGRAM
User Causes
USER GENERATED SIGNAL
Recommended Actions
CORRECT THEN RETRY
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
RERUN THE APPLICATION PROGRAM
IF PROBLEM PERSISTS THEN DO THE FOLLOWING
CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
                205054
FILE SYSTEM SERIAL NUMBER
           8
INODE NUMBER
       11238
CORE FILE NAME
/home/oracle/product/10.2.0/crs/log/node1/cssd/core
PROGRAM NAME
ocssd.bin
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
vmpcopy 6BC
PROCESSOR ID
  hw_fru_id: N/A
  hw_cpu_id: N/A
ADDITIONAL INFORMATION
pthread_k B0
??
Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/ocssd.bin SIG/6 FLDS/pthread_k VALU/b0
---------------------------------------------------------------------------
LABEL:  ERRLOG_ON
IDENTIFIER: 9DBCFDEE
Date/Time:       Fri Jun  4 17:38:53 2010
Sequence Number: 1122
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   errdemon
Description
ERROR LOGGING TURNED ON
Probable Causes
ERRDEMON STARTED AUTOMATICALLY
User Causes
/USR/LIB/ERRDEMON COMMAND
Recommended Actions
NONE
---------------------------------------------------------------------------
LABEL:  CORE_DUMP
IDENTIFIER: A924A5FC
Date/Time:       Fri Jun  4 17:27:19 2010
Sequence Number: 1121
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSPROC
Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED
Probable Causes
SOFTWARE PROGRAM
User Causes
USER GENERATED SIGNAL
Recommended Actions
CORRECT THEN RETRY
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
RERUN THE APPLICATION PROGRAM
IF PROBLEM PERSISTS THEN DO THE FOLLOWING
CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
                205054
FILE SYSTEM SERIAL NUMBER
           8
INODE NUMBER
       11238
CORE FILE NAME
/home/oracle/product/10.2.0/crs/log/node1/cssd/core
PROGRAM NAME
ocssd.bin
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
vmpcopy 6BC
PROCESSOR ID
  hw_fru_id: N/A
  hw_cpu_id: N/A
ADDITIONAL INFORMATION
pthread_k B0
??
Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/ocssd.bin SIG/6 FLDS/pthread_k VALU/b0
---------------------------------------------------------------------------
LABEL:  GS_START_ST
IDENTIFIER: AFA89905
Date/Time:       Fri Jun  4 16:54:59 2010
Sequence Number: 1120
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   grpsvcs
Description
Group Services daemon started
Probable Causes
Daemon started during system startup
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Check that Group Services daemon is running
Detail Data
DETECTING MODULE
RSCT,pgsd.C,1.62.1.8,606                     
ERROR ID
63Y7ej0nML0A/cSz./2.e.1...................
REFERENCE CODE
                                          
DIAGNOSTIC EXPLANATION
HAGS daemon started by SRC. Log file is /var/ha/log/grpsvcs_trace_1_44.
---------------------------------------------------------------------------
LABEL:  TS_START_ST
IDENTIFIER: 97419D60
Date/Time:       Fri Jun  4 16:54:55 2010
Sequence Number: 1119
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   topsvcs
Description
Topology Services daemon started
Probable Causes
Daemon started during system start-up
Daemon re-started automatically by SRC
Daemon started during installation
Daemon started manually by user
User Causes
Daemon started manually by user
Recommended Actions
Confirm that this is desirable
Detail Data
DETECTING MODULE
rsct,bootstrp.C,1.213,4440                    
ERROR ID
6UpNEL0jML0A/TNn./2.e.1...................
REFERENCE CODE
                                          
Topology Services daemon started by:
SRC
Topology Services daemon log file location
/var/ha/log/topsvcs.04.165454.ibm_cls.en_US
Topology Services daemon run directory
/var/ha/run/topsvcs.ibm_cls/
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1118
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                147882
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1117
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                151736
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1116
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172214
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
Duplicates
Number of duplicates
           2
Time of first duplicate
Fri Jun  4 11:08:41 2010
Time of last duplicate
Fri Jun  4 11:08:41 2010
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1115
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172214
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1114
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172214
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1113
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                151732
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1112
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172210
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
Duplicates
Number of duplicates
           2
Time of first duplicate
Fri Jun  4 11:08:41 2010
Time of last duplicate
Fri Jun  4 11:08:41 2010
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1111
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172210
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  CONSOLE
IDENTIFIER: 7F88E76D
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1110
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   console
Description
SOFTWARE PROGRAM ERROR
Probable Causes
SOFTWARE PROGRAM
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
REVIEW DETAILED DATA
Detail Data
USER'S PROCESS ID:
                172210
DETECTING MODULE
conwrite
FAILING MODULE
UIO_WRITE
RETURN CODE
           5
ERROR CODE
           0
---------------------------------------------------------------------------
LABEL:  RMCD_INFO_0_ST
IDENTIFIER: A6DF45AA
Date/Time:       Fri Jun  4 11:08:41 2010
Sequence Number: 1109
Machine Id:      000000000000
Node Id:         node1
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   RMCdaemon
Description
The daemon is started.
Probable Causes
The Resource Monitoring and Control daemon has been started.
User Causes
The startsrc -s ctrmc command has been executed or
the rmcctrl -s command has been executed.
Recommended Actions
Confirm that the daemon should be started.
Detail Data
DETECTING MODULE
RSCT,rmcd.c,1.56,213                          
ERROR ID
6eKora07IG0A/THl./2.e.1...................
REFERENCE CODE
                                          
---------------------------------------------------------------------------
LABEL:  TUNE_RESTRICTED
IDENTIFIER: D221BD55
Date/Time:       Fri Jun  4 11:08:28 2010
Sequence Number: 1108
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            INFO
WPAR:            Global
Resource Name:   perftune
Description
RESTRICTED TUNABLES MODIFIED AT REBOOT
Probable Causes
SYSTEM TUNING
User Causes
TUNABLE PARAMETER OF TYPE RESTRICTED HAS BEEN MODIFIED
Recommended Actions
REVIEW TUNABLE LISTS IN DETAILED DATA
Detail Data
LIST OF TUNABLE COMMANDS CONTROLLING MODIFIED RESTRICTED TUNABLES AT REBOOT, SEE FILE /etc/tunables/lastboot.log
vmo, no
---------------------------------------------------------------------------
LABEL:  DUMP_STATS
IDENTIFIER: 67145A39
Date/Time:       Fri Jun  4 11:07:57 2010
Sequence Number: 1106
Machine Id:      000000000000
Node Id:         localhost
Class:           S
Type:            UNKN
WPAR:            Global
Resource Name:   SYSDUMP
Description
SYSTEM DUMP
Probable Causes
UNEXPECTED SYSTEM HALT
User Causes
SYSTEM DUMP REQUESTED BY USER
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Failure Causes
UNEXPECTED SYSTEM HALT
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
DUMP DEVICE
/dev/lg_dumplv
DUMP SIZE
             719266816
TIME
Fri Jun  4 10:56:20 2010
DUMP TYPE (1 = PRIMARY, 2 = SECONDARY)
           1
DUMP STATUS
           0
ERROR CODE
           0
DUMP INTEGRITY
Compressed dump - Run dmpfmt with -c flag on dump after uncompressing.
FILE NAME
PROCESSOR ID
           2
---------------------------------------------------------------------------
LABEL:  FIRMWARE_EVENT
IDENTIFIER: A6D1BD62
Date/Time:       Fri Jun  4 11:06:22 2010
Sequence Number: 1105
Machine Id:      000000000000
Node Id:         localhost
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   unspecified
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Firmware Event
Detail Data
Event Information
0644 00E3 0000 0088 A600 8E00 0000 0000 0000 0000 4942 4D00 5048 0030 0100 A206
2010 0604 1606 2200 0000 0000 0000 0000 4C00 0004 0000 0000 0000 0000 0000 0000
0000 0083 0000 0000 5548 0018 0100 A206 8303 1080 0000 0000 0000 0000 0000 0000
4D54 001C 0100 A206 3931 3137 2D4D 4D41 3635 3936 4130 3200 0000 0000 4C52 0014
0100 A206 4000 0000 0000 0001 9C00 D000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  MINIDUMP_LOG
IDENTIFIER: F48137AC
Date/Time:       Fri Jun  4 11:06:22 2010
Sequence Number: 1104
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            UNKN
WPAR:            Global
Resource Name:   minidump
Description
COMPRESSED MINIMAL DUMP
Probable Causes
System dumped. Minimal Dump collected in Non-Volatile Memory.
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
Minidump Data:
4D32 0D55 2390 0054 0034 0037 00E4 01DB 0000 0000 2ADF 2400 0000 0000 F104 689A
0000 0003 4000 0002 4C09 244F 174A 1DD1 0165 6E64 0074 6100 0348 18C6 428C C5C7
3DC0 0152 5100 0000 6563 DDBA 0108 0040 0180 054C 1288 2001 A0A1 C387 0E09 0010
0020 0100 0730 88B4 8081 6323 0C00 2FC8 94B1 F382 CD99 2F64 EAB4 81C3 C60E 6563
45A4 C090 0120 4080 6D4C 1288 2001 A0A0 9D86 0082 409C 4871 622A 12A5 F84D C932
058A 9427 432A 5634 4071 4018 2610 1134 5C61 EE05 9A37 6DCA BC78 2327 CC18 3632
E1C8 7943 A6CE 183A 2F62 C070 21C3 058C 1763 E4CC 79C1 E6CD 9917 6E78 9689 8174
CE1C 3248 6996 11C9 B5AB D7AF 60C3 8A1D 4BB6 6CD9 3763 AC92 7121 268D 1BB3 22ED
B481 33E6 0D9C 3C20 6C08 19A2 006E 4310 20D0 DCF9 6246 4E9D 2F69 C8E8 00E1 E445
1005 8005 7F19 03E7 70E2 C58D 1F97 8543 078D 9C32 61C8 7C59 0342 080C 053F 7E28
D0C7 ADA0 C802 1021 744C 4210 E208 B46A D9BA 9DB8 0222 C986 2338 7B06 2D7A CDC4
1410 6101 6563 0024 0400 4010 00B4 08BA 6005 5830 E449 1328 0002 8C20 50B0 2200
7805 5FD0 5B03 A622 BC23 38A0 5504 01C0 0D10 8F47 C660 2C48 D207 308B 000E 4082
4993 E6C7 36EC 6AEA 2C08 CF48 2B22 15BD B5F2 F58E 2686 8A2F E4A9 40B7 9320 C91E
F8FE 499D 2A95 E640 9857 9B6A 6509 00AA 5692 763A F24C B952 2B27 585B 0120 8000
C822 0419 150F D622 5990 080A 2124 2A9E 2CB8 4180 9D36 5FD8 BC79 B3A6 0E9C 2F64
E4AC 9041 0366 0211 2402 1428 7011 2619 8A5A 1928 04C0 6350 2B7A EEA4 FC69 0B00
0385 9730 9B68 A260 51F3 C0CE 9FDD 4D51 461A 8114 7828 F65A 4CD4 7A33 6CD0 543E
D5DE 01E7 8433 9ABC B1F6 7EED 1974 153B A421 103F F1CC 6293 3939 2BBA E6DC DC5D
15E1 046D E3EE 9116 409C E5DC 6357 8916 7D7A 34F4 B0EB 1DD2 A5A2 6207 1009 2876
3E75 E7D5 910F D175 010A 0014 0D70 8044 9811 8417 0980 94A5 911B 7411 2480 305A
9056 D010 D640 6321 4190 C045 533C 0456 0088 02F9 40E1 203D 76D4 14E1 48E5 D514
0088 0620 4000 31E4 D424 4079 0994 9715 411B 54A2 4D45 0204 0004 0230 4960 054C
0AE8 48D3 0447 6EB5 A376 2022 E000 14E4 2831 A072 1505 E0E3 8600 BCA5 1511 2410
B121 5D44 BA72 6345 4344 8308 5553 F1F3 D71C 8895 C146 1974 9481 C70A 3108 0103
4C49 F002 C19B 71CE 5927 9E46 0CC1 2730 DCCC 8146 1B66 C851 4619 77CC 81E7 108D
59F4 0423 A4AC 6167 1A74 DC59 8311 7B5A 1401 3D70 688A 07A7 2BE0 6068 6BD7 B093
C61C 73A4 71C6 0A35 C8B0 6A45 06A4 C142 AC67 049A 861D 6528 2643 0E38 C064 0015
D8B0 29D5 3E92 5156 5641 0A0C 6051 0608 CC62 9104 913C 911E 3DF1 F4F2 0D69 206C
0001 4D3D FE58 D022 0108 7066 450A AC4B 10B5 B25C 9BED B6F1 FCF2 05B8 E292 EB63
45E8 AA8B E493 D4C6 22AF B6BF C503 8C2A F88E 0B53 B9FC A6EB 2E41 494E 8B00 5A15
614B 7073 F104 F3AD 53F9 C294 2622 036F 3B4F 01AD D4A6 305C 9D71 70C2 209C DC4B
100A 1B90 44D1 10D2 C254 4094 0024 501F 0051 4E59 E595 05FD DC33 9556 5A24 B45A
53BA B82F CD27 BEF1 2C00 0CC3 B422 D4FB 6AA5 A536 43D7 C430 CD16 45D1 0C33 3595
4517 0462 8757 7669 0EBA 5999 533C 6584 2241 5C63 6951 8E58 4579 6000 24D9 0D41
2029 F00C 8001 C06C D0CD B830 CA48 E34E 34DB 1D00 DE35 2570 628A 3006 3EB8 DF08
1CB0 77DF 85CF 5823 8E00 24FE B7E4 84AB 6539 DF7E C7A8 B98B 2D3E 8EF8 000F 4CE5
4F53 AA47 2EB8 C22D A695 63DC 1141 00C4 0258 D0E4 EE03 CC9C A308 4D74 5BE4 014D
04E4 382E 2807 1802 4EEB 6C82 43E0 8113 5584 4498 209C 0821 9014 72A9 000C 1C70
D921 4D10 8048 8088 0DE4 92BD DB29 D234 35DB B55B F4E2 DF86 E361 23E7 693D 3980
25EE 0639 244C 0D68 1FBB 9404 1307 08B0 294F 2A08 0386 468E 30BC 2F22 F2E3 1F97
BCD4 1430 8989 456A 0180 992C F231 65FD C34D 6D80 1319 E444 273B E149 4F7C F213
A046 28A8 3210 EA56 0549 02A2 14C5 2847 414A 5231 A014 4C2E 95A9 4D75 8A56 A082
C9A8 4AE5 C354 C190 200C 68D5 AB78 452B 5B19 4B57 BCF2 15B0 8445 2C63 21CB 83CC
9A4C 657E E38E 049C 2361 34F9 1E07 B422 002A 1023 7E15 9940 345E B02D 770C 221B
A441 81C2 2853 1001 0023 0548 B088 1AD9 C8C5 4454 273C 1078 604D F6D8 4647 BCC0
64A1 B288 18C5 E7A1 0156 2472 201A C08D DE90 01B1 1484 905C 8C84 CB00 8002 BA3C
E963 7A5C E3B6 DEC1 8957 988C 2028 1B44 2A72 C00C 6EBC 2340 039A 59CD 2C72 B30C
EA8C 200C 74A0 D11C 98CB A935 1004 59B3 0AFF 3A37 80A6 3D6D 7F0F 835F 4190 E924
FE05 1326 CCC4 1D41 BC06 369B 70A5 4B67 43C0 D930 90B6 149D 6D6D 00D0 48FB A459
9004 E6CC 2A8A 1340 E82A B737 F2C8 AE1B B8A4 DF8C ECA7 13CF 552E 5DB4 9B5B EC3E
27B8 78B2 3310 EE94 2731 E869 3B62 BED3 9FA3 0BA8 E906 8ABA F2A8 AE9E AC73 1DEC
D857 3AC9 A1B1 A0B8 0B80 EE78 E73B 9AD0 9196 30B1 01C5 2C92 BC2E 0180 79CE 831E
55A4 5700 EA25 0800 2210 8120 08A1 BD15 4DA8 428A B401 2918 5913 F389 0802 7980
C743 A5E6 BE8B 96B3 7005 0002 03B2 0401 2778 0222 05CD 9F1E 3321 0F20 0909 01CA
F2C7 3FE4 6635 0078 70AB 5BF9 0736 6882 82A1 9903 13EF AB4A 96A8 3AC1 46EA 8408
2210 C298 0862 010D 3EAC 83CA 02A1 0849 58A7 3BE5 2991 3154 6108 0355 C217 1E2A
518B 6AD4 A322 35A9 4A55 8487 A6E2 94A7 8228 2A52 4596 0E46 8449 125D 052B 5935
F188 7F83 A2AC A418 ACC5 54D1 22C7 4A96 B2B2 E82C 987C D022 24C0 0537 2CB2 8018
84A0 8DE3 7800 1861 3201 AA36 CC5F 16F9 473F 608B 0B78 56A4 B6B7 E522 39E4 CAB1
7C5E D2B7 E772 986B 5F57 91D8 7A83 B6B6 6D63 39F4 B0DB A956 35BA C0AD 885A 0B12
DB6F 6037 B9DD 41C7 689A 4B13 501E 37BB BF79 470D 3271 4A00 A04C 108C 6047 0478
C18A 38C6 8C40 C4A4 0902 7076 4BCA 4DE9 AC53 43B0 59D1 6A11 0523 0DAA 58F1 2DD3
DCE6 34EF 1635 8DD0 4520 559F 6961 72EA 046F 6723 DB35 B579 4D6E 6644 6DDF 7CA8
---------------------------------------------------------------------------
LABEL:  CORE_DUMP
IDENTIFIER: A924A5FC
Date/Time:       Fri Jun  4 11:06:22 2010
Sequence Number: 1103
Machine Id:      000000000000
Node Id:         localhost
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSPROC
Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED
Probable Causes
SOFTWARE PROGRAM
User Causes
USER GENERATED SIGNAL
Recommended Actions
CORRECT THEN RETRY
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
RERUN THE APPLICATION PROGRAM
IF PROBLEM PERSISTS THEN DO THE FOLLOWING
CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
                221516
FILE SYSTEM SERIAL NUMBER
           8
INODE NUMBER
       11238
CORE FILE NAME
/home/oracle/product/10.2.0/crs/log/node1/cssd/core
PROGRAM NAME
ocssd.bin
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
vmpcopy 6BC
PROCESSOR ID
  hw_fru_id: N/A
  hw_cpu_id: N/A
ADDITIONAL INFORMATION
pthread_k B0
??
Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/ocssd.bin SIG/6 FLDS/pthread_k VALU/b0
---------------------------------------------------------------------------
LABEL:  ERRLOG_ON
IDENTIFIER: 9DBCFDEE
Date/Time:       Fri Jun  4 11:08:25 2010
Sequence Number: 1102
Machine Id:      000000000000
Node Id:         localhost
Class:           O
Type:            TEMP
WPAR:            Global
Resource Name:   errdemon
Description
ERROR LOGGING TURNED ON
Probable Causes
ERRDEMON STARTED AUTOMATICALLY
User Causes
/USR/LIB/ERRDEMON COMMAND
Recommended Actions
NONE
---------------------------------------------------------------------------
LABEL:  CORE_DUMP
IDENTIFIER: A924A5FC
Date/Time:       Fri Jun  4 10:56:20 2010
Sequence Number: 1101
Machine Id:      000000000000
Node Id:         node1
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSPROC
Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED
Probable Causes
SOFTWARE PROGRAM
User Causes
USER GENERATED SIGNAL
Recommended Actions
CORRECT THEN RETRY
Failure Causes
SOFTWARE PROGRAM
Recommended Actions
RERUN THE APPLICATION PROGRAM
IF PROBLEM PERSISTS THEN DO THE FOLLOWING
CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SIGNAL NUMBER
           6
USER'S PROCESS ID:
                221516
FILE SYSTEM SERIAL NUMBER
           8
INODE NUMBER
       11238
CORE FILE NAME
/home/oracle/product/10.2.0/crs/log/node1/cssd/core
PROGRAM NAME
ocssd.bin
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER
vmpcopy 6BC
PROCESSOR ID
  hw_fru_id: N/A
  hw_cpu_id: N/A
ADDITIONAL INFORMATION
pthread_k B0
??
Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/ocssd.bin SIG/6 FLDS/pthread_k VALU/b0
---------------------------------------------------------------------------
LABEL:  FIRMWARE_EVENT
IDENTIFIER: A6D1BD62
Date/Time:       Fri Apr  9 13:20:52 2010
Sequence Number: 1095
Machine Id:      000000000000
Node Id:         localhost
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   unspecified
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Firmware Event
Detail Data
Event Information
0644 00E3 0000 0088 A600 8E00 0000 0000 0000 0000 4942 4D00 5048 0030 0100 A206
2010 0409 1820 5200 0000 0000 0000 0000 4C00 0004 0000 0000 0000 0000 0000 0000
0000 0083 0000 0000 5548 0018 0100 A206 8303 1080 0000 0000 0000 0000 0000 0000
4D54 001C 0100 A206 3931 3137 2D4D 4D41 3635 3936 4130 3200 0000 0000 4C52 0014
0100 A206 4000 0000 0000 0001 9C00 D000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  FIRMWARE_EVENT
IDENTIFIER: A6D1BD62
Date/Time:       Fri Apr  9 13:02:04 2010
Sequence Number: 1085
Machine Id:      000000000000
Node Id:         localhost
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   unspecified
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Firmware Event
Detail Data
Event Information
0644 00E3 0000 0088 A600 8E00 0000 0000 0000 0000 4942 4D00 5048 0030 0100 A206
2010 0409 1802 0400 0000 0000 0000 0000 4C00 0004 0000 0000 0000 0000 0000 0000
0000 0083 0000 0000 5548 0018 0100 A206 8303 1080 0000 0000 0000 0000 0000 0000
4D54 001C 0100 A206 3931 3137 2D4D 4D41 3635 3936 4130 3200 0000 0000 4C52 0014
0100 A206 4000 0000 0000 0001 9C00 D000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  FIRMWARE_EVENT
IDENTIFIER: A6D1BD62
Date/Time:       Fri Apr  9 12:54:47 2010
Sequence Number: 1067
Machine Id:      000000000000
Node Id:         localhost
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   unspecified
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Firmware Event
Detail Data
Event Information
0644 00E3 0000 0088 A600 8E00 0000 0000 0000 0000 4942 4D00 5048 0030 0100 A206
2010 0409 1754 4700 0000 0000 0000 0000 4C00 0004 0000 0000 0000 0000 0000 0000
0000 0083 0000 0000 5548 0018 0100 A206 8303 1080 0000 0000 0000 0000 0000 0000
4D54 001C 0100 A206 3931 3137 2D4D 4D41 3635 3936 4130 3200 0000 0000 4C52 0014
0100 A206 4000 0000 0000 0001 9C00 D000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Fri Apr  2 17:51:53 2010
Sequence Number: 1062
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Fri Apr  2 17:51:33 2010
Sequence Number: 1061
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  GOENT_RCVRY_EXIT
IDENTIFIER: F3931284
Date/Time:       Fri Apr  2 17:51:14 2010
Sequence Number: 1060
Machine Id:      000000000000
Node Id:         node1
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   ent4
Resource Class:  adapter
Resource Type:   14104003
Location:        U789D.001.DQD92Z4-P1-C1-T1
VPD:            
      2-P NIC-TX PCI-e:
        EC Level....................D76567
        Part Number.................46K6601
        Manufacture ID..............YL1026
        FRU Number..................46K6601
        Network Address.............00145E74BCB0
        ROM Level.(alterable).......EP0140
Description
ETHERNET NETWORK RECOVERY MODE
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
FILE NAME
line: 304 file: goent_intr.c
PCI ETHERNET STATISTICS
0031 38B0 0063 0853 0000 0001 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0142 106D 0000 0000 8743 7612 0000 0000 0132 1ECF 0065 4E2C 0041 C52A
0000 0000 008E E12E 0000 0000 5A23 DBDF 0000 0000 0000 0000 0000 0001 0000 1000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0015 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0008 0343 000C 0060 0C00 0000 0000 01A0 0000 0000
0000 0000 0000 0000 0000 0000 0000 41E1 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000
DEVICE DRIVER INTERNAL STATE
5555 5555 0000 0000 0000 0000
SOURCE ADDRESS
0014 5E74 BCB0
---------------------------------------------------------------------------
LABEL:  HEA_PORT_UP
IDENTIFIER: D6A51BF7
Date/Time:       Fri Apr  2 17:51:06 2010
Sequence Number: 1059
Machine Id:      000000000000
Node Id:         node1
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   ent7
Resource Class:  adapter
Resource Type:   ethernet
Location:        U789D.001.DQD92Z4-P1-C10-T1  
VPD:            
      IBM Host Ethernet Adapter:
        Network Address.............00215E026591
Description
HEA PORT UP
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
FILE NAME
line: 1946 file: hea_prim.c
HEA LOGICAL PORT STATE:
UP  
HEA PHYSICAL PORT STATE:
UP  
SENSE DATA
0031 38A8 0061 081B 0000 0000 0000 0000 0000 0000 0002 8000 0000 0300 0000 0000
0000 0225 D0F1 0000 0002 D9BF EEA2 0000 0000 0221 85A3 0065 21E8 0041 CFE9 0000
0000 015D 86C5 0000 0001 C78F C693 0000 0000 0000 0000 0000 0003 0000 0545 0000
0000 0000 0000 0000 0000 0000 0000 0000 0021 0000 0043 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
DEVICE DRIVER INTERNAL STATE
3333 3333 0000 0000 0000 0000 0000 0000 0000 0000
SOURCE ADDRESS
0021 5E02 6591
---------------------------------------------------------------------------
LABEL:  ECH_MAIN_CHAN_FAIL
IDENTIFIER: 61438179
Date/Time:       Fri Apr  2 17:51:06 2010
Sequence Number: 1058
Machine Id:      000000000000
Node Id:         node1
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   ent9
Resource Class:  adapter
Resource Type:   ibm_ech
Location:        
Description
TOTAL ETHERCHANNEL FAILURE
Probable Causes
CABLE
SWITCH
ADAPTER
Failure Causes
CABLES AND CONNECTIONS
Recommended Actions
CHECK CABLE AND ITS CONNECTIONS
IF ERROR PERSISTS, REPLACE ADAPTER CARD.
Detail Data
Whole EtherChannel is down: all main adapters failed and there is no backup adapter defined
---------------------------------------------------------------------------
LABEL:  GOENT_LINK_DOWN
IDENTIFIER: EC0BCCD4
Date/Time:       Fri Apr  2 17:51:06 2010
Sequence Number: 1057
Machine Id:      000000000000
Node Id:         node1
Class:           H
Type:            TEMP
WPAR:            Global
Resource Name:   ent4
Resource Class:  adapter
Resource Type:   14104003
Location:        U789D.001.DQD92Z4-P1-C1-T1
VPD:            
      2-P NIC-TX PCI-e:
        EC Level....................D76567
        Part Number.................46K6601
        Manufacture ID..............YL1026
        FRU Number..................46K6601
        Network Address.............00145E74BCB0
        ROM Level.(alterable).......EP0140
Description
ETHERNET DOWN
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
FILE NAME
line: 248 file: goent_intr.c
PCI ETHERNET STATISTICS
0031 38A7 0063 081B 0000 0001 0000 0000 0000 0000 0000 0000 0000 0000 0000 0001
0000 0000 0142 106D 0000 0000 8743 7612 0000 0000 0132 1ECF 0065 4E24 0041 C529
0000 0000 008E E12D 0000 0000 5A23 DBA3 0000 0000 0000 0000 0000 0001 0000 1000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0001 0000 0015 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0008 0341 000C 0060 0C00 0000 0000 01A0 0000 0000
0000 0000 0000 0000 0000 0000 0000 41E1 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000
DEVICE DRIVER INTERNAL STATE
2222 2222 0000 0000 0000 0000
SOURCE ADDRESS
0014 5E74 BCB0
Diagnostic Analysis
Diagnostic Log sequence number: 4020
Resource tested: ent4
Menu Number:  2538902
Description:

No trouble was found with this adapter.  However
Error Log Analysis indicates that there recently may
have been a network problem.
If your Ethernet adapter is connected to a network,
and if you are experiencing problems with network
communications, check for a loose or defective
cable or connection.
If a switch or another system is directly attached
to the Ethernet adapter, verify it is powered up,
configured, and functioning correctly.
---------------------------------------------------------------------------
LABEL:  HEA_PORT_DOWN
IDENTIFIER: 38A8D4BB
Date/Time:       Fri Apr  2 17:50:58 2010
Sequence Number: 1056
Machine Id:      000000000000
Node Id:         node1
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   ent7
Resource Class:  adapter
Resource Type:   ethernet
Location:        U789D.001.DQD92Z4-P1-C10-T1  
VPD:            
      IBM Host Ethernet Adapter:
        Network Address.............00215E026591
Description
HEA PORT DOWN
Recommended Actions
PERFORM PROBLEM DETERMINATION PROCEDURES
Detail Data
FILE NAME
line: 1940 file: hea_prim.c
HEA LOGICAL PORT STATE:
UP  
HEA PHYSICAL PORT STATE:
DOWN
SENSE DATA
0031 38A0 0261 0853 0000 0000 0000 0000 0000 0000 0002 8000 0000 0300 0000 0000
0000 0225 D0E5 0000 0002 D9BF CCFC 0000 0000 0221 8597 0065 21E7 0041 CFE9 0000
0000 015D 86BA 0000 0001 C78F B906 0000 0000 0000 0000 0000 0003 0000 0544 0000
0000 0000 0000 0000 0000 0000 0000 0000 0028 0000 0043 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
DEVICE DRIVER INTERNAL STATE
3333 3333 0000 0000 0000 0000 0000 0000 0000 0000
SOURCE ADDRESS
0021 5E02 6591
---------------------------------------------------------------------------
LABEL:  ECH_MAIN_CHAN_FAIL
IDENTIFIER: 61438179
Date/Time:       Fri Apr  2 17:50:56 2010
Sequence Number: 1055
Machine Id:      000000000000
Node Id:         node1
Class:           H
Type:            INFO
WPAR:            Global
Resource Name:   ent8
Resource Class:  adapter
Resource Type:   ibm_ech
Location:        
Description
TOTAL ETHERCHANNEL FAILURE
Probable Causes
CABLE
SWITCH
ADAPTER
Failure Causes
CABLES AND CONNECTIONS
Recommended Actions
CHECK CABLE AND ITS CONNECTIONS
IF ERROR PERSISTS, REPLACE ADAPTER CARD.
Detail Data
Whole EtherChannel is down: all main adapters failed and there is no backup adapter defined
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Jan  7 12:52:56 2010
Sequence Number: 1032
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Jan  7 12:52:39 2010
Sequence Number: 1031
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_CHILDGONE
IDENTIFIER: 4B219AEA
Date/Time:       Wed Aug 12 10:52:55 2009
Sequence Number: 966
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Concurrent LVM daemon forced Volume Group offline
Probable Causes
Unrecoverable event detected by Concurrent LVM daemon
Failure Causes
Lost communication with remote nodes
Lost quorum
Recommended Actions
Ensure Cluster daemons are running
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Attempt to bring the Concurrent Volume Group back online
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_NOENV
IDENTIFIER: 9BD08D55
Date/Time:       Wed Aug 12 10:52:55 2009
Sequence Number: 965
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Unable to start gsclvmd
Probable Causes
Unable to establish communication with Cluster daemons
Failure Causes
Unable to establish communication with Cluster daemons
Recommended Actions
Ensure Cluster daemons are running
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_CHILDGONE
IDENTIFIER: 4B219AEA
Date/Time:       Wed Aug 12 10:52:52 2009
Sequence Number: 963
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Concurrent LVM daemon forced Volume Group offline
Probable Causes
Unrecoverable event detected by Concurrent LVM daemon
Failure Causes
Lost communication with remote nodes
Lost quorum
Recommended Actions
Ensure Cluster daemons are running
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Attempt to bring the Concurrent Volume Group back online
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_NOENV
IDENTIFIER: 9BD08D55
Date/Time:       Wed Aug 12 10:52:52 2009
Sequence Number: 962
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Unable to start gsclvmd
Probable Causes
Unable to establish communication with Cluster daemons
Failure Causes
Unable to establish communication with Cluster daemons
Recommended Actions
Ensure Cluster daemons are running
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_CONNECTIVITY
IDENTIFIER: DB14100E
Date/Time:       Wed Aug 12 10:52:50 2009
Sequence Number: 957
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Group Services detected a failure
Probable Causes
Unable to establish communication with Cluster daemons
Failure Causes
Concurrent Volume Group forced offline
Recommended Actions
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Tue Apr 21 09:41:06 2009
Sequence Number: 834
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Tue Apr 21 09:40:55 2009
Sequence Number: 833
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  TS_HALOCAL_ER
IDENTIFIER: E60E3D52
Date/Time:       Thu Apr 16 16:54:02 2009
Sequence Number: 793
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   topsvcs
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Local adapters missing in configuration file
Probable Causes
Local adapter information missing in ODM
Local adapter information incorrect in ODM
Install Causes
Local adapter information missing in ODM
Local adapter information incorrect in ODM
Recommended Actions
Enter local adapter information in ODM
Verify and correct local adapter information in ODM
Detail Data
DETECTING MODULE
rsct,refresh.C,1.82,4798                     
ERROR ID
6GpX1a1uXut7/8gJ0/2.e.1...................
REFERENCE CODE
                                          
---------------------------------------------------------------------------
LABEL:  LVM_GS_CHILDGONE
IDENTIFIER: 4B219AEA
Date/Time:       Thu Apr 16 15:09:36 2009
Sequence Number: 781
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Concurrent LVM daemon forced Volume Group offline
Probable Causes
Unrecoverable event detected by Concurrent LVM daemon
Failure Causes
Lost communication with remote nodes
Lost quorum
Recommended Actions
Ensure Cluster daemons are running
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Attempt to bring the Concurrent Volume Group back online
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_CHILDGONE
IDENTIFIER: 4B219AEA
Date/Time:       Thu Apr 16 15:09:35 2009
Sequence Number: 780
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Concurrent LVM daemon forced Volume Group offline
Probable Causes
Unrecoverable event detected by Concurrent LVM daemon
Failure Causes
Lost communication with remote nodes
Lost quorum
Recommended Actions
Ensure Cluster daemons are running
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Attempt to bring the Concurrent Volume Group back online
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_NOENV
IDENTIFIER: 9BD08D55
Date/Time:       Thu Apr 16 15:09:35 2009
Sequence Number: 779
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Unable to start gsclvmd
Probable Causes
Unable to establish communication with Cluster daemons
Failure Causes
Unable to establish communication with Cluster daemons
Recommended Actions
Ensure Cluster daemons are running
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_CHILDGONE
IDENTIFIER: 4B219AEA
Date/Time:       Thu Apr 16 15:09:35 2009
Sequence Number: 777
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Concurrent LVM daemon forced Volume Group offline
Probable Causes
Unrecoverable event detected by Concurrent LVM daemon
Failure Causes
Lost communication with remote nodes
Lost quorum
Recommended Actions
Ensure Cluster daemons are running
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Attempt to bring the Concurrent Volume Group back online
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_NOENV
IDENTIFIER: 9BD08D55
Date/Time:       Thu Apr 16 15:09:35 2009
Sequence Number: 776
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            INFO
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Unable to start gsclvmd
Probable Causes
Unable to establish communication with Cluster daemons
Failure Causes
Unable to establish communication with Cluster daemons
Recommended Actions
Ensure Cluster daemons are running
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_CONNECTIVITY
IDENTIFIER: DB14100E
Date/Time:       Thu Apr 16 15:09:35 2009
Sequence Number: 772
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Group Services detected a failure
Probable Causes
Unable to establish communication with Cluster daemons
Failure Causes
Concurrent Volume Group forced offline
Recommended Actions
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 14:54:04 2009
Sequence Number: 768
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 14:53:53 2009
Sequence Number: 767
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 14:24:14 2009
Sequence Number: 732
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 14:23:58 2009
Sequence Number: 731
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 14:07:34 2009
Sequence Number: 656
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 14:07:24 2009
Sequence Number: 655
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 13:19:31 2009
Sequence Number: 640
Machine Id:      000000000000
Node Id:         localhost
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 13:11:47 2009
Sequence Number: 638
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 09:26:26 2009
Sequence Number: 508
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 09:26:08 2009
Sequence Number: 507
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 09:15:26 2009
Sequence Number: 500
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Thu Apr 16 09:15:09 2009
Sequence Number: 499
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Sun Mar 22 12:02:28 2009
Sequence Number: 422
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  TS_DIFF_INST_NUM_ER
IDENTIFIER: 4E8FCF52
Date/Time:       Fri Mar 20 14:09:23 2009
Sequence Number: 330
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   topsvcs
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Message received with different instance number
Topology Services being shutdown to avoid false network partition.
Probable Causes
A node is running with a different version of the configuration
User Causes
Cluster was started on a node during a Topology DARE operation
A node exists with an outdated version of the HACMP configuration
A node was down during a Topology DARE operation
Recommended Actions
Locate all nodes with an outdated version of the cluster configuration
Run a Topology DARE to propagate configuration to all the nodes
Restart cluster on nodes that were down during Topology DARE
Detail Data
DETECTING MODULE
rsct,comm.C,1.148,2398                        
ERROR ID
6GxwXC/Xbyk7/hcN//6.e.1...................
REFERENCE CODE
                                          
Message instance number
           6
Recipient's instance number
           7
Originator of message
(192.168.2.2:0x41c33156)
---------------------------------------------------------------------------
LABEL:  LVM_GS_CONNECTIVITY
IDENTIFIER: DB14100E
Date/Time:       Tue Mar 17 01:58:35 2009
Sequence Number: 239
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            PERM
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Group Services detected a failure
Probable Causes
Unable to establish communication with Cluster daemons
Failure Causes
Concurrent Volume Group forced offline
Recommended Actions
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE
Detail Data
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Tue Mar 17 01:58:35 2009
Sequence Number: 235
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Tue Mar 17 01:58:33 2009
Sequence Number: 234
Machine Id:      000000000000
Node Id:         node1
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Tue Mar  3 02:08:21 2009
Sequence Number: 181
Machine Id:      000000000000
Node Id:         localhost
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
---------------------------------------------------------------------------
LABEL:  LVM_GS_RLEAVE
IDENTIFIER: AB59ABFF
Date/Time:       Tue Mar  3 02:08:11 2009
Sequence Number: 180
Machine Id:      000000000000
Node Id:         localhost
Class:           U
Type:            UNKN
WPAR:            Global
Resource Name:   LIBLVM
Resource Class:  NONE
Resource Type:   NONE
Location:        
Description
Remote node Concurrent Volume Group failure detected
Probable Causes
Remote node Concurrent Volume Group forced offline
Failure Causes
Remote node left VGSA/VGDA groups due to failure
Recommended Actions
Examine error log on identified remote node
Detail Data
Remote Node Name
node2
Volume Group ID
00C9 6A02 0000 4C00 0000 011E 1EF7 47A3
MAJOR/MINOR DEVICE NUMBER
0028 0000
SENSE DATA
0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000收起