hacmp报错

AIX 5.3-06-02 hacmp 5.4 sp01

操作系统报如下错：
# errpt
40E9A4E1   0327134814 P S SYSPROC        SOFTWARE PROGRAM ABNORMALLY TERMINATED
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326204214 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked
3D32B80D   0326200614 P S topsvcs        NIM thread blocked

# errpt -aj 3D32B80D
---------------------------------------------------------------------------
LABEL:          TS_NIM_ERROR_STUCK_
IDENTIFIER:     3D32B80D

Date/Time:       Wed Mar 26 20:42:10 2014
Sequence Number: 134055
Machine Id:      00034641D600
Node Id:         oa_server1
Class:           S
Type:            PERM
Resource Name:   topsvcs

Description
NIM thread blocked

Probable Causes
A thread in a Topology Services Network Interface Module (NIM) process
was blocked
Topology Services NIM process cannot get timely access to CPU

User Causes
Excessive memory consumption is causing high memory contention
Excessive disk I/O is causing high memory contention

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Failure Causes
Excessive virtual memory activity prevents NIM from making progress
Excessive disk I/O traffic is interfering with paging I/O

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Detail Data
DETECTING MODULE
rsct,nim_control.C,1.39.1.18,5919            
ERROR ID
6BUfAx.WggAH/TvS/8c.e.1...................
REFERENCE CODE

Thread which was blocked
receive thread
Interval in seconds during which process was blocked
          38
Interface name
rhdisk27
---------------------------------------------------------------------------
LABEL:          TS_NIM_ERROR_STUCK_
IDENTIFIER:     3D32B80D

Date/Time:       Wed Mar 26 20:42:10 2014
Sequence Number: 134054
Machine Id:      00034641D600
Node Id:         oa_server1
Class:           S
Type:            PERM
Resource Name:   topsvcs

Description
NIM thread blocked

Probable Causes
A thread in a Topology Services Network Interface Module (NIM) process
was blocked
Topology Services NIM process cannot get timely access to CPU

User Causes
Excessive memory consumption is causing high memory contention
Excessive disk I/O is causing high memory contention

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Failure Causes
Excessive virtual memory activity prevents NIM from making progress
Excessive disk I/O traffic is interfering with paging I/O

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Detail Data
DETECTING MODULE
rsct,nim_control.C,1.39.1.18,5919            
ERROR ID
6BUfAx.WggAH/hcE/8c.e.1...................
REFERENCE CODE

Thread which was blocked
receive thread
Interval in seconds during which process was blocked
          38
Interface name
en3
---------------------------------------------------------------------------
LABEL:          TS_NIM_ERROR_STUCK_
IDENTIFIER:     3D32B80D

Date/Time:       Wed Mar 26 20:42:10 2014
Sequence Number: 134053
Machine Id:      00034641D600
Node Id:         oa_server1
Class:           S
Type:            PERM
Resource Name:   topsvcs

Description
NIM thread blocked

Probable Causes
A thread in a Topology Services Network Interface Module (NIM) process
was blocked
Topology Services NIM process cannot get timely access to CPU

User Causes
Excessive memory consumption is causing high memory contention
Excessive disk I/O is causing high memory contention

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Failure Causes
Excessive virtual memory activity prevents NIM from making progress
Excessive disk I/O traffic is interfering with paging I/O

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Detail Data
DETECTING MODULE
rsct,nim_control.C,1.39.1.18,5919            
ERROR ID
6BUfAx.WggAH/Xy9/8c.e.1...................
REFERENCE CODE

Thread which was blocked
receive thread
Interval in seconds during which process was blocked
          38
Interface name
en0
---------------------------------------------------------------------------
LABEL:          TS_NIM_ERROR_STUCK_
IDENTIFIER:     3D32B80D

Date/Time:       Wed Mar 26 20:42:09 2014
Sequence Number: 134052
Machine Id:      00034641D600
Node Id:         oa_server1
Class:           S
Type:            PERM
Resource Name:   topsvcs

Description
NIM thread blocked

Probable Causes
A thread in a Topology Services Network Interface Module (NIM) process
was blocked
Topology Services NIM process cannot get timely access to CPU

User Causes
Excessive memory consumption is causing high memory contention
Excessive disk I/O is causing high memory contention

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Failure Causes
Excessive virtual memory activity prevents NIM from making progress
Excessive disk I/O traffic is interfering with paging I/O

        Recommended Actions
        Examine I/O and memory activity on the system
        Reduce load on the system
        Tune virtual memory parameters
        Call IBM Service if problem persists

Detail Data
DETECTING MODULE
rsct,nim_control.C,1.39.1.18,5919            
ERROR ID
6BUfAx.VggAH/awW18c.e.1...................
REFERENCE CODE

Thread which was blocked
send thread
Interval in seconds during which process was blocked
          37
Interface name
en3
#
## errpt -aj 40E9A4E1
---------------------------------------------------------------------------
LABEL:          CORE_DUMP
IDENTIFIER:     40E9A4E1

Date/Time:       Thu Mar 27 13:48:09 2014
Sequence Number: 134056
Machine Id:      00034641D600
Node Id:         oa_server1
Class:           S
Type:            PERM
Resource Name:   SYSPROC

Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED

Probable Causes
SOFTWARE PROGRAM

User Causes
USER GENERATED SIGNAL

        Recommended Actions
        CORRECT THEN RETRY

Failure Causes
SOFTWARE PROGRAM

        Recommended Actions
        RERUN THE APPLICATION PROGRAM
        IF PROBLEM PERSISTS THEN DO THE FOLLOWING
        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data
SIGNAL NUMBER
           4
USER'S PROCESS ID:
                290842
FILE SYSTEM SERIAL NUMBER
          12
INODE NUMBER
        4842
PROCESSOR ID
           7
CORE FILE NAME
/domino/swep01mad/core
PROGRAM NAME
http
STACK EXECUTION DISABLED
           0
COME FROM ADDRESS REGISTER

ADDITIONAL INFORMATION
??
gcHelper 324

Symptom Data
REPORTABLE
1
INTERNAL ERROR
0
SYMPTOM CODE
PCSS/SPI2 FLDS/http SIG/4 FLDS/gcHelper VALU/324
#
麻烦各位大神帮忙分析，不胜感激！收起