能源采矿AIXOracle

AIX swap耗尽导致oracle实例down

早上发现数据库实例无法访问,ping 数据库服务器网络正常,无法telnet登陆进去。通过管理口进去重启没相应,强制重启后进入系统检查如下:# errptIDENTIFIER TIMESTAMP  T C RESOURCE_NAME  DESCRIPTION
AFA89905   0830100915 I O grpsvcs        Group Services daemon started
97419D60   0830100915 I O topsvcs        Topology Services daemon started
A6DF45AA   0830100915 I O RMCdaemon      The daemon is started.
BE0A03E5   0830100515 P H sysplanar0     ENVIRONMENTAL PROBLEM
9DBCFDEE   0830100715 T O errdemon       ERROR LOGGING TURNED ON
BE0A03E5   0830094715 P H sysplanar0     ENVIRONMENTAL PROBLEM
C5C09FFA   0830033915 P S SYSVMM         SOFTWARE PROGRAM ABNORMALLY TERMINATED
C5C09FFA   0830033915 P S SYSVMM         SOFTWARE PROGRAM ABNORMALLY TERMINATED
C5C09FFA   0830033915 P S SYSVMM         SOFTWARE PROGRAM ABNORMALLY TERMINATED
C5C09FFA   0830033915 P S SYSVMM         SOFTWARE PROGRAM ABNORMALLY TERMINATED
C5C09FFA   0830033915 P S SYSVMM         SOFTWARE PROGRAM ABNORMALLY TERMINATED
C5C09FFA   0830033915 P S SYSVMM         SOFTWARE PROGRAM ABNORMALLY TERMINATED
E18E984F   0830032415 P S SRC            SOFTWARE PROGRAM ERROR
CB4A951F   0830032415 I S SRC            SOFTWARE PROGRAM ERROR
AA8AB241   0830032215 T O clstrmgrDANGER OPERATOR NOTIFICATION
AA8AB241   0801021415 T O clstrmgrES     OPERATOR NOTIFICATION
51E537B5   0729003315 P H sysplanar0     platform_dump saved to file
A6D1BD62   0729003215 I H unspecified    Firmware Event
291D64C3   0729003115 I H sysplanar0     Platform dump data
3C81E43F   0724030315 P U topsvcs        Late in sending heartbeat
3C81E43F   0714030315 P U topsvcs        Late in sending heartbeat
3C81E43F   0627030215 P U topsvcs        Late in sending heartbeat


然后# errpt -aj C5C09FFA|pg 查看错误
---------------------------------------------------------------------------
LABEL:          PGSP_KILL
IDENTIFIER:     C5C09FFA


Date/Time:       Sun Aug 30 03:39:33 CST 2015
Sequence Number: 876
Machine Id:      00F75D6A4C00
Node Id:         calldb2
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSVMM         


Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED


Probable Causes
SYSTEM RUNNING OUT OF PAGING SPACE


Failure Causes
INSUFFICIENT PAGING SPACE DEFINED FOR THE SYSTEM
PROGRAM USING EXCESSIVE AMOUNT OF PAGING SPACE


        Recommended Actions
        DEFINE ADDITIONAL PAGING SPACE
        REDUCE PAGING SPACE REQUIREMENTS OF PROGRAM(S)


Detail Data
PROGRAM
expdp
USER'S PROCESS ID:
              29098094
PROGRAM'S PAGING SPACE USE IN 1KB BLOCKS
        2408
---------------------------------------------------------------------------
LABEL:          PGSP_KILL
IDENTIFIER:     C5C09FFA


Date/Time:       Sun Aug 30 03:39:33 CST 2015
Sequence Number: 875
Machine Id:      00F75D6A4C00
Node Id:         calldb2
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSVMM         


Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED


Probable Causes
SYSTEM RUNNING OUT OF PAGING SPACE


Failure Causes
INSUFFICIENT PAGING SPACE DEFINED FOR THE SYSTEM
PROGRAM USING EXCESSIVE AMOUNT OF PAGING SPACE


        Recommended Actions
        DEFINE ADDITIONAL PAGING SPACE
        REDUCE PAGING SPACE REQUIREMENTS OF PROGRAM(S)


Detail Data
PROGRAM
oracle
USER'S PROCESS ID:
              25559232
PROGRAM'S PAGING SPACE USE IN 1KB BLOCKS
        3668
---------------------------------------------------------------------------
LABEL:          PGSP_KILL
IDENTIFIER:     C5C09FFA


Date/Time:       Sun Aug 30 03:39:33 CST 2015
Sequence Number: 874
Machine Id:      00F75D6A4C00
Node Id:         calldb2
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSVMM         


Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED


Probable Causes
SYSTEM RUNNING OUT OF PAGING SPACE


Failure Causes
INSUFFICIENT PAGING SPACE DEFINED FOR THE SYSTEM
PROGRAM USING EXCESSIVE AMOUNT OF PAGING SPACE


        Recommended Actions
        DEFINE ADDITIONAL PAGING SPACE
        REDUCE PAGING SPACE REQUIREMENTS OF PROGRAM(S)


Detail Data
PROGRAM
oracle
USER'S PROCESS ID:
              27000900
PROGRAM'S PAGING SPACE USE IN 1KB BLOCKS
        9616
---------------------------------------------------------------------------
LABEL:          PGSP_KILL
IDENTIFIER:     C5C09FFA


Date/Time:       Sun Aug 30 03:39:33 CST 2015
Sequence Number: 873
Machine Id:      00F75D6A4C00
Node Id:         calldb2
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSVMM         


Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED


Probable Causes
SYSTEM RUNNING OUT OF PAGING SPACE


Failure Causes
INSUFFICIENT PAGING SPACE DEFINED FOR THE SYSTEM
PROGRAM USING EXCESSIVE AMOUNT OF PAGING SPACE


        Recommended Actions
        DEFINE ADDITIONAL PAGING SPACE
        REDUCE PAGING SPACE REQUIREMENTS OF PROGRAM(S)


Detail Data
PROGRAM
oracle
USER'S PROCESS ID:
              30408938
PROGRAM'S PAGING SPACE USE IN 1KB BLOCKS
       40336
---------------------------------------------------------------------------
LABEL:          PGSP_KILL
IDENTIFIER:     C5C09FFA


Date/Time:       Sun Aug 30 03:39:33 CST 2015
Sequence Number: 872
Machine Id:      00F75D6A4C00
Node Id:         calldb2
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSVMM         


Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED


Probable Causes
SYSTEM RUNNING OUT OF PAGING SPACE


Failure Causes
INSUFFICIENT PAGING SPACE DEFINED FOR THE SYSTEM
PROGRAM USING EXCESSIVE AMOUNT OF PAGING SPACE


        Recommended Actions
        DEFINE ADDITIONAL PAGING SPACE
        REDUCE PAGING SPACE REQUIREMENTS OF PROGRAM(S)


Detail Data
PROGRAM
oracle
USER'S PROCESS ID:
              33947854
PROGRAM'S PAGING SPACE USE IN 1KB BLOCKS
        1536
---------------------------------------------------------------------------
LABEL:          PGSP_KILL
IDENTIFIER:     C5C09FFA


Date/Time:       Sun Aug 30 03:39:33 CST 2015
Sequence Number: 871
Machine Id:      00F75D6A4C00
Node Id:         calldb2
Class:           S
Type:            PERM
WPAR:            Global
Resource Name:   SYSVMM         


Description
SOFTWARE PROGRAM ABNORMALLY TERMINATED


Probable Causes
SYSTEM RUNNING OUT OF PAGING SPACE


Failure Causes
INSUFFICIENT PAGING SPACE DEFINED FOR THE SYSTEM
PROGRAM USING EXCESSIVE AMOUNT OF PAGING SPACE


        Recommended Actions
        DEFINE ADDITIONAL PAGING SPACE
        REDUCE PAGING SPACE REQUIREMENTS OF PROGRAM(S)


Detail Data
PROGRAM
oracle
USER'S PROCESS ID:
              29491234
PROGRAM'S PAGING SPACE USE IN 1KB BLOCKS
           0
#
提示oracle 进程导致swap耗尽。。各位有谁能给予解决思路?
参与7

6同行回答

bayunchubayunchu  网络工程师 , gas
swap现在已经为64GB了。另外一台就没有这个情况发生。显示全部
swap现在已经为64GB了。另外一台就没有这个情况发生。收起
能源采矿 · 2015-09-09
浏览6621
huangrq_cnhuangrq_cn  存储架构师 , 中投科信
一个是oracle可能有bug,第二是扩大swap的空间。显示全部
一个是oracle可能有bug,第二是扩大swap的空间。收起
互联网服务 · 2015-09-08
浏览5007
张泽翰张泽翰  系统工程师 , 北京南天
这个估计需要收集snap包发到IBM分析解决了。显示全部
这个估计需要收集snap包发到IBM分析解决了。收起
系统集成 · 2015-08-31
浏览4906
challenge1984challenge1984  软件开发工程师 , great.com
限制系统swap使用显示全部
限制系统swap使用收起
互联网服务 · 2015-08-30
浏览4963
myciciymyciciy  IT顾问 , 某金融科技公司
查系统VMO配置参数,限制系统swap使用zhangcc326 发表于 2015-8-30 14:45     治标不治本显示全部
查系统VMO配置参数,限制系统swap使用
zhangcc326 发表于 2015-8-30 14:45



    治标不治本收起
银行 · 2015-08-30
浏览4871
zhangcc326zhangcc326  系统运维工程师 , 南京银行股份有限公司
查系统VMO配置参数,限制系统swap使用显示全部
查系统VMO配置参数,限制系统swap使用收起
银行 · 2015-08-30
浏览5125

提问者

bayunchu
网络工程师gas

相关问题

相关资料

相关文章

问题状态

  • 发布时间:2015-08-30
  • 关注会员:1 人
  • 问题浏览:12438
  • 最近回答:2015-09-09
  • X社区推广