服务器宕机原因分析

请专家帮分析一下AIX服务器突然宕机的原因,以下为日志,多谢

LABEL:          GS_START_ST

IDENTIFIER:     AFA89905

Date/Time:       Thu Jun 16 15:21:28 BEIST 2016

Sequence Number: 4461

Machine Id:      00F765834C00

Node Id:         trade2

Class:           O

Type:            INFO

WPAR:            Global

Resource Name:   grpsvcs         

Description

Group Services daemon started

Probable Causes

Daemon started during system startup

Daemon re-started automatically by SRC

Daemon started during installation

Daemon started manually by user

User Causes

Daemon started manually by user

        Recommended Actions

        Check that Group Services daemon is running

Detail Data

DETECTING MODULE

RSCT,pgsd.C,1.62.1.23,695                     

ERROR ID

63Y7ej0sBZML/Vnd.Ag.e.1...................

REFERENCE CODE

                                          

DIAGNOSTIC EXPLANATION

HAGS daemon started by SRC. Log file is /var/ha/log/grpsvcs_trace.

---------------------------------------------------------------------------

LABEL:          TS_START_ST

IDENTIFIER:     97419D60

Date/Time:       Thu Jun 16 15:21:26 BEIST 2016

Sequence Number: 4460

Machine Id:      00F765834C00

Node Id:         trade2

Class:           O

Type:            INFO

WPAR:            Global

Resource Name:   topsvcs         

Description

Topology Services daemon started

Probable Causes

Daemon started during system start-up

Daemon re-started automatically by SRC

Daemon started during installation

Daemon started manually by user

User Causes

Daemon started manually by user

        Recommended Actions

        Confirm that this is desirable

Detail Data

DETECTING MODULE

rsct,bootstrp.C,1.215.1.13,4956               

ERROR ID

6UpNEL0qBZML/8On0Ag.e.1...................

REFERENCE CODE

                                          

Topology Services daemon started by:

SRC

Topology Services daemon log file location

/var/ha/log/topsvcs.16.152126.tradecluster.en_US

Topology Services daemon run directory

/var/ha/run/topsvcs.tradecluster/

---------------------------------------------------------------------------

LABEL:          RMCD_INFO_0_ST

IDENTIFIER:     A6DF45AA

Date/Time:       Thu Jun 16 15:20:35 BEIST 2016

Sequence Number: 4459

Machine Id:      00F765834C00

Node Id:         trade2

Class:           O

Type:            INFO

WPAR:            Global

Resource Name:   RMCdaemon      

Description

The daemon is started.

Probable Causes

The Resource Monitoring and Control daemon has been started.

User Causes

The startsrc -s ctrmc command has been executed or

the rmcctrl -s command has been executed.

        Recommended Actions

        Confirm that the daemon should be started.

Detail Data

DETECTING MODULE

RSCT,rmcd.c,1.79,230                          

ERROR ID

6eKora01BZML/YgD/Ag.e.1...................

REFERENCE CODE

                                          

---------------------------------------------------------------------------

LABEL:          DUMP_STATS

IDENTIFIER:     67145A39

Date/Time:       Thu Jun 16 15:17:41 BEIST 2016

Sequence Number: 4457

Machine Id:      00F765834C00

Node Id:         trade2

Class:           S

Type:            UNKN

WPAR:            Global

Resource Name:   SYSDUMP         

Description

SYSTEM DUMP

Probable Causes

UNEXPECTED SYSTEM HALT

User Causes

SYSTEM DUMP REQUESTED BY USER

        Recommended Actions

        PERFORM PROBLEM DETERMINATION PROCEDURES

Failure Causes

UNEXPECTED SYSTEM HALT

        Recommended Actions

        PERFORM PROBLEM DETERMINATION PROCEDURES

Detail Data

DUMP DEVICE

/dev/lg_dumplv

DUMP SIZE

             649056768

TIME

Thu Jun 16 15:11:12 2016

DUMP TYPE (1 = PRIMARY, 2 = SECONDARY)

           1

DUMP STATUS

           0

ERROR CODE

0000 0000 0000 0000

DUMP INTEGRITY

Compressed dump - Run dmpfmt with -c flag on dump after uncompressing.

FILE NAME

PROCESSOR ID

           0

---------------------------------------------------------------------------

LABEL:          MINIDUMP_LOG

IDENTIFIER:     F48137AC

Date/Time:       Thu Jun 16 15:16:30 BEIST 2016

Sequence Number: 4456

Machine Id:      00F765834C00

Node Id:         trade2

Class:           O

Type:            UNKN

WPAR:            Global

Resource Name:   minidump        

Description

COMPRESSED MINIMAL DUMP

Probable Causes

System dumped. Minimal Dump collected in Non-Volatile Memory.

        Recommended Actions

        PERFORM PROBLEM DETERMINATION PROCEDURES

Detail Data

Minidump Data:

4D33 0F5C 2B84 0054 0026 0026 003B 0058 0000 0000 26AF D200 0000 0000 F045 D435

0000 0003 0008 5762 51EA 081B 03C2 6575 DDDD 0002 0003 0000 000A 000C 5762 5110

0000 0000 000E 0000 2F64 6576 2F6C 675F 6475 6D70 6C76 6563 45A4 C090 0120 0000

2C57 C444 8100 A021 8D86 1023 4A9C 98A0 4E37 3641 8414 7142 64A2 4477 EE1A 6A3B

0108 46C4 0166 000C 9248 6511 1A40 1105 4092 0165 6300 2408 0040 9060 805F 2002

4418 F2A4 0914 0001 4614 2808 6080 1900 83E2 0140 5001 D688 1A40 280A B895 0580

3B77 04B5 9D00 0483 A245 8C14 6316 7C39 C8E5 C58C 001E 0048 8047 E64C 9F01 080E

F409 8090 8810 4434 2A08 1AEC 2150 91BF 086A E4E8 B107 807F 58B3 62F5 A940 26A0

985A 9FCA 44E0 932C 80A1 0407 105D CBB6 AD54 004B 0134 05A0 D1C1 C678 A820 7C2D

2800 920C 8D04 81C5 C002 8A22 89A3 4428 8220 48F6 99DA 9895 B004 49F3 06CA 9B39

7486 BC69 0387 4D19 3A69 56CC C041 B7A0 602C D862 0668 02C7 CC1C 3370 BE90 79E3

A6CC 0A19 3168 C424 D0CA 0235 3294 8324 7932 BBF6 8A1C A463 DAD8 840F F89B E144

68DB 8E01 2339 451B 9D90 A471 43A7 8C1C 3761 D87C A15C 7CFA 0DDD 3131 DCA1 415E

FA97 3766 CCB0 5981 A365 FA3A 80D2 7C99 039F 4E9B 3764 AC40 C30C 43C8 8441 2B5C

C897 061A 5FD4 01DA 7C32 D460 5D41 1BF0 C144 0005 1800 003C 4A05 25CC 1C7B A575

1300 3F78 7147 01A1 8CC0 C95E 1070 5118 03AA 2164 134C 259E 188A 0906 EC85 0004

041D 4051 0094 0080 C58C 35FD 28A3 4C45 B245 1389 26A2 6842 277B A110 0130 6F51

2582 3B8B B5B5 648D 28BE 700B 8B5D 14D6 C054 1D5D 09C2 4904 5982 0204 1751 2403

3DB4 10C9 A48D 2F0C B317 0661 E624 678D 0698 0089 3D14 7500 4202 6A71 19CA 0DC1

E8D8 0104 3E06 3083 8F3E 91E9 510E 318D 5492 2AA8 A8A2 A9A6 A860 054F 4EFD 10E4

29A8 32C1 D196 A423 58E5 D66E 32FD C30F 3F3F 68D5 2100 8604 2996 5001 0041 814F

3C16 F440 A895 E68A 0251 F200 F055 4842 6AF4 8009 9830 9007 3891 C644 0915 3D51

0488 052E 10F5 2941 2FD0 C3C4 8404 FD0A 0001 AB06 C557 0048 9855 902B D886 4890

5F31 09A1 0CB4 0521 2202 2124 2846 10B9 143D 300A 3D31 F90B F010 FCC6 A42E 450A

0056 002C 11F0 02C1 AC28 DC42 1410 4A09 1000 09C5 4051 D75D A894 46D5 0837 404B

30AA 37B8 6B6C 04F8 C4E4 C017 1D17 4440 0F60 8042 321C 8808 2048 BE3E 1130 432E

B074 0848 3209 0C49 7059 448F C595 4F14 C365 3109 C4AC EA32 0023 0360 1702 78FD

98C1 2169 64A9 AFD3 6F29 B074 C65C 1334 F458 B37E 08C8 01A6 12E0 CC2F 85B9 CA43

AC5A F1F3 B6AC 0415 00CC 0224 E002 94C2 7793 C0E3 D831 250C 5700 802C 1252 C500

E0C2 A3DB 706F 2593 E04B 0512 0D37 80F0 8DF7 C375 33CC CBB0 8083 2577 E3FF 70AD

80AB FF80 AEB0 E6C3 3E0E D500 2977 0E51 1374 144D 110D C81C 4D11 257E 3825 5454

618B 5D30 4507 FF14 13ED A483 2E6D EE50 F5EE 3B51 9282 D08B 4FD5 C6F4 E9C6 0130

01CD AA0B D85C 824F 45EC 8440 4413 F9F4 4056 F3FC D38F A97C 91D4 7B88 1210 1209

08EC 1782 40CA CAD7 CFD6 AC0A 4012 9337 ADF8 F28E 4F02 500A 0020 C082 3DB8 2B00

0470 8034 28B2 034D 2883 28E6 2A48 2578 90BA D9D5 4E26 115C 8B00 7EC1 0212 284B

013C 1A41 F434 9834 A8B0 A067 70D1 D002 0A03 15AD 7925 26F0 6A0B 0908 0088 C410

0504 8F29 0802 8C41 0838 C8E1 0D63 2803 77E4 9087 1550 4726 25B0 0134 30A4 2178

44C9 85C0 E321 D724 4091 0508 6555 2300 C011 E654 8052 DC00 0C51 EA55 F8AC 4514

2BBA 258B 5BE4 5229 70F0 8C3B F50A 5942 A945 0E30 1113 3472 B114 3A18 E38E A0C8

163B AA91 07F6 D011 04EC 53C5 2B3A 8D8F 3EF1 6393 4AF1 8332 EC05 04BD 2203 898A

C080 14EC C306 1B50 14A3 08A2 80EF 4984 28E3 C34A F9CE 2712 F585 8D7D EE83 5F41

2450 0806 D0CF 7EB0 8C09 FE6C 5510 FEF9 2F67 0224 E037 4C36 0F72 6880 810E 8420

6428 2813 DADD 4A83 BF60 8207 7302 4200 9060 8444 1100 10EA 51A9 64A2 5001 2A84

4535 E1F8 428A C490 2D33 ACE1 5A70 6830 1EFA 1088 42A4 0311 8D48 C882 2471 8919

DAD0 1389 B243 424C B190 6759 55BE 9270 C71D CC20 8C75 EBA6 4CCC 2843 00F0 F38F

0F70 2341 B829 0039 D2D1 3006 BD23 0FDE A043 0820 9228 FBBC A30F 5421 C876 12C4

8C68 71CB 4563 9251 3502 E11D 8F8C E424 2BD9 0F37 A841 9390 EA9C 2DFF 57CB FED1

9420 33A5 484E 6B7A 4B9E DE14 003B C5A9 4D01 B000 4F8E 3160 E433 1FFA 8462 4AAE

A1F2 7DF1 83C0 2B63 4955 FC3D D4A7 B884 CB00 59B0 0493 D1C3 086D 2248 031F 784C

004C B082 0531 2606 DDB2 4136 2CF3 01CD 2C01 3401 0884 7C54 930D D754 2195 A002

86B5 98EC 9B6B 09A7 0D7D 42CE 28F6 F087 411C 6211 8F18 9377 3251 9E04 41C1 45EB

79CF 8280 7455 DB5B C21D 7B90 0780 02E0 A804 3199 6531 0B00 CDAA D107 F821 0806

DE28 1287 36B6 B477 FCC1 0B2A 3A52 24C2 568D 4108 4147 076A C855 D5D6 9DB7 5DE4

107E 9052 8248 F207 4BB0 060F 8480 093C C094 200C 302A 2893 4ACA F495 E494 AB4C

65FC C830 55AA C212 7F9C D8DF 50A3 1517 3550 C35D 07A8 0217 0401 4CB2 AE95 2267

2DE6 0595 B741 46BC B599 2698 AB4C A409 2CBE FC82 1179 25EA F5A0 22DA 82FC 5506

6E09 27CE 089B 43C6 9813 B1E9 5C27 6329 E2D8 783A 31B2 9395 A2D3 A838 DA7C BAC5

0400 68C2 1D83 F002 CF82 D658 655C 1588 45AC 4621 3C42 A100 60A8 6B29 B2E2 3B0E

A182 7BE4 5A8D D588 843F EC36 2697 3DA4 8A43 7CC7 2538 A0B8 0038 AE17 E8F0 0654

9462 1960 2A0C B964 3A5E A1F6 D4CA 3F0D 2A50 ABBC E52B 7739 CB43 6D80 74C5 47DD

A59E A5A9 4E7B AA2A 09C2 4A34 74D7 BBF5 C31F 0BB1 2A13 02E4 9205 4D38 E000 D681

8EF6 0A13 BEC4 1CDE 7C7F C4D6 5FB0 E2BE 3C3A 817E 2B15 840C 9EC5 D001 5E00 BDFC

3B67 81BE 0BC1 32A4 E160 6552 581D 3E18 9D8A 6527 1295 F8D8 0B03 40B2 F4D4 F0AA

38FC D1DE B2E5 0400 78C2 1D95 4006 135B 1A9F 6D81 B5AC D5B8 8420 C058 C673 8C89

---------------------------------------------------------------------------

LABEL:          KERNEL_ABEND

IDENTIFIER:     0975DD6C

Date/Time:       Thu Jun 16 15:16:30 BEIST 2016

Sequence Number: 4455

Machine Id:      00F765834C00

Node Id:         trade2

Class:           S

Type:            PERM

WPAR:            Global

Resource Name:   ABEND           

Description

KERNEL ABNORMALLY TERMINATED

Probable Causes

SOFTWARE PROGRAM

Failure Causes

SOFTWARE PROGRAM

        Recommended Actions

        OBTAIN DUMP

        CONTACT APPROPRIATE SERVICE REPRESENTATIVE

Detail Data

ABEND CODE

EEEE 0000 DA27 8030

ABEND DETAIL

0000 0000 0366 0083

ABEND CALLER

0000 0000 0054 8B68

MACHINE STATE REGISTER, MSR

8000 0000 0002 9032

---------------------------------------------------------------------------

LABEL:          ERRLOG_ON

IDENTIFIER:     9DBCFDEE

Date/Time:       Thu Jun 16 15:19:58 BEIST 2016

Sequence Number: 4454

Machine Id:      00F765834C00

Node Id:         trade2

Class:           O

Type:            TEMP

WPAR:            Global

Resource Name:   errdemon        

Description

ERROR LOGGING TURNED ON

Probable Causes

ERRDEMON STARTED AUTOMATICALLY

User Causes

/USR/LIB/ERRDEMON COMMAND

        Recommended Actions

        NONE

---------------------------------------------------------------------------

LABEL:          SC_DISK_PCM_ERR4

IDENTIFIER:     0C10BB8C

Date/Time:       Thu Jun 16 04:04:29 BEIST 2016

Sequence Number: 4453

Machine Id:      00F765834C00

Node Id:         trade2

Class:           H

Type:            INFO

WPAR:            Global

Resource Name:   hdisk4         

Resource Class:  disk

Resource Type:   mpioapdisk

Location:        U78AA.001.WZSGV8F-P1-C5-T1-W20250080E518F458-L2000000000000

VPD:            

        Manufacturer................IBM     

        Machine Type and Model......1814      FAStT

        ROS Level and ID............31303630

        Serial Number...............

        Device Specific.(Z0)........0000053245005032

        Device Specific.(Z1)........

Description

ARRAY CONFIGURATION CHANGED

Probable Causes

ARRAY CONTROLLER

CABLES AND CONNECTIONS

Failure Causes

ARRAY CONTROLLER

CABLES AND CONNECTIONS

        Recommended Actions

        NO ACTION NECESSARY

Detail Data

PATH ID

           1

SENSE DATA

0600 0308 0000 FF04 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 7000 0600

0000 009E 0000 0000 3FC6 0600 0000 0000 0000 0000 0000 5524 0000 0000 0000 0000

0008 5000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 5351 3133 3030 3036

3030 2020 2020 2020 0777 1800 0002 0000 0700 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0005 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000

0000 0000 0000 0000 0000 45EC C99D 3036 3135 3136 2F31 3235 3635 3200 0000 0000

0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 01F4

8541 0080 0080

---------------------------------------------------------------------------

LABEL:          SC_DISK_PCM_ERR4

IDENTIFIER:     0C10BB8C

参与6

1同行回答

zcomezcome  系统工程师 , zzz
这种一般是bug了,得看dump数据。显示全部

这种一般是bug了,得看dump数据。

收起
IT咨询服务 · 2016-06-17
浏览3359

提问者

fhf0702
技术支持zdinfo
擅长领域: 服务器AIX日志分析

相关问题

相关资料

相关文章

问题状态

  • 发布时间:2016-06-17
  • 关注会员:3 人
  • 问题浏览:4831
  • 最近回答:2016-06-17
  • X社区推广