情况描述如下:在2015/1/5号早上9点10几分的时候,正在跑批时,数据库出现自动宕机,我看日志是在2015-01-05-09.14分时候最后有个日志是
db2pd -temptable finished,看到以前的日志,每次自动宕机最后一条日志都是db2pd -temptable finished,
这时大约在(9:25的时候去重新去启动了数据库服务,启动了之后这次却发现数据库无法连接了(之前出现宕机时重启服务一般会自动好了),连接时提示数据库状态不一致,报出了SQL1015N的错误信息。
然后用命令LIST UTILITIES 查发现进度一直处于0%,等了数小时(超过10小时)还一直是这个状态。(要是平时数据库在自动恢复也至少也会进度的)
日志很多看不太懂,请专家帮忙分析一下原因,及有什么好的解决方案能让数据库恢复正常,感激不尽!
环境说明:
操作系统:Microsoft(R)Windows(R) Server 2003 Enterprise
x64 Edition
系统类型:x64-basedPC
数据库版本:DB2 v9.7 32位
下面是打出的最新db2dialog.log信息:
2015-01-05-09.12.18.359000+480 I42056869H544 LEVEL: Warning
PID : 396 TID : 1936 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-59 APPID: 32.221.32.89.1263.150104094754
AUTHID : PAS
EDUID : 1936 EDUNAME: db2lrid 0
FUNCTION: DB2 UDB, database utilities, sqlulPrintPhaseMsg, probe:314
DATA #1 : String, 103 bytes
LOADID: 4000.2015-01-05-09.12.17.093005.0 (4;34)
Starting BUILD phase at 2015-01-05 09:12:19.045436.
2015-01-05-09.12.18.390000+480 I42057415H545 LEVEL: Warning
PID : 396 TID : 1936 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-59 APPID: 32.221.32.89.1263.150104094754
AUTHID : PAS
EDUID : 1936 EDUNAME: db2lrid 0
FUNCTION: DB2 UDB, database utilities, sqlulPrintPhaseMsg, probe:314
DATA #1 : String, 104 bytes
LOADID: 4000.2015-01-05-09.12.17.093005.0 (4;34)
Completed BUILD phase at 2015-01-05 09:12:19.088127.
2015-01-05-09.12.18.937000+480 E42057962H493 LEVEL: Warning
PID : 396 TID : 4000 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-59 APPID: 32.221.32.89.1263.150104094754
AUTHID : PAS
EDUID : 4000 EDUNAME: db2agent (JRPAS) 0
FUNCTION: DB2 UDB, data management, sqldBeginIndexCreate, probe:1
MESSAGE : ADM5540W Rebuilding "1" indexes on table "PAS .JKSJ_HCMX".
2015-01-05-09.12.18.937000+480 E42058457H554 LEVEL: Warning
PID : 396 TID : 4000 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-59 APPID: 32.221.32.89.1263.150104094754
AUTHID : PAS
EDUID : 4000 EDUNAME: db2agent (JRPAS) 0
FUNCTION: DB2 UDB, data management, sqldIndexCreate, probe:1
MESSAGE : ADM5541W Rebuilding index with IID "1" in object with ID "393" and
table space ID "8" on table "PAS .JKSJ_HCMX".
2015-01-05-09.12.19.093000+480 E42059013H513 LEVEL: Warning
PID : 396 TID : 4000 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-59 APPID: 32.221.32.89.1263.150104094754
AUTHID : PAS
EDUID : 4000 EDUNAME: db2agent (JRPAS) 0
FUNCTION: DB2 UDB, data management, sqldEndIndexCreate, probe:1
MESSAGE : ADM5542W Indexes on table "PAS .JKSJ_HCMX" are successfully
rebuilt.
2015-01-05-09.12.19.093000+480 E42059528H636 LEVEL: Warning
PID : 396 TID : 4000 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-59 APPID: 32.221.32.89.1263.150104094754
AUTHID : PAS
EDUID : 4000 EDUNAME: db2agent (JRPAS) 0
FUNCTION: DB2 UDB, data management, sqldEndNoLogList, probe:1
MESSAGE : ADM5530W The COMMIT processing of table "PAS .JKSJ_HCMX" that
used NOT LOGGED INITIALLY has been initiated. It is recommended that
you take a backup of this table's table space(s).
2015-01-05-09.12.19.109000+480 I42060166H484 LEVEL: Warning
PID : 396 TID : 4000 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-59 APPID: 32.221.32.89.1263.150104094754
AUTHID : PAS
EDUID : 4000 EDUNAME: db2agent (JRPAS) 0
FUNCTION: DB2 UDB, database utilities, sqluvtld_route_in, probe:841
DATA #1 :
Starting LOAD operation (S) (1) (I).
2015-01-05-09.14.14.796000+480 I42060652H336 LEVEL: Event
PID : 396 TID : 592 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 592 EDUNAME: db2pdbe 0
FUNCTION: DB2 UDB, RAS/PD component, pdDumpTempTableStats, probe:3485
START : db2pd -temptable started
2015-01-05-09.14.14.812000+480 I42060990H337 LEVEL: Event
PID : 396 TID : 592 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 592 EDUNAME: db2pdbe 0
FUNCTION: DB2 UDB, RAS/PD component, pdDumpTempTableStats, probe:3594
START : db2pd -temptable finished
2015-01-05-09.25.22.843000+480 I42061329H327 LEVEL: Event
PID : 3096 TID : 3172 PROC : DB2START.EXE
INSTANCE: DB2 NODE : 000
EDUID : 3172
FUNCTION: DB2 UDB, base sys utilities, sqleIssueStartStop, probe:21
DATA #1 :
Single node instance [0] is starting
2015-01-05-09.25.22.906000+480 I42061658H1553 LEVEL: Event
PID : 3096 TID : 3172 PROC : DB2START.EXE
INSTANCE: DB2 NODE : 000
EDUID : 3172
FUNCTION: DB2 UDB, base sys utilities, sqleStartStopSingleNode, probe:1130
DATA #1 : String, 39 bytes
D:PROGRA~1IBMSQLLIBbinDB2STAR2.EXE
DATA #2 : Hexdump, 256 bytes
0x00127634 : 443A 5C50 524F 4752 417E 315C 4942 4D5C D:PROGRA~1IBM
0x00127644 : 5351 4C4C 4942 5C62 696E 5C44 4232 5354 SQLLIBbinDB2ST
0x00127654 : 4152 322E 4558 4500 4E4F 4D53 4700 0000 AR2.EXE.NOMSG...
0x00127664 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x00127674 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x00127684 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x00127694 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x001276A4 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x001276B4 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x001276C4 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x001276D4 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x001276E4 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x001276F4 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x00127704 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x00127714 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
0x00127724 : 0000 0000 0000 0000 0000 0000 0000 0000 ................
2015-01-05-09.25.24.156000+480 I42063213H547 LEVEL: Event
PID : 2880 TID : 2656 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2656 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqkfBufferManager::initBufferManager, probe:10
START : Starting FCM Buffer Manager
1 resource group(s).
4224 buffers allocated.
Maximum number of buffers supported is 8192.
Total buffer manager memory required for 1 logical and 0 fail-over partition(s) is 18481164 bytes.
2015-01-05-09.25.24.171000+480 I42063762H551 LEVEL: Event
PID : 2880 TID : 2656 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2656 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqkfChannelManager::initChannelManager, probe:10
START : Starting FCM Channel Manager
1 resource group(s).
128 channels allocated.
Maximum number of channels supported is 8192.
Total channel manager memory required for 1 logical and 0 fail-over partitions(s) is 409616 bytes.
2015-01-05-09.25.24.171000+480 I42064315H550 LEVEL: Event
PID : 2880 TID : 2656 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2656 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqkfSessionManager::initSessionManager, probe:10
START : Starting FCM Session Manager
1 resource group(s).
128 sessions allocated.
Maximum number of sessions supported is 64000.
Total session manager memory required for 1 logical and 0 fail-over partitions is 1396736 bytes.
2015-01-05-09.25.24.171000+480 I42064867H428 LEVEL: Event
PID : 2880 TID : 2656 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2656 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqlkf_init_allocate_shared, probe:500
START : FCM infrastructure started
FCM running in non-shared mode; FCM parallelism degree: 1; comm protocol: TCP/IPv4
2015-01-05-09.25.24.250000+480 I42065297H386 LEVEL: Severe
PID : 2880 TID : 2656 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2656 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, base sys utilities, sqleInitSysCtlr, probe:66
DATA #1 : Hexdump, 4 bytes
0x0320FC20 : 4C00 0000 L...
2015-01-05-09.25.24.250000+480 E42065685H856 LEVEL: Critical
PID : 2880 TID : 2276 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2276 EDUNAME: db2ipccm 0
FUNCTION: DB2 UDB, base sys utilities, sqle_panic, probe:10
MESSAGE : ADM14001C An unexpected and critical error has occurred: "Panic".
The instance may have been shutdown as a result. "Automatic" FODC
(First Occurrence Data Capture) has been invoked and diagnostic
information has been recorded in directory
"C:DOCUME~1ALLUSE~1APPLIC~1IBMDB2DB2COPY1DB2FODC_Panic_2015-0
1-05-09.25.24.250000". Please look in this directory for detailed
evidence about what happened and contact IBM support if necessary to
diagnose the problem.
2015-01-05-09.25.24.250000+480 I42066543H382 LEVEL: Severe
PID : 2880 TID : 2656 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2656 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, base sys utilities, sqleSysCtlr, probe:15
DATA #1 : Hexdump, 4 bytes
0x0320FC50 : 4C00 0000 L...
2015-01-05-09.25.24.296000+480 I42066927H420 LEVEL: Severe
PID : 2880 TID : 2276 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2276 EDUNAME: db2ipccm 0
FUNCTION: DB2 UDB, base sys utilities, sqle_panic, probe:10
MESSAGE : sqle_panic: Panic/sleep =
DATA #1 : Hexdump, 4 bytes
0xFB10019C : 0000 0000 ....
2015-01-05-09.25.24.296000+480 I42067349H391 LEVEL: Severe
PID : 2880 TID : 2276 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2276 EDUNAME: db2ipccm 0
FUNCTION: DB2 UDB, base sys utilities, sqleCleanupResources, probe:5
DATA #1 : Hexdump, 4 bytes
0x0371F83C : 0000 0000 ....
2015-01-05-09.25.24.296000+480 I42067742H467 LEVEL: Severe
PID : 2880 TID : 2276 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2276 EDUNAME: db2ipccm 0
FUNCTION: DB2 UDB, base sys utilities, sqleCleanupResources, probe:10
DATA #1 : Hexdump, 20 bytes
0x004181C8 : 496E 7374 616E 6365 206E 6F74 2061 6374 Instance not act
0x004181D8 : 6976 6500 ive.
2015-01-05-09.25.24.296000+480 I42068211H351 LEVEL: Severe
PID : 2880 TID : 2276 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2276 EDUNAME: db2ipccm 0
FUNCTION: DB2 UDB, base sys utilities, sqleCleanupResources, probe:6
MESSAGE : Cleaning up ipc resources and exiting.
2015-01-05-09.25.24.515000+480 I42068564H436 LEVEL: Error
PID : 3096 TID : 3172 PROC : DB2START.EXE
INSTANCE: DB2 NODE : 000
EDUID : 3172
FUNCTION: DB2 UDB, oper system services, sqloStartDB2Service, probe:50
DATA #2 : Hexdump, 25 bytes
0x00126A20 : 4442 322D 3020 3A20 BDF8 B3CC D2E2 CDE2 DB2-0 : ........
0x00126A30 : D6D5 D6B9 A1A3 0D0A 0A .........
2015-01-05-09.25.24.515000+480 I42069002H439 LEVEL: Severe
PID : 3096 TID : 3172 PROC : DB2START.EXE
INSTANCE: DB2 NODE : 000
EDUID : 3172
FUNCTION: DB2 UDB, base sys utilities, sqleStartStopSingleNode, probe:70
DATA #1 : Hexdump, 25 bytes
0x00126FF4 : 4442 322D 3020 3A20 BDF8 B3CC D2E2 CDE2 DB2-0 : ........
0x00127004 : D6D5 D6B9 A1A3 0D0A 0A .........
2015-01-05-09.25.24.515000+480 I42069443H320 LEVEL: Event
PID : 3096 TID : 3172 PROC : DB2START.EXE
INSTANCE: DB2 NODE : 000
EDUID : 3172
FUNCTION: DB2 UDB, base sys utilities, sqleIssueStartStop, probe:190
DATA #1 :
Single node stopped [-1032].
2015-01-05-09.31.04.500000+480 I42069765H547 LEVEL: Event
PID : 392 TID : 2148 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2148 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqkfBufferManager::initBufferManager, probe:10
START : Starting FCM Buffer Manager
1 resource group(s).
4224 buffers allocated.
Maximum number of buffers supported is 8192.
Total buffer manager memory required for 1 logical and 0 fail-over partition(s) is 18481164 bytes.
2015-01-05-09.31.04.593000+480 I42070314H551 LEVEL: Event
PID : 392 TID : 2148 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2148 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqkfChannelManager::initChannelManager, probe:10
START : Starting FCM Channel Manager
1 resource group(s).
128 channels allocated.
Maximum number of channels supported is 8192.
Total channel manager memory required for 1 logical and 0 fail-over partitions(s) is 409616 bytes.
2015-01-05-09.31.04.609000+480 I42070867H550 LEVEL: Event
PID : 392 TID : 2148 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2148 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqkfSessionManager::initSessionManager, probe:10
START : Starting FCM Session Manager
1 resource group(s).
128 sessions allocated.
Maximum number of sessions supported is 64000.
Total session manager memory required for 1 logical and 0 fail-over partitions is 1396736 bytes.
2015-01-05-09.31.04.609000+480 I42071419H428 LEVEL: Event
PID : 392 TID : 2148 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 2148 EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, fast comm manager, sqlkf_init_allocate_shared, probe:500
START : FCM infrastructure started
FCM running in non-shared mode; FCM parallelism degree: 1; comm protocol: TCP/IPv4
2015-01-05-09.31.06.453000+480 E42071849H327 LEVEL: Event
PID : 392 TID : 772 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000
EDUID : 772
FUNCTION: DB2 UDB, base sys utilities, DB2StartMain, probe:911
MESSAGE : ADM7513W Database manager has started.
START : DB2 DBM
2015-01-05-09.44.40.843000+480 I42072178H517 LEVEL: Event
PID : 392 TID : 2328 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-33 APPID: 32.221.32.89.1486.150105014440
AUTHID : PAS
EDUID : 2328 EDUNAME: db2agent (JRPAS) 0
FUNCTION: DB2 UDB, config/install, sqlfLogUpdateCfgParam, probe:20
CHANGE : CFG DB JRPAS : "Database_memory" From: "818920" To: "786432"
2015-01-05-09.44.42.062000+480 I42072697H448 LEVEL: Warning
PID : 392 TID : 2328 PROC : db2syscs.exe
INSTANCE: DB2 NODE : 000 DB : JRPAS
APPHDL : 0-33 APPID: 32.221.32.89.1486.150105014440
AUTHID : PAS
EDUID : 2328 EDUNAME: db2agent (JRPAS) 0
FUNCTION: DB2 UDB, base sys utilities, sqledint, probe:30
MESSAGE : Crash Recovery is needed.
2015-01-05-09.44.45.453000+480 E42073147H478 LEVEL: Event
PID