IDS9.30宕机求助,信息详尽,不断更新

RS6000小机,操作系统AIX53,数据库和应用在一台服务器上,数据库链接方式是共享内存
有批量和联机应用,数据库每晚都会重启。 请各位帮忙诊断下,谢谢各位
online.log:
09:58:53  Maximum server connections 95
10:03:06  Logical Log 305974 Complete.
10:03:09  Process exited with return code 142: /bin/sh /bin/sh -c /informix/etc/log_full.sh 2 23 "Logical Log 305974 Complete." "Logical Log 305974 Complete."
10:03:53  Fuzzy Checkpoint Completed:  duration was 0 seconds, 433 buffers not flushed.
10:03:53  Checkpoint loguniq 305975, logpos 0xf4dd4

10:03:53  Maximum server connections 100
10:08:54  Fuzzy Checkpoint Completed:  duration was 0 seconds, 428 buffers not flushed.
10:08:54  Checkpoint loguniq 305975, logpos 0x856dac

10:08:54  Maximum server connections 100
10:10:01  Logical Log 305975 Complete.
10:10:04  Process exited with return code 142: /bin/sh /bin/sh -c /informix/etc/log_full.sh 2 23 "Logical Log 305975 Complete." "Logical Log 305975 Complete."
10:13:54  Fuzzy Checkpoint Completed:  duration was 0 seconds, 418 buffers not flushed.
10:13:54  Checkpoint loguniq 305976, logpos 0x55dd4c

10:13:54  Maximum server connections 100
10:16:57  Logical Log 305976 Complete.
10:17:00  Process exited with return code 142: /bin/sh /bin/sh -c /informix/etc/log_full.sh 2 23 "Logical Log 305976 Complete." "Logical Log 305976 Complete."
10:17:13  Assert Failed: No Exception Handler
10:17:13  Informix Dynamic Server Version 9.30.FC2R1   
10:17:13   Who: Session(102, agent2@BRCC_MAIN_P650, 1344510, -2143274408)
                Thread(125, sqlexec, 7000000803d0288, 5)
                File: mtex.c Line: 415
10:17:13   Results: Exception Caught. Type: MT_EX_OS, Context: mem
10:17:13   Action: Please notify Informix Technical Support.
10:17:13  stack trace for pid 1541092 written to /informix/tmp/af.465c1a8
10:17:13   See Also: /informix/tmp/af.465c1a8, shmem.465c1a8.0
10:18:54  Fuzzy Checkpoint Completed:  duration was 0 seconds, 273 buffers not flushed.
10:18:54  Checkpoint loguniq 305977, logpos 0x2758d4

10:18:54  Maximum server connections 100
10:19:44  mtex.c, line 415, thread 125, proc id 1541092, No Exception Handler.
10:19:44  Fatal error in ADM VP at mt.c:11890
10:19:44  Unexpected virtual processor termination, pid = 1541092, exit = 0x100

10:19:44  PANIC: Attempting to bring system down
10:24:41  Informix Dynamic Server Started.


帖写我认为有用的信息:
LAST SQL的锁等待时间是20S 太长
Sess  SQL            Current            Iso Lock       SQL  ISAM F.E.
Id    Stmt type      Database           Lvl Mode       ERR  ERR  Vers
102   -              agentdb            CR  Wait 20    0    0    9.03

onstat -g ath
*125     7000000d06e6900  7000000803d0288  2    running                 5cpu        sqlexec

onstat  -u
7000000803d0288 *---P--- 102      agent2   -        0                0    1     76       922

附件:

附件图标af.rar (188.83 KB)

附件图标af20140214.rar (190.55 KB)

参与8

8同行回答

hujinqianhujinqian软件开发工程师华润万家
回复 4# liaosnet 廖神怎么判断出来是bug的?显示全部
回复 4# liaosnet


廖神怎么判断出来是bug的?收起
互联网服务 · 2014-02-18
浏览2172
pooh81pooh81项目经理农信社
回复 2# liaosnet 谢谢,虽然对bug不是很理解,很佩服,很感激显示全部
回复 2# liaosnet
谢谢,虽然对bug不是很理解,很佩服,很感激收起
银行 · 2014-02-17
浏览2252
pooh81pooh81项目经理农信社
又宕了一次,{:2_26:} 还能帮忙看看么显示全部

又宕了一次,{:2_26:} 还能帮忙看看么收起
银行 · 2014-02-17
浏览2204
liaosnetliaosnet信息分析/架构师gbasedbt.com
回复 4# pooh81     BUG的话,只有升级。显示全部
回复 4# pooh81


    BUG的话,只有升级。收起
IT咨询服务 · 2014-02-15
浏览2167
pooh81pooh81项目经理农信社
回复 3# liaosnet 谢谢,触发这个Bug的原因是不是:一个带游标的SQL在做insert或Update,同时又有一个SQL去读insert或update的内容?除了升级数据库,还有什么办法能避免出现这个Bug呢?显示全部
回复 3# liaosnet

谢谢,触发这个Bug的原因是不是:
一个带游标的SQL在做insert或Update,同时又有一个SQL去读insert或update的内容?

除了升级数据库,还有什么办法能避免出现这个Bug呢?收起
银行 · 2014-02-15
浏览2234
liaosnetliaosnet信息分析/架构师gbasedbt.com
看起来似乎是BUG.IC61836        ASSERTION FAILURE IN GETROW WHEN SQ_NFETCH IS ATTEMPTED ON CURSOR THAT IS OPEN ON AN INSERT STATEMENT显示全部
看起来似乎是BUG.
IC61836        ASSERTION FAILURE IN GETROW WHEN SQ_NFETCH IS ATTEMPTED ON CURSOR THAT IS OPEN ON AN INSERT STATEMENT收起
IT咨询服务 · 2014-02-15
浏览2252
pooh81pooh81项目经理农信社
帖参数Configuration File: /informix/etc/onconfig#**************************************************************************##                           INFORMIX SOFTWARE, INC.##&nb...显示全部
帖参数
Configuration File: /informix/etc/onconfig
#**************************************************************************
#
#                           INFORMIX SOFTWARE, INC.
#
#  Title:        onconfig.std
#  Description: Informix Dynamic Server Configuration Parameters
#
#**************************************************************************

# Root Dbspace Configuration

ROOTNAME        rootdbs         # Root dbspace name
ROOTPATH        /informix_data/rootdbs # Path for device containing root dbspace
ROOTOFFSET      0               # Offset of root dbspace into device (Kbytes)
ROOTSIZE        300000          # Size of root dbspace (Kbytes)

# Disk Mirroring Configuration Parameters

MIRROR          0               # Mirroring flag (Yes = 1, No = 0)
MIRRORPATH                      # Path for device containing mirrored root
MIRROROFFSET    0               # Offset into mirrored device (Kbytes)

# Physical Log Configuration

PHYSDBS         phydbs          # Location (dbspace) of physical log
PHYSFILE        150000          # Physical log file size (Kbytes)

# Logical Log Configuration

LOGFILES        30              # Number of logical log files
LOGSIZE         5000            # Logical log size (Kbytes)

# Diagnostics

MSGPATH         /informix/online.log # System message log file path
CONSOLE         /dev/console    # System console message path
ALARMPROGRAM    /informix/etc/log_full.sh # Alarm program path
SYSALARMPROGRAM /informix/etc/evidence.sh # System Alarm program path
TBLSPACE_STATS  1               

# System Archive Tape Device

TAPEDEV         /dev/null       # Tape device path       
TAPEBLK         16              # Tape block size (Kbytes)
TAPESIZE        10240           # Maximum amount of data to put on tape (Kbytes)

# Log Archive Tape Device

LTAPEDEV        /dev/null       # Log tape device path
LTAPEBLK        16              # Log tape block size (Kbytes)
LTAPESIZE       10240           # Max amount of data to put on log tape (Kbytes)

# Optical

STAGEBLOB                       # Informix Dynamic Server/Optical staging area

# System Configuration

SERVERNUM       1               # Unique id corresponding to a Dynamic Server instance
DBSERVERNAME    online          # Name of default database server
DBSERVERALIASES atm             # List of alternate dbservernames
#NETTYPE         ipcshm,1,150,CPU # Configure poll thread(s) for nettype
#NETTYPE         ipcshm,2,100,CPU # Configure poll thread(s) for nettype
NETTYPE         ipcshm,4,100,CPU # Configure poll thread(s) for nettype
NETTYPE         soctcp,3,8,NET  # Configure poll thread(s) for nettype
DEADLOCK_TIMEOUT 60              # Max time to wait of lock in distributed env.
RESIDENT        0               # Forced residency flag (Yes = 1, No = 0)

MULTIPROCESSOR  1               # 0 for single-processor, 1 for multi-processor
#NUMCPUVPS       1               # Number of user (cpu) vps
#NUMCPUVPS       2               # Number of user (cpu) vps
NUMCPUVPS       4               # Number of user (cpu) vps
#SINGLE_CPU_VP   1               # If non-zero, limit number of cpu vps to one
SINGLE_CPU_VP   0               # If non-zero, limit number of cpu vps to one

NOAGE           1               # Process aging
AFF_SPROC       0               # Affinity start processor
#AFF_NPROCS      0               # Affinity number of processors
AFF_NPROCS      2               # Affinity number of processors

# Shared Memory Parameters

LOCKS           2000000         # Maximum number of locks
BUFFERS         400000          # Maximum number of shared buffers
NUMAIOVPS       2               # Number of IO vps
PHYSBUFF        1024            # Physical log buffer size (Kbytes)
LOGBUFF         1024            # Logical log buffer size (Kbytes)
CLEANERS        4               # Number of buffer cleaner processes
SHMBASE         0x700000000000000 # Shared memory base address
SHMVIRTSIZE     20000           # initial virtual shared memory segment size
SHMADD          8192            # Size of new shared memory segments (Kbytes)
SHMTOTAL        0               # Total shared memory (Kbytes). 0=>unlimited
CKPTINTVL       300             # Check point interval (in sec)
LRUS            8               # Number of LRU queues
LRU_MAX_DIRTY   60              # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY   50              # LRU percent dirty end cleaning limit
LTXHWM          50              # Long transaction high water mark percentage
LTXEHWM         60              # Long transaction high water mark (exclusive)
TXTIMEOUT       0x12c             # Transaction timeout (in sec)
STACKSIZE       64              # Stack size (Kbytes)

# System Page Size
# BUFFSIZE - Dynamic Server no longer supports this configuration parameter.
#            To determine the page size used by Dynamic Server on your platform
#            see the last line of output from the command, 'onstat -b'.


# Recovery Variables
# OFF_RECVRY_THREADS:
# Number of parallel worker threads during fast recovery or an offline restore.
# ON_RECVRY_THREADS:
# Number of parallel worker threads during an online restore.

OFF_RECVRY_THREADS 10              # Default number of offline worker threads
ON_RECVRY_THREADS 10              # Default number of online worker threads

# Data Replication Variables
# DRAUTO: 0 manual, 1 retain type, 2 reverse type
DRINTERVAL      30              # DR max time between DR buffer flushes (in sec)
DRTIMEOUT       30              # DR network timeout (in sec)
DRLOSTFOUND     /informix/etc/dr.lostfound # DR lost+found file path

# CDR Variables
CDR_EVALTHREADS 1,2             # evaluator threads (per-cpu-vp,additional)
CDR_DSLOCKWAIT  5               # DS lockwait timeout (seconds)
CDR_QUEUEMEM    4096            # Maximum amount of memory for any CDR queue (Kbytes)
CDR_NIFCOMPRESS 0               # Link level compression (-1 never, 0 none, 9 max)

# Backup/Restore variables
BAR_ACT_LOG     /informix/bar_act.log
BAR_MAX_BACKUP  6               
BAR_RETRY       1               
BAR_NB_XPORT_COUNT 10              
BAR_XFER_BUF_SIZE 31              

# Informix Storage Manager variables
ISM_DATA_POOL   ISMData         # If the data pool name is changed, be sure to
                                # update $INFORMIXDIR/bin/onbar.  Change to
                                # ism_catalog -create_bootstrap -pool
ISM_LOG_POOL    ISMLogs         

# Read Ahead Variables
RA_PAGES        8               # Number of pages to attempt to read ahead
RA_THRESHOLD    4               # Number of pages left before next group

# DBSPACETEMP:
# Dynamic Server equivalent of DBTEMP for SE. This is the list of dbspaces
# that the Dynamic Server SQL Engine will use to create temp tables etc.
# If specified it must be a colon separated list of dbspaces that exist
# when the Dynamic Server system is brought online.  If not specified, or if
# all dbspaces specified are invalid, various ad hoc queries will create
# temporary files in /tmp instead.

DBSPACETEMP     tmpdbs          # Default temp dbspaces

# DUMP*:
# The following parameters control the type of diagnostics information which
# is preserved when an unanticipated error condition (assertion failure) occurs
# during Dynamic Server operations.
# For DUMPSHMEM, DUMPGCORE and DUMPCORE 1 means Yes, 0 means No.

DUMPDIR         /informix/tmp   # Preserve diagnostics in this directory
DUMPSHMEM       1               # Dump a copy of shared memory
DUMPGCORE       0               # Dump a core image using 'gcore'
DUMPCORE        0               # Dump a core image (Warning:this aborts Dynamic Server)
DUMPCNT         1               # Number of shared memory or gcore dumps for
                                # a single user's session

FILLFACTOR      90              # Fill factor for building indexes

# method for Dynamic Server to use when determining current time
USEOSTIME       0               # 0: use internal time(fast), 1: get time from OS(slow)

# Parallel Database Queries (pdq)
MAX_PDQPRIORITY 100             # Maximum allowed pdqpriority
DS_MAX_QUERIES  10              # Maximum number of decision support queries
DS_TOTAL_MEMORY 80000           # Decision support memory (Kbytes)
DS_MAX_SCANS    1048576         # Maximum number of decision support scans       
DATASKIP        off             # List of dbspaces to skip

# OPTCOMPIND
# 0 => Nested loop joins will be preferred (where
#      possible) over sortmerge joins and hash joins.
# 1 => If the transaction isolation mode is not
#      "repeatable read", optimizer behaves as in (2)
#      below.  Otherwise it behaves as in (0) above.
# 2 => Use costs regardless of the transaction isolation
#      mode.  Nested loop joins are not necessarily
#      preferred.  Optimizer bases its decision purely
#      on costs.
OPTCOMPIND      2               # To hint the optimizer

ONDBSPACEDOWN   2               # Dbspace down option: 0 = CONTINUE, 1 = ABORT, 2 = WAIT
OPCACHEMAX      0               # Maximum optical cache size (Kbytes)

# HETERO_COMMIT (Gateway participation in distributed transactions)
# 1 => Heterogeneous Commit is enabled
# 0 (or any other value) => Heterogeneous Commit is disabled
HETERO_COMMIT   0               

# Optimization goal: -1 = ALL_ROWS(Default), 0 = FIRST_ROWS
OPT_GOAL        -1              

# Optimizer DIRECTIVES ON (1/Default) or OFF (0)
DIRECTIVES      1               

# Status of restartable restore
RESTARTABLE_RESTORE off            
CDR_SERIAL      0,0             # Serial Column Sequence
CDR_DBSPACE                     # dbspace for syscdr database
CDR_QHDR_DBSPACE                 # CDR queue dbspace (default same as catalog)
CDR_QDATA_SBSPACE                 # CDR queue smart blob space
CDR_QDATA_SBFLAGS 2               # Log/no-log (default no log)   
BAR_DEBUG_LOG   /tmp/bar_dbug.log # ON-Bar Debug Log - not in /tmp please
BAR_PROGRESS_FREQ 0               
SBSPACENAME                     # Default smartblob space name - this is where blobs
SYSSBSPACENAME                  # Default smartblob space for use by the Informix
BLOCKTIMEOUT    3600            # Default timeout for system block
ALLOW_NEWLINE   0               # embedded newlines(Yes = 1, No = 0 or anything but 1)
JVPJAVAHOME                     # JRE installation root directory
JVPHOME                         # Krakatoa installation directory
JVPPROPFILE     .jvpprops       # JVP property file
JDKVERSION                      # JDK version supported by this server
JVPJAVALIB                     
JVPJAVAVM       libjava.so      
JVPCLASSPATH收起
银行 · 2014-02-15
浏览2295
liaosnetliaosnet信息分析/架构师gbasedbt.com
ftp://aix.software.ibm.com/softw ... 0.UC4.unixbugs.htmlbug_number      159108description     CORRUPT SQLI MSG CAN CAUSE MEMRY CORRUPTION AND CRASH ENGINE, SANITY CHECKS IN SQ_RETTYPE CAN PREVENT CORRUPTIONprod...显示全部
ftp://aix.software.ibm.com/softw ... 0.UC4.unixbugs.html

bug_number      159108
description     CORRUPT SQLI MSG CAN CAUSE MEMRY CORRUPTION AND CRASH ENGINE, SANITY CHECKS IN SQ_RETTYPE CAN PREVENT CORRUPTION
product_code    ONLINE
component_code  SQL

应该还是BUG.

BTW: 有空就升一下级吧。。。旧版本的可能连BUG都难找。。收起
IT咨询服务 · 2014-02-15
浏览2191

提问者

pooh81
项目经理农信社

相关问题

问题状态

  • 发布时间:2014-02-15
  • 关注会员:0 人
  • 问题浏览:13213
  • 最近回答:2014-02-18
  • X社区推广