KETTLE ETL 提数 无法连接数据库?

背景: kettle 工具提数从源数据库到目标数据库。
但是 由于kettle 启动的时候,上面的job都是并发的,感觉服务器的物理内存都被吃掉了,基本上是99%满了。报错日志如下:
ERROR 21-10 04:43:15,507 - 表输入TABLE A - An error occurred, processing will be stopped:
Error occured while trying to connect to the database

Error connecting to database: (using class oracle.jdbc.driver.OracleDriver)
无法从套接字读取更多的数据

ERROR 21-10 04:43:15,507 - TABLE B表输入 - An error occurred, processing will be stopped:
Error occured while trying to connect to the database

Error connecting to database: (using class oracle.jdbc.driver.OracleDriver)
无法从套接字读取更多的数据

ERROR 21-10 04:43:15,507 - TABLE B表输出 - An error occurred intialising this step:
Error occured while trying to connect to the database

Error connecting to database: (using class oracle.jdbc.driver.OracleDriver)
无法从套接字读取更多的数据

ERROR 21-10 04:43:15,507 - 表输出TABLE C 4 - An error occurred intialising this step:
Error occured while trying to connect to the database

Error connecting to database: (using class oracle.jdbc.driver.OracleDriver)
无法从套接字读取更多的数据

ERROR 21-10 04:43:15,507 - TABLE B表输出 - 错误初始化步骤[TABLE B表输出]
ERROR 21-10 04:43:15,507 - 表输入TABLE A - 错误初始化步骤[表输入TABLE A]
ERROR 21-10 04:43:15,507 - TABLE B表输入 - 错误初始化步骤[TABLE B表输入]
ERROR 21-10 04:43:15,507 - 表输出TABLE C 4 - 错误初始化步骤[表输出TABLE C 4]
ERROR 21-10 04:43:15,508 - TABLE B - 步骤 [TABLE B表输入.0] 初始化失败!
ERROR 21-10 04:43:15,508 - TABLE C - 步骤 [表输入TABLE A.0] 初始化失败!
ERROR 21-10 04:43:15,508 - TABLE B - 步骤 [TABLE B表输出.0] 初始化失败!
ERROR 21-10 04:43:15,508 - TABLE C - 步骤 [表输出TABLE C 4.0] 初始化失败!
INFO 21-10 04:43:15,508 - TABLE B表输入 - Finished reading query, closing connection.
INFO 21-10 04:43:15,508 - 表输入TABLE D - Finished reading query, closing connection.
ERROR 21-10 04:43:15,509 - TABLE B表输出 - Unexpected error rolling back the database connection.
INFO 21-10 04:43:15,511 - 表输入TABLE E - Finished reading query, closing connection.
INFO 21-10 04:43:15,515 - 表输入TABLE G - Finished reading query, closing connection.
INFO 21-10 04:43:15,518 - 表输入TABLE A - Finished reading query, closing connection.
ERROR 21-10 04:43:15,518 - 表输出TABLE C 4 - Unexpected error rolling back the database connection.
ERROR 21-10 04:43:15,585 - 表输出TABLE C 4 - org.pentaho.di.core.exception.KettleDatabaseException:
Unable to get database metadata from this database connection

at java.lang.Thread.run (Thread.java:745)
at org.pentaho.di.job.entries.job.JobEntryJobRunner.run (JobEntryJobRunner.java:63)
at org.pentaho.di.job.Job.execute (Job.java:420)
at org.pentaho.di.job.Job.execute (Job.java:642)
at org.pentaho.di.job.Job.execute (Job.java:642)
at org.pentaho.di.job.Job.execute (Job.java:503)
at org.pentaho.di.job.entries.trans.JobEntryTrans.execute (JobEntryTrans.java:1024)
at org.pentaho.di.trans.Trans.execute (Trans.java:364)
at org.pentaho.di.trans.Trans.prepareExecution (Trans.java:766)
at org.pentaho.di.trans.steps.tableoutput.TableOutput.dispose (TableOutput.java:697)
at org.pentaho.di.core.database.Database.rollback (Database.java:717)
at org.pentaho.di.core.database.Database.rollback (Database.java:728)
at org.pentaho.di.core.database.Database.getDatabaseMetaData (Database.java:3139)

at org.pentaho.di.core.database.Database.getDatabaseMetaData(Database.java:3143)
at org.pentaho.di.core.database.Database.rollback(Database.java:728)
at org.pentaho.di.core.database.Database.rollback(Database.java:717)
at org.pentaho.di.trans.steps.tableoutput.TableOutput.dispose(TableOutput.java:697)
at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:766)
at org.pentaho.di.trans.Trans.execute(Trans.java:364)
at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:1024)
at org.pentaho.di.job.Job.execute(Job.java:503)
at org.pentaho.di.job.Job.execute(Job.java:642)
at org.pentaho.di.job.Job.execute(Job.java:642)
at org.pentaho.di.job.Job.execute(Job.java:420)
at org.pentaho.di.job.entries.job.JobEntryJobRunner.run(JobEntryJobRunner.java:63)
at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.NullPointerException

at org.pentaho.di.core.database.Database.getDatabaseMetaData(Database.java:3139)
... 12 more

ERROR 21-10 04:43:15,585 - TABLE B表输出 - org.pentaho.di.core.exception.KettleDatabaseException:
Unable to get database metadata from this database connection

at java.lang.Thread.run (Thread.java:745)
at org.pentaho.di.job.entries.job.JobEntryJobRunner.run (JobEntryJobRunner.java:63)
at org.pentaho.di.job.Job.execute (Job.java:420)
at org.pentaho.di.job.Job.execute (Job.java:642)
at org.pentaho.di.job.Job.execute (Job.java:642)
at org.pentaho.di.job.Job.execute (Job.java:503)
at org.pentaho.di.job.entries.trans.JobEntryTrans.execute (JobEntryTrans.java:1024)
at org.pentaho.di.trans.Trans.execute (Trans.java:364)
at org.pentaho.di.trans.Trans.prepareExecution (Trans.java:766)
at org.pentaho.di.trans.steps.tableoutput.TableOutput.dispose (TableOutput.java:697)
at org.pentaho.di.core.database.Database.rollback (Database.java:717)
at org.pentaho.di.core.database.Database.rollback (Database.java:728)
at org.pentaho.di.core.database.Database.getDatabaseMetaData (Database.java:3139)

at org.pentaho.di.core.database.Database.getDatabaseMetaData(Database.java:3143)
at org.pentaho.di.core.database.Database.rollback(Database.java:728)
at org.pentaho.di.core.database.Database.rollback(Database.java:717)
at org.pentaho.di.trans.steps.tableoutput.TableOutput.dispose(TableOutput.java:697)
at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:766)
at org.pentaho.di.trans.Trans.execute(Trans.java:364)
at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:1024)
at org.pentaho.di.job.Job.execute(Job.java:503)
at org.pentaho.di.job.Job.execute(Job.java:642)
at org.pentaho.di.job.Job.execute(Job.java:642)
at org.pentaho.di.job.Job.execute(Job.java:420)
at org.pentaho.di.job.entries.job.JobEntryJobRunner.run(JobEntryJobRunner.java:63)
at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.NullPointerException

at org.pentaho.di.core.database.Database.getDatabaseMetaData(Database.java:3139)
... 12 more

ERROR 21-10 04:43:15,727 - TRUNCATE TABLE F - 执行该作业项时发生了一个错误:
Error occured while trying to connect to the database

另外,我发现一个问题,为什么手动执行kettle 提数非常快,而且不报错,但是服务器上做了个定时任务启动kette就经常报错那?即便是手动执行和服务器上去执行内存基本上都是99%以上,但是手动执行就是不报错而且还很快,难道手动执行 不往日志表里面写日志这个节省时间了吗?

求大家集思广益,求帮忙?还是去想办法优化sql那。

参与3

1同行回答

yyj1827yyj1827  软件开发工程师 , hn
并发减少些,一点一点试。。。是不是内存不足,没法连接到数据库。。。显示全部

并发减少些,一点一点试。。。
是不是内存不足,没法连接到数据库。。。

收起
软件开发 · 2017-11-17
浏览15804
王磊磊 邀答

提问者

jdw_cognos
数据仓库工程师IBM
擅长领域: 商业智能大数据cognos

相关问题

相关资料

相关文章

问题状态

  • 发布时间:2017-10-21
  • 关注会员:2 人
  • 问题浏览:16328
  • 最近回答:2017-11-17
  • X社区推广