我有一个mapreduce实现,其中mapper的输入是hbase扫描结果(每行一个map),reduce阶段将输出直接存储在特定的hbase表中。
为了指定mapper和reducer参数,我使用 TableMapReduceUtil
助手类如下所示:
TableMapReduceUtil.initTableMapperJob(
"Simulations_Data", // input HBase table name
scan_job_DMV, // Scan instance to control CF and attribute selection <key, value>
HBaseMapper.class, // mapper
TimestepNode.class, // mapper output key
DoubleArrayWritable.class, // mapper output value
to_return);
TableMapReduceUtil.initTableReducerJob(
"SimulationsResults", // output table
HBaseReducer.class, // reducer class
to_return);
工作环境为hbase-0.98.4-hadoop2和hadoop-2.6.0
如果我使用eclipse(版本:luna service release 2-4.4.2)运行代码,那么一切都可以完美地工作。但是,如果我从eclipse生成一个可运行的jar文件(使用 Package required lib
,并尝试从终端运行,出现以下错误:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.11-hadoop2.jar
at org.apache.hadoop.fs.Path.initialize(Path.java:206)
at org.apache.hadoop.fs.Path.<init>(Path.java:172)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(TableMapReduceUtil.java:823)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:776)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars(TableMapReduceUtil.java:692)
at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:737)
at D2C_mapreduce_main.get_DMV_Job(D2C_mapreduce_main.java:130)
at D2C_mapreduce_main.main(D2C_mapreduce_main.java:102)
... 5 more
Caused by: java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.11-hadoop2.jar
at java.net.URI.checkPath(URI.java:1804)
at java.net.URI.<init>(URI.java:752)
at org.apache.hadoop.fs.Path.initialize(Path.java:203)
... 12 more
这一问题在这里得到了部分解决:
hadoop java.net.urisyntaxexception:绝对uri中的相对路径:rsrc:hbase-common-0.98.1-hadoop2.jar
http://mail-archives.apache.org/mod_mbox/hbase-user/201310.mbox/%3cloom.20131029t195007-578@post.gmane.org%3e
但尽管如此,也许由于缺乏在线资源,我认为这对社区来说是一个公开的问题。
暂无答案!
目前还没有任何答案,快来回答吧!