我的hadoop2.6.5 ha，从mysql导入数据时，运行error java.lang.StackOverflower

63lcw9qa  于 2021-05-29  发布在  Hadoop
关注(0)|答案(2)|浏览(241)

在hadoop2.6.5ha中，当我使用sqoop1.4.6从mysql数据导入数据时，出现以下错误。
操作系统：centos6.5位
有人面临同样的问题吗？如果我错过了任何地方？？

[hadoop@dns app]$ sqoop import --connect jdbc:mysql://localhost:3306/hive --username hive --password hive --table DBS --m 1 --target-dir /user/test3
Warning: /home/hadoop/app/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/app/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/09/01 16:54:11 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
17/09/01 16:54:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
17/09/01 16:54:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
17/09/01 16:54:11 INFO tool.CodeGenTool: Beginning code generation
17/09/01 16:54:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `DBS` AS t LIMIT 1
17/09/01 16:54:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `DBS` AS t LIMIT 1
17/09/01 16:54:11 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/app/hadoop
Note: /tmp/sqoop-hadoop/compile/53650552d9b1969139bf57841c0c9aa1/DBS.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
17/09/01 16:54:14 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/53650552d9b1969139bf57841c0c9aa1/DBS.jar
17/09/01 16:54:14 WARN manager.MySQLManager: It looks like you are importing from mysql.
17/09/01 16:54:14 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
17/09/01 16:54:14 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
17/09/01 16:54:14 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
17/09/01 16:54:14 INFO mapreduce.ImportJobBase: Beginning import of DBS
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/app/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/app/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
17/09/01 16:54:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/09/01 16:54:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
17/09/01 16:54:14 WARN fs.FileSystem: "cluster1" is a deprecated filesystem name. Use "hdfs://cluster1/" instead.
17/09/01 16:54:20 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
17/09/01 16:54:20 WARN fs.FileSystem: "cluster1" is a deprecated filesystem name. Use "hdfs://cluster1/" instead.
Exception in thread "main" java.lang.StackOverflowError
        at org.apache.commons.collections.map.AbstractMapDecorator.containsKey(AbstractMapDecorator.java:83)
        at org.apache.hadoop.conf.Configuration.isDeprecated(Configuration.java:558)
        at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:605)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1185)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470)
        at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444)
[hadoop@dns app]$
Javahadoopmysql

2条答案

按热度按时间
dgjrabp2

dgjrabp21#

这个 StackOverFlowError 发生的原因是java调用堆栈已增长，并且运行jvm时没有更多内存来容纳更多的调用堆栈详细信息。
请仔细查看java错误堆栈并注意-有以下两个函数

public static FileContext getFileContext(final URI defaultFsUri, final Configuration aConf)
public static FileContext getFileContext(final Configuration aConf)

在某些情况下，第一个getfilecontext调用第二个getfilecontext，第二个getfilecontext调用第一个getfilecontext。这意味着它的递归调用。
就你而言 core-site.xml 必须包含属性 fs.defaultFS 有价值的 cluster1 . 相反，它应该是 hdfs://cluster1 . 如果设置 fs.defaultFS 到正确的url hdfs://cluster1 .

赞(0）分享  回复(0）举报  2021-05-29
piwo6bdm

piwo6bdm2#

在您的示例中，core-site.xml必须包含值为cluster1的属性fs.defaultfs。相反，它应该是hdfs://cluster1. 如果您将fs.defaultfs设置为正确的url，则递归调用将消失hdfs://cluster1.

赞(0）分享  回复(0）举报  2021-05-29
我来回答

相关问题

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新问答

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com