使用apache sqoop从msql导入数据-在查找匹配的``时出现意外的eof

8xiog9wr  于 2021-07-15  发布在  Hadoop
关注(0)|答案(0)|浏览(334)

正在尝试使用apachesqoop从mysql数据库导入数据。多集群环境是在hadoop中设置的(namenode和两个datanode),我想用sqoop将mysql中的数据导入hdfs文件系统,但是它给出了以下错误连接是可以的,我可以用sqoop命令列出表。还创建了Map缩减器。
sqoop导入命令。

sqoop import --connect jdbc:mysql://192.168.1.98:3306/test --username root -P --table user_details --target-dir testdata -m 1

来自控制台的错误日志-

2021-02-09 09:38:59,697 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoopuser/compile/7a0f90e2187e19a5cd9403a6908f8cfd/user_details.jar
2021-02-09 09:38:59,713 WARN manager.MySQLManager: It looks like you are importing from mysql.
2021-02-09 09:38:59,713 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
2021-02-09 09:38:59,713 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
2021-02-09 09:38:59,713 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
2021-02-09 09:38:59,742 INFO mapreduce.ImportJobBase: Beginning import of user_details
2021-02-09 09:38:59,743 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2021-02-09 09:39:00,004 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
2021-02-09 09:39:00,774 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2021-02-09 09:39:00,932 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
2021-02-09 09:39:01,967 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/hadoopuser/.staging/job_1612843617470_0001
2021-02-09 09:39:07,007 INFO db.DBInputFormat: Using read commited transaction isolation
2021-02-09 09:39:07,070 INFO mapreduce.JobSubmitter: number of splits:1
2021-02-09 09:39:07,285 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1612843617470_0001
2021-02-09 09:39:07,286 INFO mapreduce.JobSubmitter: Executing with tokens: []
2021-02-09 09:39:07,516 INFO conf.Configuration: resource-types.xml not found
2021-02-09 09:39:07,516 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2021-02-09 09:39:08,003 INFO impl.YarnClientImpl: Submitted application application_1612843617470_0001
2021-02-09 09:39:08,049 INFO mapreduce.Job: The url to track the job: http://hadoop-master.test.net:8088/proxy/application_1612843617470_0001/
2021-02-09 09:39:08,050 INFO mapreduce.Job: Running job: job_1612843617470_0001
2021-02-09 09:39:14,119 INFO mapreduce.Job: Job job_1612843617470_0001 running in uber mode : false
2021-02-09 09:39:14,119 INFO mapreduce.Job:  map 0% reduce 0%
2021-02-09 09:39:14,146 INFO mapreduce.Job: Job job_1612843617470_0001 failed with state FAILED due to: Application application_1612843617470_0001 failed 2 times due to AM Container for appattempt_1612843617470_0001_000002 exited with  exitCode: 2
Failing this attempt.Diagnostics: [2021-02-09 09:39:13.714]Exception from container-launch.
Container id: container_1612843617470_0001_02_000001
Exit code: 2

[2021-02-09 09:39:13.749]Container exited with a non-zero exit code 2. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
/home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612843617470_0001/container_1612843617470_0001_02_000001/launch_container.sh: line 130: unexpected EOF while looking for matching `"'

[2021-02-09 09:39:13.760]Container exited with a non-zero exit code 2. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
/home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612843617470_0001/container_1612843617470_0001_02_000001/launch_container.sh: line 130: unexpected EOF while looking for matching `"'

For more detailed output, check the application tracking page: http://hadoop-master.test.net:8088/cluster/app/application_1612843617470_0001 Then click on links to logs of each attempt.
. Failing the application.
2021-02-09 09:39:14,164 INFO mapreduce.Job: Counters: 0
2021-02-09 09:39:14,169 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2021-02-09 09:39:14,171 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 13.386 seconds (0 bytes/sec)
2021-02-09 09:39:14,174 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2021-02-09 09:39:14,174 INFO mapreduce.ImportJobBase: Retrieved 0 records.
2021-02-09 09:39:14,174 ERROR tool.ImportTool: Import failed: Import job failed!

数据节点日志-

2021-02-09 08:33:03,829 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1612839739191_0001_02_000001 transitioned from LOCALIZING to SCHEDULED
2021-02-09 08:33:03,829 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.scheduler.ContainerScheduler: Starting container [container_1612839739191_0001_02_000001]
2021-02-09 08:33:03,911 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: launchContainer: [bash, /home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001/default_container_executor.sh]
2021-02-09 08:33:03,912 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1612839739191_0001_02_000001 transitioned from SCHEDULED to RUNNING
2021-02-09 08:33:03,912 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1612839739191_0001_02_000001
2021-02-09 08:33:03,944 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1612839739191_0001_02_000001 is : 2
2021-02-09 08:33:03,945 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception from container-launch with container ID: container_1612839739191_0001_02_000001 and exit code: 2
ExitCodeException exitCode=2: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:1008)
    at org.apache.hadoop.util.Shell.run(Shell.java:901)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1213)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:294)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:491)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:303)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:102)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
2021-02-09 08:33:03,946 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception from container-launch.

    2021-02-09 08:33:03,946 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Container id: container_1612839739191_0001_02_000001
    2021-02-09 08:33:03,946 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exit code: 2
    2021-02-09 08:33:03,947 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container launch failed : Container exited with a non-zero exit code 2. 
    2021-02-09 08:33:03,989 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1612839739191_0001_02_000001 transitioned from RUNNING to EXITED_WITH_FAILURE
    2021-02-09 08:33:03,995 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001
    2021-02-09 08:33:04,006 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerCleanup: Cleaning up container container_1612839739191_0001_02_000001
    2021-02-09 08:33:04,010 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1612839739191_0001_02_000001 transitioned from EXITED_WITH_FAILURE to DONE
    2021-02-09 08:33:04,010 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Removing container_1612839739191_0001_02_000001 from application application_1612839739191_0001
    2021-02-09 08:33:04,010 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1612839739191_0001_02_000001
    2021-02-09 08:33:04,010 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1612839739191_0001
    2021-02-09 08:33:04,047 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001/launch_container.sh
    2021-02-09 08:33:04,047 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: delete returned false for path: [/home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001/launch_container.sh]
    2021-02-09 08:33:04,047 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001/container_tokens
    2021-02-09 08:33:04,047 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: delete returned false for path: [/home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001/container_tokens]
    2021-02-09 08:33:04,047 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001/sysfs
    2021-02-09 08:33:04,047 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: delete returned false for path: [/home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001/container_1612839739191_0001_02_000001/sysfs]
    2021-02-09 08:33:05,022 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1612839739191_0001_02_000001]
    2021-02-09 08:33:05,025 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1612839739191_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP
    2021-02-09 08:33:05,025 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1612839739191_0001
    2021-02-09 08:33:05,026 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.ApplicationImpl: Application application_1612839739191_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
    2021-02-09 08:33:05,026 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: Scheduling Log Deletion for application: application_1612839739191_0001, with delay of 10800 seconds
    2021-02-09 08:33:05,029 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoopuser/tmp/nm-local-dir/usercache/hadoopuser/appcache/application_1612839739191_0001
    2021-02-09 08:42:19,817 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: Cache Size Before Clean: 30949793, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题