为什么我在使用pydoop复制hdfs上的文件时会遇到文件未找到异常?

vmjh9lq9  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(173)

下面是我使用pydoopapi的python代码。我有一个文本文件 /home/progen/test.txt . 在运行脚本将此文件复制到hdfs时,出现了一个错误。

import pydoop.hdfs as hdfs

     local_path = '/home/progen/test.txt'

     hdfs_path = '/spark_user/' 

     host = 'master'

     port = 9000

     hdfsobj = hdfs.hdfs(host, port, user='spark', groups=['supergroup'])

     hdfsobj.copy(local_path, hdfsobj, hdfs_path)

我得到错误:“文件不存在”

hdfsCopyImpl(src=/home/progen/test.txt, dst=/spark_user/, deleteSource=0): FileUtil#copy error:
java.io.FileNotFoundException: File does not exist: /home/progen/test.txt
    at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1122)
    at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
Traceback (most recent call last):
  File "hdfs.py", line 8, in <module>
    hdfsobj.copy(local_path, hdfsobj, hdfs_path)
  File "/usr/local/lib/python2.7/dist-packages/pydoop/hdfs/fs.py", line 304, in copy
    return self.fs.copy(from_path, to_hdfs, to_path)

hdfsCopyImpl(src=/home/progen/test.txt, dst=/spark_user/, deleteSource=0): FileUtil#copy error:
java.io.FileNotFoundException: File does not exist: /home/progen/test.txt

at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1122)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
Traceback (most recent call last):
  File "hdfs.py", line 8, in <module>
    hdfsobj.copy(local_path, hdfsobj, hdfs_path)
  File "/usr/local/lib/python2.7/dist-packages/pydoop/hdfs/fs.py", line 304, in copy
    return self.fs.copy(from_path, to_hdfs, to_path)

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题