pig无法从localstorage读取数据

9udxz4iz  于 2021-06-21  发布在  Pig
关注(0)|答案(1)|浏览(282)

我运行了以下命令,包括在pig中的load和dump命令:我的文件在本地,我在独立的机器上。

grunt> custs = LOAD 'custs' USING PigStorage(',') As (id:long,name:chararray,surname:chararray,age:int,prof:chararray);

grunt> dump custs;

我有个错误:

HadoopVersion   PigVersion  UserId  StartedAt   FinishedAt  Features
2.7.1   0.15.0  sujit   2016-04-21 13:05:51 2016-04-21 13:05:51 UNKNOWN

Failed!

Failed Jobs:
JobId   Alias   Feature Message Outputs
N/A custs   MAP_ONLY    Message: java.lang.NullPointerException

Input(s):
Failed to read data from "file:///home/sujit/pig_data/custs"

Job DAG:
null

2016-04-21 13:05:51,277 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2016-04-21 13:05:51,278 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias custs
Details at logfile: /home/sujit/pig_data/pig_1461220631975.log

但该文件存在,并具有所需的权限

sujit@sujit-pc:~/pig_data$ ls -l
total 4824
-rwxrwxrwx 1 sujit sujit  391355 Jun  9  2012 custs
hsgswve4

hsgswve41#

我收到这个错误是因为我还没有将本地文件复制到hdfs。
要将文件从本地复制到hdfs,可以使用以下命令

hdfs dfs -copyFromLocal yourFileNameHere.csv ~/yourFolder/yourFileNameHere.csv
  • 注意您必须先在hdfs中创建一个文件夹。这是通过如下命令完成的-
hdfs dfs -mkdir /home/ubuntu/yourFolder

将本地文件复制到hdfs后,我不再收到错误。

相关问题