不使用hadoop命令运行hadoop java代码

j91ykkif  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(256)
public void readFile(String file) throws IOException {
    Cofiguration conf = new Configuration();
    conf.addResource(new Path("/usr/local/hadoop-2.7.3/etc/hadoop/core-site.xml"))
    conf.addResource(new Path("/usr/local/hadoop-2.7.3/etc/hadoop/hdfs-site.xml"))
    conf.addResource(new Path("/usr/local/hadoop-2.7.3/etc/hadoop/mapred-site.xml"))
}
FileSystem fileSystem = FileSystem.get(conf);
System.out.println("DefaultFS:      " + cong.get("fs.defaultFS"));
System.out.println("Home directory: " + fileSystem.getHomeDirectory());

Path path = new Path(file);
if(!fileSystem.exists(path)) {
    System.out.println("File " + file + " does not exists");
    return;
}

我对hadoop非常陌生,我想知道是否可以使用“java-jar”执行这个HadoopJava客户机代码。
我的代码使用“hadoopjar”命令工作。但是,当我尝试使用“java-jar”而不是“hadoop-jar”执行此代码时,它无法在hdfs中找到文件,方法gethomeditory()返回一个不存在的本地路径。
是否未正确添加配置文件?为什么代码只有在hadoop命令下执行时才能工作?

9udxz4iz

9udxz4iz1#

Instead of passing a Path object, pass the file path as string

    conf.addResource("/usr/local/hadoop-2.7.3/etc/hadoop/core-site.xml");
    conf.addResource("/usr/local/hadoop-2.7.3/etc/hadoop/hdfs-site.xml");
    conf.addResource("/usr/local/hadoop-2.7.3/etc/hadoop/mapred-site.xml");

Or else you could add these files to classpath and try.

    conf.addResource("core-site.xml");
    conf.addResource("hdfs-site.xml");
    conf.addResource("mapred-site.xml");

相关问题