驱动程序类编译错误-hadoop mapreduce

cnh2zyt3  于 2021-06-02  发布在  Hadoop
关注(0)|答案(3)|浏览(345)

我有3.java文件

1) Mapper.java
2) Reducer.java
3) Driver.java

我试图在命令行使用驱动程序类编译hadoopmapreduce程序,但它显示了以下错误

Driver.java:39: error: cannot find symbol
        job.setMapperClass(Mapper.class);
                           ^
  symbol:   class Mapper
  location: class Driver
Driver.java:40: error: cannot find symbol
        job.setReducerClass(Reducer.class);

如何解决上述错误。下面是驱动程序类中的run方法

public boolean runnerParsing(String inputPath, String outputPath) throws IOException, ClassNotFoundException, InterruptedException {
         Configuration conf = new Configuration();

        Job job = new Job(conf, "Parsing");
        job.setJarByClass(Driver.class);

        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);

        job.setMapperClass(Mapper.class);
        job.setReducerClass(Reducer.class);
        //job.setNumReduceTasks(0);

        job.setInputFormatClass(TextInputFormat.class);
        job.setOutputFormatClass(TextOutputFormat.class);

        FileInputFormat.addInputPath(job, new Path(inputPath));
        FileOutputFormat.setOutputPath(job, new Path(outputPath));

        return job.waitForCompletion(true);

     }
wtlkbnrh

wtlkbnrh1#

我是这么想的:
mapper.java/reducer.java中没有存储mapper/reducer。
在另一个包中有mapper/reducer类
你能把你的Map器和还原器类也添加到这个问题中吗?

eni9jsuy

eni9jsuy2#

在Garry tukington和gabriele modena的“hadoop大数据”的第3章中,有一个模式top n的例子,在toptenhashtag类的驱动程序代码中,有一条引用了另一个java类的指令:
job.setjarbyclass(hashtagcount.class);
在我的电脑中,hashtagcount.java和toptenhashtag.java这两个文件都位于:/home/hduser/playground/src中
我的编译命令如下(并且对我有效):
javac-classpath$hadoop\u home/share/hadoop/common/lib/activation-1.1.jar:$hadoop\u home/share/hadoop/common/hadoop-common-2.7.1.jar:$hadoop\u home/share/hadoop/common/lib/:/usr/hadoop/hadoop-2.7.1/share/hadoop/mapreduce/-d playground/classes7 playground/src/toptenhashtag.java\playground/src/hashtagcount.java
这是创建.jar文件的命令:jar-cvf playground/toptenhashtag.jar-c playground/classes7/。
最后,这是启动mapreduce作业的命令:
hadoop jar/home/hduser/playground/toptenhashtag.jar com.learninghadoop2.mapreduce.toptenhashtag/user/hduser/inxyz/outhashxyz

a64a0gku

a64a0gku3#

您需要编译所有java文件,如下所示: javac -classpath /usr/local/hadoop/hadoop-core-1.2.1.jar -d compiled_classes Driver.java Mapper.java Reducer.java 请注意,根据hadoop的安装方式,类路径值可能会略有变化。
如果您需要进一步的帮助,请看这篇文章,它可能会帮助您:http://www.bigdatatutes.com/getting-started-with-big-data/

相关问题