我有3.java文件
1) Mapper.java
2) Reducer.java
3) Driver.java
我试图在命令行使用驱动程序类编译hadoopmapreduce程序,但它显示了以下错误
Driver.java:39: error: cannot find symbol
job.setMapperClass(Mapper.class);
^
symbol: class Mapper
location: class Driver
Driver.java:40: error: cannot find symbol
job.setReducerClass(Reducer.class);
如何解决上述错误。下面是驱动程序类中的run方法
public boolean runnerParsing(String inputPath, String outputPath) throws IOException, ClassNotFoundException, InterruptedException {
Configuration conf = new Configuration();
Job job = new Job(conf, "Parsing");
job.setJarByClass(Driver.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
job.setMapperClass(Mapper.class);
job.setReducerClass(Reducer.class);
//job.setNumReduceTasks(0);
job.setInputFormatClass(TextInputFormat.class);
job.setOutputFormatClass(TextOutputFormat.class);
FileInputFormat.addInputPath(job, new Path(inputPath));
FileOutputFormat.setOutputPath(job, new Path(outputPath));
return job.waitForCompletion(true);
}
3条答案
按热度按时间wtlkbnrh1#
我是这么想的:
mapper.java/reducer.java中没有存储mapper/reducer。
在另一个包中有mapper/reducer类
你能把你的Map器和还原器类也添加到这个问题中吗?
eni9jsuy2#
在Garry tukington和gabriele modena的“hadoop大数据”的第3章中,有一个模式top n的例子,在toptenhashtag类的驱动程序代码中,有一条引用了另一个java类的指令:
job.setjarbyclass(hashtagcount.class);
在我的电脑中,hashtagcount.java和toptenhashtag.java这两个文件都位于:/home/hduser/playground/src中
我的编译命令如下(并且对我有效):
javac-classpath$hadoop\u home/share/hadoop/common/lib/activation-1.1.jar:$hadoop\u home/share/hadoop/common/hadoop-common-2.7.1.jar:$hadoop\u home/share/hadoop/common/lib/:/usr/hadoop/hadoop-2.7.1/share/hadoop/mapreduce/-d playground/classes7 playground/src/toptenhashtag.java\playground/src/hashtagcount.java
这是创建.jar文件的命令:jar-cvf playground/toptenhashtag.jar-c playground/classes7/。
最后,这是启动mapreduce作业的命令:
hadoop jar/home/hduser/playground/toptenhashtag.jar com.learninghadoop2.mapreduce.toptenhashtag/user/hduser/inxyz/outhashxyz
a64a0gku3#
您需要编译所有java文件,如下所示:
javac -classpath /usr/local/hadoop/hadoop-core-1.2.1.jar -d compiled_classes Driver.java Mapper.java Reducer.java
请注意,根据hadoop的安装方式,类路径值可能会略有变化。如果您需要进一步的帮助,请看这篇文章,它可能会帮助您:http://www.bigdatatutes.com/getting-started-with-big-data/