我在hadoop0.20.2版本上开发了一个带有reduce-side-join的mapreduce。所有java类编译时都没有任何错误。我的程序能正常工作。但是这个版本的hadoop在测试虚拟机上。在我真正的集群上，我有另一个版本的hadoop（hadoop2.00-cdh4.1.2）。当我想编译java类时，它不起作用。我得到100个错误，如下所示：
JoinMapper.java:8: package org.apache.hadoop.filecache does not exist
import org.apache.hadoop.filecache.DistributedCache;
^
JoinMapper.java:9: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.Path;
^
JoinMapper.java:10: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.Text;
^
JoinMapper.java:11: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
JoinMapper.java:12: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.FileInputFormat;
^
JoinMapper.java:13: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.FileOutputFormat;
^
JoinMapper.java:14: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.InputFormat;
^
JoinMapper.java:15: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.JobClient;
^
JoinMapper.java:16: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.JobConf;
^
JoinMapper.java:17: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.MapReduceBase;
^
JoinMapper.java:18: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.Mapper;
^
JoinMapper.java:19: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.OutputCollector;
^
JoinMapper.java:20: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.Reporter;
^
JoinMapper.java:21: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.TextOutputFormat;
^
JoinMapper.java:22: package org.apache.hadoop.mapreduce.lib.input does not exist
import org.apache.hadoop.mapreduce.lib.input.MultipleInputs;
^
JoinMapper.java:23: package org.apache.hadoop.util does not exist
import org.apache.hadoop.util.Tool;
^
JoinMapper.java:24: package org.apache.hadoop.util does not exist
import org.apache.hadoop.util.ToolRunner;
^
JoinMapper.java:26: cannot find symbol
symbol: class MapReduceBase
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> {
^
JoinMapper.java:26: cannot find symbol
symbol: class Mapper
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> {
^
JoinMapper.java:26: cannot find symbol
symbol: class LongWritable
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> {
^
JoinMapper.java:26: cannot find symbol
symbol: class Text
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> {
^
TextPair.java:2: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
TextPair.java:4: cannot find symbol
symbol: class WritableComparable
public class TextPair implements WritableComparable<TextPair> {
^
TextPair.java:4: interface expected here
public class TextPair implements WritableComparable<TextPair> {
^
JoinMapper.java:26: cannot find symbol
symbol: class Text
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> {
^
JoinMapper.java:26: interface expected here
public class JoinMapper extends MapReduceBase implements Mapper<LongWritable, Text, TextPair, Text> {
^
NcdcRecordParser.java:2: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;我知道这个版本没有我需要的软件包。但我不知道怎么解决我的问题。有人能帮我吗？
3条答案
按热度按时间j2qf4p5b1#
apachehadoop在0.20和0.23之间对其内部包进行了重大的重新命名（也称为2.0.0）。这包括重命名
org.apache.hadoop.mapred至org.apache.hadoop.mapreduce.hadoop的cloudera发行版有单独的maven构件，您可以依赖这些构件来使用旧版本进行构建。工件的名称列在cdh4maven页面上；你想要版本为
2.0.0-mr1-cdh4.1.2.另请参见使用mapred或mapreduce包创建hadoop作业是否更好？有关是否要移植到较新api的一些详细信息。
qaxu7uf22#
据我所知，这些java包应该仍然存在于hadoop的最新版本中。
我认为你的问题是hadoopjar不在你的类路径中。当我试图在没有它们的情况下编译map reduce项目时，也会遇到类似的错误。
如果您使用的是javac，请尝试以下操作（这使用的是我的ubuntucdh4 dev机器上的路径位置）：
3ks5zfa03#
对于maven，请尝试param-dhadoop.profile=23。另外，请参见https://issues.apache.org/jira/browse/hbase-4327?page=com.atlassian.jira.plugin.system.issuetabpanels：注解选项卡panel&focusedcommentid=13098596#注解-13098596