线程“main”java.io.ioexception中的异常:无法初始化集群

3hvapo4f  于 2021-05-29  发布在  Hadoop
关注(0)|答案(1)|浏览(569)

我试图在windows中的eclipse上运行一个简单的hadoop map reduce程序。我得到以下异常。

Exception in thread "main" java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses.
at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:121)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:83)
at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:76)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1188)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1184)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Unknown Source)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.connect(Job.java:1183)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1212)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1236)
at com.hadoop.mapreduce.WordCountDriverClass.main(WordCountDriverClass.java:41)

这些是我在项目中添加的jar文件。

com.google.guava_1.6.0.jar
commons-configuration-1.7.jar
commons-lang-2.6.jar
commons-logging-1.1.3.jar
commons.collections-3.2.1.jar
guava-13.0.1.jar
hadoop-annotations-2.7.2.jar
hadoop-auth-2.6.0.jar
hadoop-common-2.3.0.jar
hadoop-common.jar
hadoop-mapreduce-client-core-2.0.2-alpha.jar
hadoop-mapreduce-client-core-2.7.2.jar
hadoop-mapreduce-client-jobclient-2.2.0.jar
hadoop-test-1.2.1.jar
log4j-1.2.17.jar
slf4j-api-1.7.7.jar
slf4j-simple-1.6.1.jar

在控制台中检查异常消息之后,我添加了这些jar文件。但我不能理解这个例外。谁能帮我修一下这个吗。
这是我的驾驶课。

Configuration conf = new Configuration();

        // Creating a job
        Job job = Job.getInstance(conf,"WordCountDriverClass");
        job.setJarByClass(WordCountDriverClass.class);
        job.setMapperClass(WordCountMapper.class);
        job.setReducerClass(WordCountReducer.class);

        job.setNumReduceTasks(2);
        job.setInputFormatClass(KeyValueTextInputFormat.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);
        FileInputFormat.addInputPath(job, new Path("inputfiles"));
        FileOutputFormat.setOutputPath(job, new Path("outputfiles"));

        job.waitForCompletion(true);
lvjbypge

lvjbypge1#

看起来您正在运行wordcount示例,它需要的是1.2.1hadoop核心和2.2.0hadoop公共。如果改用maven,配置应该像

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>1.2.1</version>
</dependency>
<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-common</artifactId>
    <version>2.2.0</version>
</dependency>

相关问题