我希望能够在mr作业的map阶段设置某种变量或标志,以便在作业完成后进行检查。我认为用一些代码来演示我想要的最好的方法是:p.s我正在使用hadoop2.2.0
public class MRJob {
public static class MapperTest
extends Mapper<Object, Text, Text, IntWritable>{
public void map(Object key, Text value, Context context
) throws IOException, InterruptedException {
//Do some computation to get new value and key
...
//Check if new value equal to some condition e.g if(value < 1) set global variable to true
context.write(newKey, newValue);
}
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(new Configuration(), "word_count");
//set job configs
job.waitForCompletion(true);
//Here I want to be able to check if my global variable has been set to true by any one of the mappers
}
}
1条答案
按热度按时间lmyy7pcs1#
使用
Counter
关于那件事。作业完成后,您可以检查: