本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.estimateNumberOfReducers()
方法的一些代码示例,展示了Utilities.estimateNumberOfReducers()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.estimateNumberOfReducers()
方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:estimateNumberOfReducers
[英]Estimate the number of reducers needed for this job, based on job input, and configuration parameters. The output of this method should only be used if the output of this MapRedTask is not being used to populate a bucketed table and the user has not specified the number of reducers to use.
[中]根据作业输入和配置参数,估计此作业所需的减速器数量。仅当此MapRedTask的输出未用于填充带扣表格且用户未指定要使用的减速机数量时,才应使用此方法的输出。
代码示例来源:origin: apache/hive
inputSummary = Utilities.getInputSummary(driverContext.getCtx(), work.getMapWork(), null);
int reducers = Utilities.estimateNumberOfReducers(conf, inputSummary, work.getMapWork(),
work.isFinalMapRed());
rWork.setNumReduceTasks(reducers);
代码示例来源:origin: apache/drill
inputSummary = Utilities.getInputSummary(driverContext.getCtx(), work.getMapWork(), null);
int reducers = Utilities.estimateNumberOfReducers(conf, inputSummary, work.getMapWork(),
work.isFinalMapRed());
rWork.setNumReduceTasks(reducers);
代码示例来源:origin: com.facebook.presto.hive/hive-apache
inputSummary = Utilities.getInputSummary(driverContext.getCtx(), work.getMapWork(), null);
int reducers = Utilities.estimateNumberOfReducers(conf, inputSummary, work.getMapWork(),
work.isFinalMapRed());
rWork.setNumReduceTasks(reducers);
内容来源于网络,如有侵权,请联系作者删除!