org.apache.hadoop.hive.ql.exec.Utilities.estimateNumberOfReducers()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(1.5k)|赞(0)|评价(0)|浏览(92)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.estimateNumberOfReducers()方法的一些代码示例,展示了Utilities.estimateNumberOfReducers()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.estimateNumberOfReducers()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:estimateNumberOfReducers

Utilities.estimateNumberOfReducers介绍

[英]Estimate the number of reducers needed for this job, based on job input, and configuration parameters. The output of this method should only be used if the output of this MapRedTask is not being used to populate a bucketed table and the user has not specified the number of reducers to use.
[中]根据作业输入和配置参数,估计此作业所需的减速器数量。仅当此MapRedTask的输出未用于填充带扣表格且用户未指定要使用的减速机数量时,才应使用此方法的输出。

代码示例

代码示例来源:origin: apache/hive

inputSummary =  Utilities.getInputSummary(driverContext.getCtx(), work.getMapWork(), null);
int reducers = Utilities.estimateNumberOfReducers(conf, inputSummary, work.getMapWork(),
                         work.isFinalMapRed());
rWork.setNumReduceTasks(reducers);

代码示例来源:origin: apache/drill

inputSummary =  Utilities.getInputSummary(driverContext.getCtx(), work.getMapWork(), null);
int reducers = Utilities.estimateNumberOfReducers(conf, inputSummary, work.getMapWork(),
                         work.isFinalMapRed());
rWork.setNumReduceTasks(reducers);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

inputSummary =  Utilities.getInputSummary(driverContext.getCtx(), work.getMapWork(), null);
int reducers = Utilities.estimateNumberOfReducers(conf, inputSummary, work.getMapWork(),
                         work.isFinalMapRed());
rWork.setNumReduceTasks(reducers);

相关文章

微信公众号

最新文章

更多

Utilities类方法