org.apache.hadoop.hive.ql.exec.Utilities.getFooterCount()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(3.2k)|赞(0)|评价(0)|浏览(95)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.getFooterCount()方法的一些代码示例,展示了Utilities.getFooterCount()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.getFooterCount()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:getFooterCount

Utilities.getFooterCount介绍

[英]Get footer line count for a table.
[中]获取表的页脚行数。

代码示例

代码示例来源:origin: apache/drill

private void addSplitsForGroup(List<Path> dirs, TableScanOperator tableScan, JobConf conf,
  InputFormat inputFormat, Class<? extends InputFormat> inputFormatClass, int splits,
  TableDesc table, List<InputSplit> result) throws IOException {
 Utilities.copyTablePropertiesToConf(table, conf);
 if (tableScan != null) {
  pushFilters(conf, tableScan);
 }
 FileInputFormat.setInputPaths(conf, dirs.toArray(new Path[dirs.size()]));
 conf.setInputFormat(inputFormat.getClass());
 int headerCount = 0;
 int footerCount = 0;
 if (table != null) {
  headerCount = Utilities.getHeaderCount(table);
  footerCount = Utilities.getFooterCount(table, conf);
  if (headerCount != 0 || footerCount != 0) {
   // Input file has header or footer, cannot be splitted.
   HiveConf.setLongVar(conf, ConfVars.MAPREDMINSPLITSIZE, Long.MAX_VALUE);
  }
 }
 InputSplit[] iss = inputFormat.getSplits(conf, splits);
 for (InputSplit is : iss) {
  result.add(new HiveInputSplit(is, inputFormatClass.getName()));
 }
}

代码示例来源:origin: apache/hive

if (table != null) {
 headerCount = Utilities.getHeaderCount(table);
 footerCount = Utilities.getFooterCount(table, conf);
 if (headerCount != 0 || footerCount != 0) {

代码示例来源:origin: apache/hive

footerCount = Utilities.getFooterCount(currDesc.getTableDesc(), job);

代码示例来源:origin: apache/drill

footerCount = Utilities.getFooterCount(currDesc.getTableDesc(), job);

代码示例来源:origin: apache/hive

if (table != null) {
 headerCount = Utilities.getHeaderCount(table);
 footerCount = Utilities.getFooterCount(table, jobConf);

代码示例来源:origin: apache/drill

if (table != null) {
 headerCount = Utilities.getHeaderCount(table);
 footerCount = Utilities.getFooterCount(table, jobConf);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

private void addSplitsForGroup(List<Path> dirs, TableScanOperator tableScan, JobConf conf,
  InputFormat inputFormat, Class<? extends InputFormat> inputFormatClass, int splits,
  TableDesc table, List<InputSplit> result) throws IOException {
 Utilities.copyTablePropertiesToConf(table, conf);
 if (tableScan != null) {
  pushFilters(conf, tableScan);
 }
 FileInputFormat.setInputPaths(conf, dirs.toArray(new Path[dirs.size()]));
 conf.setInputFormat(inputFormat.getClass());
 int headerCount = 0;
 int footerCount = 0;
 if (table != null) {
  headerCount = Utilities.getHeaderCount(table);
  footerCount = Utilities.getFooterCount(table, conf);
  if (headerCount != 0 || footerCount != 0) {
   // Input file has header or footer, cannot be splitted.
   conf.setLong(
     ShimLoader.getHadoopShims().getHadoopConfNames().get("MAPREDMINSPLITSIZE"),
     Long.MAX_VALUE);
  }
 }
 InputSplit[] iss = inputFormat.getSplits(conf, splits);
 for (InputSplit is : iss) {
  result.add(new HiveInputSplit(is, inputFormatClass.getName()));
 }
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

footerCount = Utilities.getFooterCount(currDesc.getTableDesc(), job);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

if (table != null) {
 headerCount = Utilities.getHeaderCount(table);
 footerCount = Utilities.getFooterCount(table, jobConf);

相关文章

微信公众号

最新文章

更多

Utilities类方法