org.apache.hadoop.hive.ql.exec.Utilities.createEmptyBuckets()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(1.7k)|赞(0)|评价(0)|浏览(80)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.createEmptyBuckets()方法的一些代码示例,展示了Utilities.createEmptyBuckets()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.createEmptyBuckets()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:createEmptyBuckets

Utilities.createEmptyBuckets介绍

[英]Check the existence of buckets according to bucket specification. Create empty buckets if needed.
[中]根据铲斗规格检查铲斗是否存在。如果需要,创建空桶。

代码示例

代码示例来源:origin: apache/hive

Utilities.createEmptyBuckets(hconf, emptyBuckets, mbc.isCompressed, mbc.tableInfo, reporter);

代码示例来源:origin: apache/hive

createEmptyBuckets(
  hconf, emptyBuckets, conf.getCompressed(), conf.getTableInfo(), reporter);
filesKept.addAll(emptyBuckets);

代码示例来源:origin: apache/drill

createEmptyBuckets(hconf, emptyBuckets, conf, reporter);
perfLogger.PerfLogEnd("FileSinkOperator", "CreateEmptyBuckets");

代码示例来源:origin: com.facebook.presto.hive/hive-apache

public static void mvFileToFinalPath(Path specPath, Configuration hconf,
  boolean success, Log log, DynamicPartitionCtx dpCtx, FileSinkDesc conf,
  Reporter reporter) throws IOException,
  HiveException {
 FileSystem fs = specPath.getFileSystem(hconf);
 Path tmpPath = Utilities.toTempPath(specPath);
 Path taskTmpPath = Utilities.toTaskTempPath(specPath);
 if (success) {
  if (fs.exists(tmpPath)) {
   // remove any tmp file or double-committed output files
   ArrayList<String> emptyBuckets =
     Utilities.removeTempOrDuplicateFiles(fs, tmpPath, dpCtx);
   // create empty buckets if necessary
   if (emptyBuckets.size() > 0) {
    createEmptyBuckets(hconf, emptyBuckets, conf, reporter);
   }
   // move to the file destination
   log.info("Moving tmp dir: " + tmpPath + " to: " + specPath);
   Utilities.renameOrMoveFiles(fs, tmpPath, specPath);
  }
 } else {
  fs.delete(tmpPath, true);
 }
 fs.delete(taskTmpPath, true);
}

相关文章

微信公众号

最新文章

更多

Utilities类方法