org.apache.hadoop.hive.ql.exec.Utilities.parseSplitBucket()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(1.6k)|赞(0)|评价(0)|浏览(77)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.parseSplitBucket()方法的一些代码示例,展示了Utilities.parseSplitBucket()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.parseSplitBucket()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:parseSplitBucket

Utilities.parseSplitBucket介绍

暂无

代码示例

代码示例来源:origin: apache/hive

private InputSplit[] pruneBuckets(MapWork work, InputSplit[] splits) {
 final BitSet buckets = work.getIncludedBuckets();
 final String bucketIn = buckets.toString();
 List<InputSplit> filteredSplits = new ArrayList<InputSplit>(splits.length / 2);
 for (InputSplit split : splits) {
  final int bucket = Utilities.parseSplitBucket(split);
  if (bucket < 0 || buckets.get(bucket)) {
   // match or UNKNOWN
   filteredSplits.add(split);
  } else {
   LOG.info("Pruning with IN ({}) - removing {}", bucketIn, split);
  }
 }
 if (filteredSplits.size() < splits.length) {
  // reallocate only if any filters pruned
  splits = filteredSplits.toArray(new InputSplit[filteredSplits.size()]);
 }
 return splits;
}

代码示例来源:origin: apache/drill

private InputSplit[] pruneBuckets(MapWork work, InputSplit[] splits) {
 final BitSet buckets = work.getIncludedBuckets();
 final String bucketIn = buckets.toString();
 List<InputSplit> filteredSplits = new ArrayList<InputSplit>(splits.length / 2);
 for (InputSplit split : splits) {
  final int bucket = Utilities.parseSplitBucket(split);
  if (bucket < 0 || buckets.get(bucket)) {
   // match or UNKNOWN
   filteredSplits.add(split);
  } else {
   LOG.info("Pruning with IN ({}) - removing {}", bucketIn, split);
  }
 }
 if (filteredSplits.size() < splits.length) {
  // reallocate only if any filters pruned
  splits = filteredSplits.toArray(new InputSplit[filteredSplits.size()]);
 }
 return splits;
}

相关文章

微信公众号

最新文章

更多

Utilities类方法