org.apache.hadoop.hive.ql.exec.Utilities.isEmptyPath()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(3.9k)|赞(0)|评价(0)|浏览(78)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.isEmptyPath()方法的一些代码示例,展示了Utilities.isEmptyPath()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.isEmptyPath()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:isEmptyPath

Utilities.isEmptyPath介绍

暂无

代码示例

代码示例来源:origin: apache/drill

@Override
 public Path call() throws Exception {
  if (!this.skipDummy && isEmptyPath(this.job, this.path, this.ctx)) {
   return createDummyFileForEmptyPartition(this.path, this.job, this.work, this.hiveScratchDir);
  }
  return this.path;
 }
}

代码示例来源:origin: apache/hive

@Override
 public Path call() throws Exception {
  if (!this.skipDummy && isEmptyPath(this.job, this.path, this.ctx)) {
   return createDummyFileForEmptyPartition(this.path, this.job, this.work.getPathToPartitionInfo().get(this.path),
       this.hiveScratchDir);
  }
  return this.path;
 }
}

代码示例来源:origin: apache/hive

private PartitionDesc changePartitionToMetadataOnly(PartitionDesc desc,
  Path path) {
 if (desc == null) {
  return null;
 }
 boolean isEmpty = false;
 try {
  isEmpty = Utilities.isEmptyPath(physicalContext.getConf(), path);
 } catch (IOException e) {
  LOG.error("Cannot determine if the table is empty", e);
 }
 desc.setInputFileFormatClass(
   isEmpty ? ZeroRowsInputFormat.class : OneNullRowInputFormat.class);
 desc.setOutputFileFormatClass(HiveIgnoreKeyTextOutputFormat.class);
 desc.getProperties().setProperty(serdeConstants.SERIALIZATION_LIB,
   NullStructSerDe.class.getName());
 return desc;
}

代码示例来源:origin: apache/drill

private PartitionDesc changePartitionToMetadataOnly(PartitionDesc desc, Path path) {
 if (desc == null) return null;
 boolean isEmpty = false;
 try {
  isEmpty = Utilities.isEmptyPath(physicalContext.getConf(), path);
 } catch (IOException e) {
  LOG.error("Cannot determine if the table is empty", e);
 }
 desc.setInputFileFormatClass(
   isEmpty ? ZeroRowsInputFormat.class : OneNullRowInputFormat.class);
 desc.setOutputFileFormatClass(HiveIgnoreKeyTextOutputFormat.class);
 desc.getProperties().setProperty(serdeConstants.SERIALIZATION_LIB,
  NullStructSerDe.class.getName());
 return desc;
}

代码示例来源:origin: apache/hive

public static boolean isEmptyPath(JobConf job, Path dirPath, Context ctx)
  throws Exception {
 if (ctx != null) {
  ContentSummary cs = ctx.getCS(dirPath);
  if (cs != null) {
   if (LOG.isDebugEnabled()) {
    LOG.debug("Content Summary cached for {} length: {} num files: {} " +
      "num directories: {}", dirPath, cs.getLength(), cs.getFileCount(),
      cs.getDirectoryCount());
   }
   return (cs.getLength() == 0 && cs.getFileCount() == 0 && cs.getDirectoryCount() <= 1);
  } else {
   LOG.debug("Content Summary not cached for {}", dirPath);
  }
 }
 return isEmptyPath(job, dirPath);
}

代码示例来源:origin: apache/drill

public static boolean isEmptyPath(JobConf job, Path dirPath, Context ctx)
  throws Exception {
 if (ctx != null) {
  ContentSummary cs = ctx.getCS(dirPath);
  if (cs != null) {
   LOG.info("Content Summary " + dirPath + "length: " + cs.getLength() + " num files: "
     + cs.getFileCount() + " num directories: " + cs.getDirectoryCount());
   return (cs.getLength() == 0 && cs.getFileCount() == 0 && cs.getDirectoryCount() <= 1);
  } else {
   LOG.info("Content Summary not cached for " + dirPath);
  }
 }
 return isEmptyPath(job, dirPath);
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

if (!Utilities.isEmptyPath(job, dirPath)) {
 FileInputFormat.addInputPath(job, dirPath);
} else {

代码示例来源:origin: com.facebook.presto.hive/hive-apache

&& isEmptyPath(job, path, ctx)) {
path = createDummyFileForEmptyPartition(path, job, work,
   hiveScratchDir, alias, sequenceNumber++);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

public static boolean isEmptyPath(JobConf job, Path dirPath, Context ctx)
  throws Exception {
 if (ctx != null) {
  ContentSummary cs = ctx.getCS(dirPath);
  if (cs != null) {
   LOG.info("Content Summary " + dirPath + "length: " + cs.getLength() + " num files: "
     + cs.getFileCount() + " num directories: " + cs.getDirectoryCount());
   return (cs.getLength() == 0 && cs.getFileCount() == 0 && cs.getDirectoryCount() <= 1);
  } else {
   LOG.info("Content Summary not cached for " + dirPath);
  }
 }
 return isEmptyPath(job, dirPath);
}

相关文章

微信公众号

最新文章

更多

Utilities类方法