org.apache.hadoop.hive.ql.exec.Utilities.getFileExtension()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(6.0k)|赞(0)|评价(0)|浏览(96)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.getFileExtension()方法的一些代码示例,展示了Utilities.getFileExtension()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.getFileExtension()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:getFileExtension

Utilities.getFileExtension介绍

[英]Based on compression option and configured output codec - get extension for output file. This is only required for text files - not sequencefiles
[中]基于压缩选项和配置的输出编解码器-获取输出文件的扩展名。这仅适用于文本文件,不适用于序列文件

代码示例

代码示例来源:origin: apache/hive

/**
 * Based on compression option and configured output codec - get extension for output file. This
 * is only required for text files - not sequencefiles
 *
 * @param jc
 *          Job Configuration
 * @param isCompressed
 *          Whether the output file is compressed or not
 * @return the required file extension (example: .gz)
 * @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
 */
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
 return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}

代码示例来源:origin: apache/drill

/**
 * Based on compression option and configured output codec - get extension for output file. This
 * is only required for text files - not sequencefiles
 *
 * @param jc
 *          Job Configuration
 * @param isCompressed
 *          Whether the output file is compressed or not
 * @return the required file extension (example: .gz)
 * @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
 */
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
 return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}

代码示例来源:origin: apache/hive

public void initializeBucketPaths(int filesIdx, String taskId, boolean isNativeTable,
  boolean isSkewedStoredAsSubDirectories) {
 if (isNativeTable) {
  String extension = Utilities.getFileExtension(jc, isCompressed, hiveOutputFormat);
  String taskWithExt = extension == null ? taskId : taskId + extension;
  if (!isMmTable) {

代码示例来源:origin: apache/hive

@Test
public void testGetFileExtension() {
 JobConf jc = new JobConf();
 assertEquals("No extension for uncompressed unknown format", "",
   getFileExtension(jc, false, null));
 assertEquals("No extension for compressed unknown format", "",
   getFileExtension(jc, true, null));
 assertEquals("No extension for uncompressed text format", "",
   getFileExtension(jc, false, new HiveIgnoreKeyTextOutputFormat()));
 assertEquals("Deflate for uncompressed text format", ".deflate",
   getFileExtension(jc, true, new HiveIgnoreKeyTextOutputFormat()));
 assertEquals("No extension for uncompressed default format", "",
   getFileExtension(jc, false));
 assertEquals("Deflate for uncompressed default format", ".deflate",
   getFileExtension(jc, true));
 String extension = ".myext";
 jc.set("hive.output.file.extension", extension);
 assertEquals("Custom extension for uncompressed unknown format", extension,
   getFileExtension(jc, false, null));
 assertEquals("Custom extension for compressed unknown format", extension,
   getFileExtension(jc, true, null));
 assertEquals("Custom extension for uncompressed text format", extension,
   getFileExtension(jc, false, new HiveIgnoreKeyTextOutputFormat()));
 assertEquals("Custom extension for uncompressed text format", extension,
   getFileExtension(jc, true, new HiveIgnoreKeyTextOutputFormat()));
}

代码示例来源:origin: apache/drill

String extension = Utilities.getFileExtension(jc, isCompressed, hiveOutputFormat);
if (!bDynParts && !this.isSkewedStoredAsSubDirectories) {
 fsp.finalPaths[filesIdx] = fsp.getFinalPath(taskId, parent, extension);

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

/**
 * get the final output path of a given FileOutputFormat.
 *
 * @param parent
 *          parent dir of the expected final output path
 * @param jc
 *          job configuration
 * @deprecated
 */
@Deprecated
public static Path getOutputFormatFinalPath(Path parent, String taskId, JobConf jc,
  HiveOutputFormat<?, ?> hiveOutputFormat, boolean isCompressed,
  Path defaultFinalPath) throws IOException {
 if (hiveOutputFormat instanceof HiveIgnoreKeyTextOutputFormat) {
  return new Path(parent, taskId
    + Utilities.getFileExtension(jc, isCompressed));
 }
 return defaultFinalPath;
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

/**
 * get the final output path of a given FileOutputFormat.
 *
 * @param parent
 *          parent dir of the expected final output path
 * @param jc
 *          job configuration
 * @deprecated
 */
@Deprecated
public static Path getOutputFormatFinalPath(Path parent, String taskId, JobConf jc,
  HiveOutputFormat<?, ?> hiveOutputFormat, boolean isCompressed,
  Path defaultFinalPath) throws IOException {
 if (hiveOutputFormat instanceof HiveIgnoreKeyTextOutputFormat) {
  return new Path(parent, taskId
    + Utilities.getFileExtension(jc, isCompressed));
 }
 return defaultFinalPath;
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

/**
 * Based on compression option and configured output codec - get extension for output file. This
 * is only required for text files - not sequencefiles
 *
 * @param jc
 *          Job Configuration
 * @param isCompressed
 *          Whether the output file is compressed or not
 * @return the required file extension (example: .gz)
 * @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
 */
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
 return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

/**
 * Based on compression option and configured output codec - get extension for output file. This
 * is only required for text files - not sequencefiles
 *
 * @param jc
 *          Job Configuration
 * @param isCompressed
 *          Whether the output file is compressed or not
 * @return the required file extension (example: .gz)
 * @deprecated Use {@link #getFileExtension(JobConf, boolean, HiveOutputFormat)}
 */
@Deprecated
public static String getFileExtension(JobConf jc, boolean isCompressed) {
 return getFileExtension(jc, isCompressed, new HiveIgnoreKeyTextOutputFormat());
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

String extension = Utilities.getFileExtension(jc, isCompressed,
  hiveOutputFormat);
if (!bDynParts) {

代码示例来源:origin: com.facebook.presto.hive/hive-apache

String extension = Utilities.getFileExtension(jc, isCompressed, hiveOutputFormat);
if (!bDynParts && !this.isSkewedStoredAsSubDirectories) {
 fsp.finalPaths[filesIdx] = fsp.getFinalPath(taskId, parent, extension);

相关文章

微信公众号

最新文章

更多

Utilities类方法