org.apache.hadoop.hive.ql.exec.Utilities.replaceTaskIdFromFilename()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(6.1k)|赞(0)|评价(0)|浏览(79)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.replaceTaskIdFromFilename()方法的一些代码示例,展示了Utilities.replaceTaskIdFromFilename()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.replaceTaskIdFromFilename()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:replaceTaskIdFromFilename

Utilities.replaceTaskIdFromFilename介绍

[英]Replace the task id from the filename. It is assumed that the filename is derived from the output of getTaskId
[中]替换文件名中的任务id。假定文件名是从GetTaskKid的输出中派生出来的

代码示例

代码示例来源:origin: apache/hive

/**
 * Replace the task id from the filename. It is assumed that the filename is derived from the
 * output of getTaskId
 *
 * @param filename
 *          filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
 */
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
 return replaceTaskIdFromFilename(filename, String.valueOf(bucketNum));
}

代码示例来源:origin: apache/drill

/**
 * Replace the task id from the filename. It is assumed that the filename is derived from the
 * output of getTaskId
 *
 * @param filename
 *          filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
 */
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
 return replaceTaskIdFromFilename(filename, String.valueOf(bucketNum));
}

代码示例来源:origin: apache/hive

private static void addBucketFileIfMissing(List<Path> result,
  HashMap<String, FileStatus> taskIDToFile, String taskID1, Path bucketPath, int j) {
 String taskID2 = replaceTaskId(taskID1, j);
 if (!taskIDToFile.containsKey(taskID2)) {
  // create empty bucket, file name should be derived from taskID2
  URI bucketUri = bucketPath.toUri();
  String path2 = replaceTaskIdFromFilename(bucketUri.getPath().toString(), j);
  Utilities.FILE_OP_LOGGER.trace("Creating an empty bucket file {}", path2);
  result.add(new Path(bucketUri.getScheme(), bucketUri.getAuthority(), path2));
 }
}

代码示例来源:origin: apache/hive

public static String replaceTaskIdFromFilename(String filename, String fileId) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, fileId);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: apache/drill

public static String replaceTaskIdFromFilename(String filename, String fileId) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, fileId);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: apache/hive

Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);
this.finalPaths[writerOffset] = new Path(bDynParts ? buildTmpPath() : parent, bucketName);
this.outPaths[writerOffset] = new Path(buildTaskOutputTempPath(), bucketName);

代码示例来源:origin: apache/drill

String path2 = replaceTaskIdFromFilename(bucketUri.getPath().toString(), j);
 result.add(new Path(bucketUri.getScheme(), bucketUri.getAuthority(), path2));
String path2 = replaceTaskIdFromFilename(bucketUri.getPath().toString(), j);
result.add(new Path(bucketUri.getScheme(), bucketUri.getAuthority(), path2));

代码示例来源:origin: apache/hive

taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);

代码示例来源:origin: apache/drill

taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);

代码示例来源:origin: apache/hive

if (conf.getDpSortState().equals(DPSortState.PARTITION_BUCKET_SORTED)) {
 String buckNum = row.get(row.size() - 1);
 taskId = Utilities.replaceTaskIdFromFilename(taskId, buckNum);
 pathKey = dpAndLbDir + Path.SEPARATOR + taskId;

代码示例来源:origin: apache/drill

if(conf.getDpSortState().equals(DPSortState.PARTITION_BUCKET_SORTED)) {
 String buckNum = row.get(row.size() - 1);
 taskId = Utilities.replaceTaskIdFromFilename(taskId, buckNum);
 pathKey = appendToSource(taskId, dpDir);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

/**
 * Replace the task id from the filename. It is assumed that the filename is derived from the
 * output of getTaskId
 *
 * @param filename
 *          filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
 */
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
 return replaceTaskIdFromFilename(filename, String.valueOf(bucketNum));
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

public static String replaceTaskIdFromFilename(String filename, String fileId) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, fileId);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

/**
 * Replace the task id from the filename. It is assumed that the filename is derived from the
 * output of getTaskId
 *
 * @param filename
 *          filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
 */
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, bucketNum);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

if (!taskIDToFile.containsKey(taskID2)) {
 String path2 = replaceTaskIdFromFilename(bucketPath.toUri().getPath().toString(), j);
 result.add(path2);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

if (!taskIDToFile.containsKey(taskID2)) {
 String path2 = replaceTaskIdFromFilename(bucketPath.toUri().getPath().toString(), j);
 result.add(path2);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

LOG.info("replace taskId from execContext ");
taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);

代码示例来源:origin: com.facebook.presto.hive/hive-apache

if(conf.getDpSortState().equals(DPSortState.PARTITION_BUCKET_SORTED)) {
 String buckNum = row.get(row.size() - 1);
 taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), buckNum);
 pathKey = appendToSource(taskId, dpDir);

相关文章

微信公众号

最新文章

更多

Utilities类方法