本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.replaceTaskIdFromFilename()
方法的一些代码示例,展示了Utilities.replaceTaskIdFromFilename()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.replaceTaskIdFromFilename()
方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:replaceTaskIdFromFilename
[英]Replace the task id from the filename. It is assumed that the filename is derived from the output of getTaskId
[中]替换文件名中的任务id。假定文件名是从GetTaskKid的输出中派生出来的
代码示例来源:origin: apache/hive
/**
* Replace the task id from the filename. It is assumed that the filename is derived from the
* output of getTaskId
*
* @param filename
* filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
*/
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
return replaceTaskIdFromFilename(filename, String.valueOf(bucketNum));
}
代码示例来源:origin: apache/drill
/**
* Replace the task id from the filename. It is assumed that the filename is derived from the
* output of getTaskId
*
* @param filename
* filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
*/
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
return replaceTaskIdFromFilename(filename, String.valueOf(bucketNum));
}
代码示例来源:origin: apache/hive
private static void addBucketFileIfMissing(List<Path> result,
HashMap<String, FileStatus> taskIDToFile, String taskID1, Path bucketPath, int j) {
String taskID2 = replaceTaskId(taskID1, j);
if (!taskIDToFile.containsKey(taskID2)) {
// create empty bucket, file name should be derived from taskID2
URI bucketUri = bucketPath.toUri();
String path2 = replaceTaskIdFromFilename(bucketUri.getPath().toString(), j);
Utilities.FILE_OP_LOGGER.trace("Creating an empty bucket file {}", path2);
result.add(new Path(bucketUri.getScheme(), bucketUri.getAuthority(), path2));
}
}
代码示例来源:origin: apache/hive
public static String replaceTaskIdFromFilename(String filename, String fileId) {
String taskId = getTaskIdFromFilename(filename);
String newTaskId = replaceTaskId(taskId, fileId);
String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
return (ret);
}
代码示例来源:origin: apache/drill
public static String replaceTaskIdFromFilename(String filename, String fileId) {
String taskId = getTaskIdFromFilename(filename);
String newTaskId = replaceTaskId(taskId, fileId);
String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
return (ret);
}
代码示例来源:origin: apache/hive
Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);
this.finalPaths[writerOffset] = new Path(bDynParts ? buildTmpPath() : parent, bucketName);
this.outPaths[writerOffset] = new Path(buildTaskOutputTempPath(), bucketName);
代码示例来源:origin: apache/drill
String path2 = replaceTaskIdFromFilename(bucketUri.getPath().toString(), j);
result.add(new Path(bucketUri.getScheme(), bucketUri.getAuthority(), path2));
String path2 = replaceTaskIdFromFilename(bucketUri.getPath().toString(), j);
result.add(new Path(bucketUri.getScheme(), bucketUri.getAuthority(), path2));
代码示例来源:origin: apache/hive
taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);
代码示例来源:origin: apache/drill
taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);
代码示例来源:origin: apache/hive
if (conf.getDpSortState().equals(DPSortState.PARTITION_BUCKET_SORTED)) {
String buckNum = row.get(row.size() - 1);
taskId = Utilities.replaceTaskIdFromFilename(taskId, buckNum);
pathKey = dpAndLbDir + Path.SEPARATOR + taskId;
代码示例来源:origin: apache/drill
if(conf.getDpSortState().equals(DPSortState.PARTITION_BUCKET_SORTED)) {
String buckNum = row.get(row.size() - 1);
taskId = Utilities.replaceTaskIdFromFilename(taskId, buckNum);
pathKey = appendToSource(taskId, dpDir);
代码示例来源:origin: com.facebook.presto.hive/hive-apache
/**
* Replace the task id from the filename. It is assumed that the filename is derived from the
* output of getTaskId
*
* @param filename
* filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
*/
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
return replaceTaskIdFromFilename(filename, String.valueOf(bucketNum));
}
代码示例来源:origin: com.facebook.presto.hive/hive-apache
public static String replaceTaskIdFromFilename(String filename, String fileId) {
String taskId = getTaskIdFromFilename(filename);
String newTaskId = replaceTaskId(taskId, fileId);
String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
return (ret);
}
代码示例来源:origin: org.apache.hadoop.hive/hive-exec
/**
* Replace the task id from the filename. It is assumed that the filename is derived from the
* output of getTaskId
*
* @param filename
* filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
*/
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
String taskId = getTaskIdFromFilename(filename);
String newTaskId = replaceTaskId(taskId, bucketNum);
String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
return (ret);
}
代码示例来源:origin: org.apache.hadoop.hive/hive-exec
if (!taskIDToFile.containsKey(taskID2)) {
String path2 = replaceTaskIdFromFilename(bucketPath.toUri().getPath().toString(), j);
result.add(path2);
代码示例来源:origin: com.facebook.presto.hive/hive-apache
if (!taskIDToFile.containsKey(taskID2)) {
String path2 = replaceTaskIdFromFilename(bucketPath.toUri().getPath().toString(), j);
result.add(path2);
代码示例来源:origin: com.facebook.presto.hive/hive-apache
taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);
代码示例来源:origin: org.apache.hadoop.hive/hive-exec
LOG.info("replace taskId from execContext ");
taskId = Utilities.replaceTaskIdFromFilename(taskId, this.getExecContext().getFileId());
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), bucketNum);
代码示例来源:origin: com.facebook.presto.hive/hive-apache
if(conf.getDpSortState().equals(DPSortState.PARTITION_BUCKET_SORTED)) {
String buckNum = row.get(row.size() - 1);
taskId = Utilities.replaceTaskIdFromFilename(Utilities.getTaskId(hconf), buckNum);
pathKey = appendToSource(taskId, dpDir);
内容来源于网络,如有侵权,请联系作者删除!