org.apache.hadoop.hive.ql.exec.Utilities.replaceTaskId()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(4.2k)|赞(0)|评价(0)|浏览(88)

本文整理了Java中org.apache.hadoop.hive.ql.exec.Utilities.replaceTaskId()方法的一些代码示例,展示了Utilities.replaceTaskId()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Utilities.replaceTaskId()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.exec.Utilities
类名称:Utilities
方法名:replaceTaskId

Utilities.replaceTaskId介绍

[英]Replace taskId with input bucketNum. For example, if taskId is 000000 and bucketNum is 1, return should be 000001; if taskId is (ds%3D1)000000 and bucketNum is 1, return should be (ds%3D1)000001. This method is different from the replaceTaskId(String, String) method. In this method, the pattern is in taskId.
[中]将taskId替换为输入bucketNum。例如,如果taskId为000000,bucketNum为1,则返回值应为000001;如果taskId为(ds%3D1)000000,bucketNum为1,则返回值应为(ds%3D1)000001。此方法不同于replaceTaskId(String,String)方法。在这个方法中,模式在taskId中。

代码示例

代码示例来源:origin: apache/hive

private static String getPathName(int taskId) {
 return Utilities.replaceTaskId("000000", taskId) + "_0";
}

代码示例来源:origin: apache/hive

public static String replaceTaskIdFromFilename(String filename, String fileId) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, fileId);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: apache/drill

public static String replaceTaskIdFromFilename(String filename, String fileId) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, fileId);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: apache/hive

@Test
public void testReplaceTaskId() {
 String taskID = "000000";
 int bucketNum = 1;
 String newTaskID = Utilities.replaceTaskId(taskID, bucketNum);
 Assert.assertEquals("000001", newTaskID);
 taskID = "(ds%3D1)000001";
 newTaskID = Utilities.replaceTaskId(taskID, 5);
 Assert.assertEquals("(ds%3D1)000005", newTaskID);
}

代码示例来源:origin: apache/hive

private static void addBucketFileIfMissing(List<Path> result,
  HashMap<String, FileStatus> taskIDToFile, String taskID1, Path bucketPath, int j) {
 String taskID2 = replaceTaskId(taskID1, j);
 if (!taskIDToFile.containsKey(taskID2)) {
  // create empty bucket, file name should be derived from taskID2
  URI bucketUri = bucketPath.toUri();
  String path2 = replaceTaskIdFromFilename(bucketUri.getPath().toString(), j);
  Utilities.FILE_OP_LOGGER.trace("Creating an empty bucket file {}", path2);
  result.add(new Path(bucketUri.getScheme(), bucketUri.getAuthority(), path2));
 }
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

private static String replaceTaskId(String taskId, int bucketNum) {
 return replaceTaskId(taskId, String.valueOf(bucketNum));
}

代码示例来源:origin: apache/drill

Path bucketPath = taskIDToFile.values().iterator().next().getPath();
 for (int j = 0; j < dpCtx.getNumBuckets(); ++j) {
  String taskID2 = replaceTaskId(taskID1, j);
  if (!taskIDToFile.containsKey(taskID2)) {
Path bucketPath = taskIDToFile.values().iterator().next().getPath();
for (int j = 0; j < conf.getTable().getNumBuckets(); ++j) {
 String taskID2 = replaceTaskId(taskID1, j);
 if (!taskIDToFile.containsKey(taskID2)) {

代码示例来源:origin: com.facebook.presto.hive/hive-apache

public static String replaceTaskIdFromFilename(String filename, String fileId) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, fileId);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

/**
 * Replace the task id from the filename. It is assumed that the filename is derived from the
 * output of getTaskId
 *
 * @param filename
 *          filename to replace taskid "0_0" or "0_0.gz" by 33 to "33_0" or "33_0.gz"
 */
public static String replaceTaskIdFromFilename(String filename, int bucketNum) {
 String taskId = getTaskIdFromFilename(filename);
 String newTaskId = replaceTaskId(taskId, bucketNum);
 String ret = replaceTaskIdFromFilename(filename, taskId, newTaskId);
 return (ret);
}

代码示例来源:origin: org.apache.hadoop.hive/hive-exec

Path bucketPath = taskIDToFile.values().iterator().next().getPath();
for (int j = 0; j < dpCtx.getNumBuckets(); ++j) {
 String taskID2 = replaceTaskId(taskID1, j);
 if (!taskIDToFile.containsKey(taskID2)) {

代码示例来源:origin: com.facebook.presto.hive/hive-apache

Path bucketPath = taskIDToFile.values().iterator().next().getPath();
for (int j = 0; j < dpCtx.getNumBuckets(); ++j) {
 String taskID2 = replaceTaskId(taskID1, j);
 if (!taskIDToFile.containsKey(taskID2)) {

相关文章

微信公众号

最新文章

更多

Utilities类方法