org.apache.hadoop.hive.ql.metadata.Hive.isSubDir()方法的使用及代码示例

x33g5p2x  于2022-01-20 转载在 其他  
字(3.6k)|赞(0)|评价(0)|浏览(156)

本文整理了Java中org.apache.hadoop.hive.ql.metadata.Hive.isSubDir()方法的一些代码示例,展示了Hive.isSubDir()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Hive.isSubDir()方法的具体详情如下:
包路径:org.apache.hadoop.hive.ql.metadata.Hive
类名称:Hive
方法名:isSubDir

Hive.isSubDir介绍

暂无

代码示例

代码示例来源:origin: apache/hive

public static void clearDestForSubDirSrc(final HiveConf conf, Path dest,
  Path src, boolean isSrcLocal) throws IOException {
 FileSystem destFS = dest.getFileSystem(conf);
 FileSystem srcFS = src.getFileSystem(conf);
 if (isSubDir(src, dest, srcFS, destFS, isSrcLocal)) {
  final Path fullSrcPath = getQualifiedPathWithoutSchemeAndAuthority(src, srcFS);
  final Path fullDestPath = getQualifiedPathWithoutSchemeAndAuthority(dest, destFS);
  if (fullSrcPath.equals(fullDestPath)) {
   return;
  }
  Path parent = fullSrcPath;
  while (!parent.getParent().equals(fullDestPath)) {
   parent = parent.getParent();
  }
  FileStatus[] existingFiles = destFS.listStatus(
    dest, FileUtils.HIDDEN_FILES_PATH_FILTER);
  for (FileStatus fileStatus : existingFiles) {
   if (!fileStatus.getPath().getName().equals(parent.getName())) {
    destFS.delete(fileStatus.getPath(), true);
   }
  }
 }
}

代码示例来源:origin: apache/hive

private void deleteOldPathForReplace(Path destPath, Path oldPath, HiveConf conf, boolean purge,
  PathFilter pathFilter, boolean isNeedRecycle) throws HiveException {
 Utilities.FILE_OP_LOGGER.debug("Deleting old paths for replace in " + destPath
   + " and old path " + oldPath);
 boolean isOldPathUnderDestf = false;
 try {
  FileSystem oldFs = oldPath.getFileSystem(conf);
  FileSystem destFs = destPath.getFileSystem(conf);
  // if oldPath is destf or its subdir, its should definitely be deleted, otherwise its
  // existing content might result in incorrect (extra) data.
  // But not sure why we changed not to delete the oldPath in HIVE-8750 if it is
  // not the destf or its subdir?
  isOldPathUnderDestf = isSubDir(oldPath, destPath, oldFs, destFs, false);
  if (isOldPathUnderDestf) {
   cleanUpOneDirectoryForReplace(oldPath, oldFs, pathFilter, conf, purge, isNeedRecycle);
  }
 } catch (IOException e) {
  if (isOldPathUnderDestf) {
   // if oldPath is a subdir of destf but it could not be cleaned
   throw new HiveException("Directory " + oldPath.toString()
     + " could not be cleaned up.", e);
  } else {
   //swallow the exception since it won't affect the final result
   LOG.warn("Directory " + oldPath.toString() + " cannot be cleaned: " + e, e);
  }
 }
}

代码示例来源:origin: apache/drill

public static void clearDestForSubDirSrc(final HiveConf conf, Path dest,
  Path src, boolean isSrcLocal) throws IOException {
 FileSystem destFS = dest.getFileSystem(conf);
 FileSystem srcFS = src.getFileSystem(conf);
 if (isSubDir(src, dest, srcFS, destFS, isSrcLocal)) {
  final Path fullSrcPath = getQualifiedPathWithoutSchemeAndAuthority(src, srcFS);
  final Path fullDestPath = getQualifiedPathWithoutSchemeAndAuthority(dest, destFS);
  if (fullSrcPath.equals(fullDestPath)) {
   return;
  }
  Path parent = fullSrcPath;
  while (!parent.getParent().equals(fullDestPath)) {
   parent = parent.getParent();
  }
  FileStatus[] existingFiles = destFS.listStatus(
    dest, FileUtils.HIDDEN_FILES_PATH_FILTER);
  for (FileStatus fileStatus : existingFiles) {
   if (!fileStatus.getPath().getName().equals(parent.getName())) {
    destFS.delete(fileStatus.getPath(), true);
   }
  }
 }
}

代码示例来源:origin: apache/hive

boolean srcIsSubDirOfDest = isSubDir(srcf, destf, srcFs, destFs, isSrcLocal),
  destIsSubDirOfSrc = isSubDir(destf, srcf, destFs, srcFs, false);
final String msg = "Unable to move source " + srcf + " to destination " + destf;
try {

代码示例来源:origin: apache/drill

isOldPathUnderDestf = isSubDir(oldPath, destf, oldFs, destFs, false);
if (isOldPathUnderDestf) {

代码示例来源:origin: apache/drill

boolean destIsSubDir = isSubDir(srcf, destf, srcFs, destFs, isSrcLocal);
try {
 if (inheritPerms || replace) {

代码示例来源:origin: com.facebook.presto.hive/hive-apache

boolean destIsSubDir = isSubDir(srcf, destf, fs, isSrcLocal);
try {
 if (inheritPerms || replace) {

相关文章

微信公众号

最新文章

更多

Hive类方法