本文整理了Java中org.apache.gobblin.util.WriterUtils.getWriterStagingDir()方法的一些代码示例，展示了WriterUtils.getWriterStagingDir()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。WriterUtils.getWriterStagingDir()方法的具体详情如下：
包路径：org.apache.gobblin.util.WriterUtils
类名称：WriterUtils
方法名：getWriterStagingDir

WriterUtils.getWriterStagingDir介绍

[英]Get the Path corresponding the to the directory a given org.apache.gobblin.writer.DataWriter should be writing its staging data. The staging data directory is determined by combining the ConfigurationKeys#WRITER_STAGING_DIR and the ConfigurationKeys#WRITER_FILE_PATH.
[中]获取与给定组织的目录对应的路径。阿帕奇。戈布林。作家DataWriter应该正在写入其暂存数据。暂存数据目录由ConfigurationKeys#WRITER_staging_DIR和ConfigurationKeys#WRITER_FILE_路径组合而成。

代码示例

代码示例来源：origin: apache/incubator-gobblin

/**
 * Get the staging {@link Path} for {@link org.apache.gobblin.writer.DataWriter} that has attemptId in the path.
 */
public static Path getWriterStagingDir(State state, int numBranches, int branchId, String attemptId) {
 Preconditions.checkArgument(attemptId != null && !attemptId.isEmpty(), "AttemptId cannot be null or empty: " + attemptId);
 return new Path(getWriterStagingDir(state, numBranches, branchId), attemptId);
}

代码示例来源：origin: apache/incubator-gobblin

/**
 * Cleanup staging data of a Gobblin task.
 *
 * @param state a {@link State} instance storing task configuration properties
 * @param logger a {@link Logger} used for logging
 */
public static void cleanTaskStagingData(State state, Logger logger) throws IOException {
 int numBranches = state.getPropAsInt(ConfigurationKeys.FORK_BRANCHES_KEY, 1);
 for (int branchId = 0; branchId < numBranches; branchId++) {
  String writerFsUri = state.getProp(
    ForkOperatorUtils.getPropertyNameForBranch(ConfigurationKeys.WRITER_FILE_SYSTEM_URI, numBranches, branchId),
    ConfigurationKeys.LOCAL_FS_URI);
  FileSystem fs = getFsWithProxy(state, writerFsUri, WriterUtils.getFsConfiguration(state));
  Path stagingPath = WriterUtils.getWriterStagingDir(state, numBranches, branchId);
  if (fs.exists(stagingPath)) {
   logger.info("Cleaning up staging directory " + stagingPath.toUri().getPath());
   if (!fs.delete(stagingPath, true)) {
    throw new IOException("Clean up staging directory " + stagingPath.toUri().getPath() + " failed");
   }
  }
  Path outputPath = WriterUtils.getWriterOutputDir(state, numBranches, branchId);
  if (fs.exists(outputPath)) {
   logger.info("Cleaning up output directory " + outputPath.toUri().getPath());
   if (!fs.delete(outputPath, true)) {
    throw new IOException("Clean up output directory " + outputPath.toUri().getPath() + " failed");
   }
  }
 }
}

代码示例来源：origin: apache/incubator-gobblin

ParallelRunner parallelRunner = getParallelRunner(fs, closer, parallelRunnerThreads, parallelRunners);
Path stagingPath = WriterUtils.getWriterStagingDir(state, numBranches, branchId);
if (fs.exists(stagingPath)) {
 logger.info("Cleaning up staging directory " + stagingPath.toUri().getPath());

代码示例来源：origin: apache/incubator-gobblin

.getWriterStagingDir(state, numBranches, branchId, this.writerAttemptIdOptional.get())
  : WriterUtils.getWriterStagingDir(state, numBranches, branchId);
this.outputDir = getOutputDir(state);
this.copyableDatasetMetadata =

代码示例来源：origin: apache/incubator-gobblin

@Test
public void testGetWriterDir() {
 State state = new State();
 state.setProp(ConfigurationKeys.WRITER_STAGING_DIR, TEST_WRITER_STAGING_DIR);
 state.setProp(ConfigurationKeys.WRITER_OUTPUT_DIR, TEST_WRITER_OUTPUT_DIR);
 state.setProp(ConfigurationKeys.WRITER_FILE_PATH, TEST_WRITER_FILE_PATH);
 Assert.assertEquals(WriterUtils.getWriterStagingDir(state, 0, 0), new Path(TEST_WRITER_STAGING_DIR,
   TEST_WRITER_FILE_PATH));
 state.setProp(ConfigurationKeys.WRITER_STAGING_DIR + ".0", TEST_WRITER_STAGING_DIR);
 state.setProp(ConfigurationKeys.WRITER_OUTPUT_DIR + ".0", TEST_WRITER_OUTPUT_DIR);
 state.setProp(ConfigurationKeys.WRITER_FILE_PATH + ".0", TEST_WRITER_FILE_PATH);
 Assert.assertEquals(WriterUtils.getWriterStagingDir(state, 2, 0), new Path(TEST_WRITER_STAGING_DIR,
   TEST_WRITER_FILE_PATH));
 state.setProp(ConfigurationKeys.WRITER_STAGING_DIR + ".1", TEST_WRITER_STAGING_DIR);
 state.setProp(ConfigurationKeys.WRITER_OUTPUT_DIR + ".1", TEST_WRITER_OUTPUT_DIR);
 state.setProp(ConfigurationKeys.WRITER_FILE_PATH + ".1", TEST_WRITER_FILE_PATH);
 Assert.assertEquals(WriterUtils.getWriterStagingDir(state, 2, 1), new Path(TEST_WRITER_STAGING_DIR,
   TEST_WRITER_FILE_PATH));
}

代码示例来源：origin: apache/incubator-gobblin

.getWriterStagingDir(properties, this.numBranches, this.branchId, this.writerAttemptIdOptional.get())
  : WriterUtils.getWriterStagingDir(properties, this.numBranches, this.branchId);
this.stagingFile = new Path(writerStagingDir, this.fileName);

代码示例来源：origin: org.apache.gobblin/gobblin-utility

/**
 * Get the staging {@link Path} for {@link org.apache.gobblin.writer.DataWriter} that has attemptId in the path.
 */
public static Path getWriterStagingDir(State state, int numBranches, int branchId, String attemptId) {
 Preconditions.checkArgument(attemptId != null && !attemptId.isEmpty(), "AttemptId cannot be null or empty: " + attemptId);
 return new Path(getWriterStagingDir(state, numBranches, branchId), attemptId);
}

代码示例来源：origin: org.apache.gobblin/gobblin-utility

/**
 * Cleanup staging data of a Gobblin task.
 *
 * @param state a {@link State} instance storing task configuration properties
 * @param logger a {@link Logger} used for logging
 */
public static void cleanTaskStagingData(State state, Logger logger) throws IOException {
 int numBranches = state.getPropAsInt(ConfigurationKeys.FORK_BRANCHES_KEY, 1);
 for (int branchId = 0; branchId < numBranches; branchId++) {
  String writerFsUri = state.getProp(
    ForkOperatorUtils.getPropertyNameForBranch(ConfigurationKeys.WRITER_FILE_SYSTEM_URI, numBranches, branchId),
    ConfigurationKeys.LOCAL_FS_URI);
  FileSystem fs = getFsWithProxy(state, writerFsUri, WriterUtils.getFsConfiguration(state));
  Path stagingPath = WriterUtils.getWriterStagingDir(state, numBranches, branchId);
  if (fs.exists(stagingPath)) {
   logger.info("Cleaning up staging directory " + stagingPath.toUri().getPath());
   if (!fs.delete(stagingPath, true)) {
    throw new IOException("Clean up staging directory " + stagingPath.toUri().getPath() + " failed");
   }
  }
  Path outputPath = WriterUtils.getWriterOutputDir(state, numBranches, branchId);
  if (fs.exists(outputPath)) {
   logger.info("Cleaning up output directory " + outputPath.toUri().getPath());
   if (!fs.delete(outputPath, true)) {
    throw new IOException("Clean up output directory " + outputPath.toUri().getPath() + " failed");
   }
  }
 }
}

代码示例来源：origin: org.apache.gobblin/gobblin-utility

ParallelRunner parallelRunner = getParallelRunner(fs, closer, parallelRunnerThreads, parallelRunners);
Path stagingPath = WriterUtils.getWriterStagingDir(state, numBranches, branchId);
if (fs.exists(stagingPath)) {
 logger.info("Cleaning up staging directory " + stagingPath.toUri().getPath());

代码示例来源：origin: org.apache.gobblin/gobblin-data-management

.getWriterStagingDir(state, numBranches, branchId, this.writerAttemptIdOptional.get())
  : WriterUtils.getWriterStagingDir(state, numBranches, branchId);
this.outputDir = getOutputDir(state);
this.copyableDatasetMetadata =

代码示例来源：origin: org.apache.gobblin/gobblin-core

.getWriterStagingDir(properties, this.numBranches, this.branchId, this.writerAttemptIdOptional.get())
  : WriterUtils.getWriterStagingDir(properties, this.numBranches, this.branchId);
this.stagingFile = new Path(writerStagingDir, this.fileName);