com.google.api.services.bigquery.model.JobConfiguration.setLoad()方法的使用及代码示例

x33g5p2x  于2022-01-22 转载在 其他  
字(8.9k)|赞(0)|评价(0)|浏览(66)

本文整理了Java中com.google.api.services.bigquery.model.JobConfiguration.setLoad()方法的一些代码示例，展示了JobConfiguration.setLoad()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。JobConfiguration.setLoad()方法的具体详情如下：
包路径：com.google.api.services.bigquery.model.JobConfiguration
类名称：JobConfiguration
方法名：setLoad

JobConfiguration.setLoad介绍

[英][Pick one] Configures a load job.
[中][选择一个]配置加载作业。

代码示例

代码示例来源：origin: googleapis/google-cloud-java

.setLoad(loadConfigurationPb);

代码示例来源：origin: googleapis/google-cloud-java

.setLoad(loadConfigurationPb);

代码示例来源：origin: googleapis/google-cloud-java

@Test
public void testIncomplete() {
 // https://github.com/googleapis/google-cloud-java/issues/2357
 com.google.api.services.bigquery.model.Job job =
   new com.google.api.services.bigquery.model.Job()
     .setStatistics(
       new com.google.api.services.bigquery.model.JobStatistics()
         .setCreationTime(1234L)
         .setStartTime(5678L));
 job.setConfiguration(
   new com.google.api.services.bigquery.model.JobConfiguration()
     .setCopy(new com.google.api.services.bigquery.model.JobConfigurationTableCopy()));
 assertThat(JobStatistics.fromPb(job)).isInstanceOf(CopyStatistics.class);
 job.setConfiguration(
   new com.google.api.services.bigquery.model.JobConfiguration()
     .setLoad(new com.google.api.services.bigquery.model.JobConfigurationLoad()));
 assertThat(JobStatistics.fromPb(job)).isInstanceOf(LoadStatistics.class);
 job.setConfiguration(
   new com.google.api.services.bigquery.model.JobConfiguration()
     .setExtract(new com.google.api.services.bigquery.model.JobConfigurationExtract()));
 assertThat(JobStatistics.fromPb(job)).isInstanceOf(ExtractStatistics.class);
 job.setConfiguration(
   new com.google.api.services.bigquery.model.JobConfiguration()
     .setQuery(new com.google.api.services.bigquery.model.JobConfigurationQuery()));
 assertThat(JobStatistics.fromPb(job)).isInstanceOf(QueryStatistics.class);
}

代码示例来源：origin: com.google.cloud.bigdataoss/bigquery-connector

/**
 * Defines names of counters we track for each operation.
 *
 *  There are two types of counters: -- METHOD_NAME : Number of successful invocations of method
 * METHOD. -- METHOD_NAME_TIME : Total inclusive time spent in method METHOD.
 */
public enum Counter {
 BYTES_WRITTEN,
 CLOSE_CALLS,
 CLOSE_TOTAL_TIME,
 JOBS_INSERTED,
 WRITE_CALLS,
 WRITE_TOTAL_TIME,
}

代码示例来源：origin: org.apache.beam/beam-sdks-java-io-google-cloud-platform

/**
 * {@inheritDoc}
 *
 * <p>Tries executing the RPC for at most {@code MAX_RPC_RETRIES} times until it succeeds.
 *
 * @throws IOException if it exceeds {@code MAX_RPC_RETRIES} attempts.
 */
@Override
public void startLoadJob(JobReference jobRef, JobConfigurationLoad loadConfig)
  throws InterruptedException, IOException {
 Job job =
   new Job()
     .setJobReference(jobRef)
     .setConfiguration(new JobConfiguration().setLoad(loadConfig));
 startJob(job, errorExtractor, client);
}

代码示例来源：origin: stackoverflow.com

TableSchema schema = new TableSchema();
   schema.setFields(new ArrayList<TableFieldSchema>());
   JacksonFactory JACKSON = new JacksonFactory();
   JACKSON.createJsonParser(new FileInputStream("schema.json"))
   .parseArrayAndClose(schema.getFields(), TableFieldSchema.class, null);
   schema.setFactory(JACKSON);
   TableReference destTable = new TableReference();
   destTable.setProjectId(projectId);
   destTable.setDatasetId(datasetId);
   destTable.setTableId(tableId);
   FileContent content = new FileContent("application/octet-stream", new File(csv));
   Job job = new Job();
   JobConfiguration config = new JobConfiguration();
   JobConfigurationLoad configLoad = new JobConfigurationLoad();
   configLoad.setSchema(schema);
   configLoad.setDestinationTable(destTable);
   configLoad.setEncoding("UTF-8");
   configLoad.setCreateDisposition("CREATE_IF_NEEDED");
   config.setLoad(configLoad);
   job.setConfiguration(config);
   Insert insert = bigquery.jobs().insert(projectId, job, content);
   insert.setProjectId(projectId);
   JobReference jobRef = insert.execute().getJobReference();
   String jobId = jobRef.getJobId();

代码示例来源：origin: StreakYC/mache

loadConfig.setDestinationTable(table);
config.setLoad(loadConfig);
job.setConfiguration(config);
Insert insert = bigquery.jobs().insert(exporterConfig.getBigqueryProjectId(), job);

代码示例来源：origin: org.apache.beam/beam-sdks-java-io-google-cloud-platform

@Override
public void startLoadJob(JobReference jobRef, JobConfigurationLoad loadConfig)
  throws IOException {
 synchronized (allJobs) {
  verifyUniqueJobId(jobRef.getJobId());
  Job job = new Job();
  job.setJobReference(jobRef);
  job.setConfiguration(new JobConfiguration().setLoad(loadConfig));
  job.setKind(" bigquery#job");
  job.setStatus(new JobStatus().setState("PENDING"));
  // Copy the files to a new location for import, as the temporary files will be deleted by
  // the caller.
  if (loadConfig.getSourceUris().size() > 0) {
   ImmutableList.Builder<ResourceId> sourceFiles = ImmutableList.builder();
   ImmutableList.Builder<ResourceId> loadFiles = ImmutableList.builder();
   for (String filename : loadConfig.getSourceUris()) {
    sourceFiles.add(FileSystems.matchNewResource(filename, false /* isDirectory */));
    loadFiles.add(
      FileSystems.matchNewResource(
        filename + ThreadLocalRandom.current().nextInt(), false /* isDirectory */));
   }
   FileSystems.copy(sourceFiles.build(), loadFiles.build());
   filesForLoadJobs.put(jobRef.getProjectId(), jobRef.getJobId(), loadFiles.build());
  }
  allJobs.put(jobRef.getProjectId(), jobRef.getJobId(), new JobInfo(job));
 }
}

代码示例来源：origin: com.google.cloud.bigdataoss/bigquery-connector

config.setLoad(loadConfig);

代码示例来源：origin: com.google.cloud/google-cloud-bigquery

.setLoad(loadConfigurationPb);

代码示例来源：origin: com.google.cloud/gcloud-java-bigquery

.setLoad(loadConfigurationPb);

代码示例来源：origin: com.google.cloud/google-cloud-bigquery

.setLoad(loadConfigurationPb);

代码示例来源：origin: com.google.cloud/gcloud-java-bigquery

loadConfigurationPb.setProjectionFields(projectionFields);
return new com.google.api.services.bigquery.model.JobConfiguration()
  .setLoad(loadConfigurationPb);

代码示例来源：origin: com.google.gcloud/gcloud-java-bigquery

.setLoad(loadConfigurationPb);

代码示例来源：origin: com.google.gcloud/gcloud-java-bigquery

com.google.api.services.bigquery.model.JobConfiguration toPb() {
 JobConfigurationLoad loadConfigurationPb = new JobConfigurationLoad();
 loadConfigurationPb.setDestinationTable(destinationTable.toPb());
 if (createDisposition != null) {
  loadConfigurationPb.setCreateDisposition(createDisposition.toString());
 }
 if (writeDisposition != null) {
  loadConfigurationPb.setWriteDisposition(writeDisposition.toString());
 }
 if (csvOptions() != null) {
  CsvOptions csvOptions = csvOptions();
  loadConfigurationPb.setFieldDelimiter(csvOptions.fieldDelimiter())
    .setAllowJaggedRows(csvOptions.allowJaggedRows())
    .setAllowQuotedNewlines(csvOptions.allowQuotedNewLines())
    .setEncoding(csvOptions.encoding())
    .setQuote(csvOptions.quote())
    .setSkipLeadingRows(csvOptions.skipLeadingRows());
 }
 if (schema != null) {
  loadConfigurationPb.setSchema(schema.toPb());
 }
 if (formatOptions != null) {
  loadConfigurationPb.setSourceFormat(formatOptions.type());
 }
 loadConfigurationPb.setMaxBadRecords(maxBadRecords);
 loadConfigurationPb.setIgnoreUnknownValues(ignoreUnknownValues);
 loadConfigurationPb.setProjectionFields(projectionFields);
 return new com.google.api.services.bigquery.model.JobConfiguration()
   .setLoad(loadConfigurationPb);
}

代码示例来源：origin: io.digdag/digdag-standards

@Override
protected JobConfiguration jobConfiguration(String projectId)
{
  JobConfigurationLoad cfg = new JobConfigurationLoad()
      .setSourceUris(sourceUris(params));
  if (params.has("schema")) {
    cfg.setSchema(tableSchema(params));
  }
  Optional<DatasetReference> defaultDataset = params.getOptional("dataset", String.class)
      .transform(Bq::datasetReference);
  String destinationTable = params.get("destination_table", String.class);
  cfg.setDestinationTable(tableReference(projectId, defaultDataset, destinationTable));
  params.getOptional("create_disposition", String.class).transform(cfg::setCreateDisposition);
  params.getOptional("write_disposition", String.class).transform(cfg::setWriteDisposition);
  params.getOptional("source_format", String.class).transform(cfg::setSourceFormat);
  params.getOptional("field_delimiter", String.class).transform(cfg::setFieldDelimiter);
  params.getOptional("skip_leading_rows", int.class).transform(cfg::setSkipLeadingRows);
  params.getOptional("encoding", String.class).transform(cfg::setEncoding);
  params.getOptional("quote", String.class).transform(cfg::setQuote);
  params.getOptional("max_bad_records", int.class).transform(cfg::setMaxBadRecords);
  params.getOptional("allow_quoted_newlines", boolean.class).transform(cfg::setAllowQuotedNewlines);
  params.getOptional("allow_jagged_rows", boolean.class).transform(cfg::setAllowJaggedRows);
  params.getOptional("ignore_unknown_values", boolean.class).transform(cfg::setIgnoreUnknownValues);
  params.getOptional("projection_fields", new TypeReference<List<String>>() {}).transform(cfg::setProjectionFields);
  params.getOptional("autodetect", boolean.class).transform(cfg::setAutodetect);
  params.getOptional("schema_update_options", new TypeReference<List<String>>() {}).transform(cfg::setSchemaUpdateOptions);
  return new JobConfiguration()
      .setLoad(cfg);
}

内容来源于网络，如有侵权，请联系作者删除！

JobConfiguration

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com