org.talend.daikon.avro.AvroUtils.createEmptySchema()方法的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(7.4k)|赞(0)|评价(0)|浏览(106)

本文整理了Java中org.talend.daikon.avro.AvroUtils.createEmptySchema()方法的一些代码示例，展示了AvroUtils.createEmptySchema()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。AvroUtils.createEmptySchema()方法的具体详情如下：
包路径：org.talend.daikon.avro.AvroUtils
类名称：AvroUtils
方法名：createEmptySchema

AvroUtils.createEmptySchema介绍

[英]Creates empty record schema, i.e. record schema, which has no fields. "EmptyRecord" name is used for this schema
[中]创建没有字段的空记录模式，即记录模式。“EmptyRecord”名称用于此架构

代码示例

代码示例来源：origin: org.talend.components/simplefileio-runtime

@Override
public Schema getSchema() {
  // Simple schema container.
  final Schema[] s = new Schema[] { AvroUtils.createEmptySchema() };
  // Try to get one record and determine its schema in a callback.
  getSample(1, new Consumer<IndexedRecord>() {
    @Override
    public void accept(IndexedRecord in) {
      s[0] = in.getSchema();
    }
  });
  // Return the discovered schema.
  return s[0];
}

代码示例来源：origin: Talend/components

@Override
public Schema getSchema() {
  // Simple schema container.
  final Schema[] s = new Schema[] { AvroUtils.createEmptySchema() };
  // Try to get one record and determine its schema in a callback.
  getSample(1, new Consumer<IndexedRecord>() {
    @Override
    public void accept(IndexedRecord in) {
      s[0] = in.getSchema();
    }
  });
  // Return the discovered schema.
  return s[0];
}

代码示例来源：origin: org.talend.components/components-adapter-beam

/**
 * Create a new schema by extracting elements from the inputSchema that are *not* present in the keyPaths
 * 
 * @param inputSchema a schema
 * @param keyPaths a list of path to element that will be considered as keys
 * @return a new schema
 */
public static Schema extractValues(Schema inputSchema, List<String> keyPaths) {
  if (inputSchema == null) {
    return AvroUtils.createEmptySchema();
  }
  Schema schema = extractValues(inputSchema, keyPaths, "");
  if (schema == null) {
    return AvroUtils.createEmptySchema();
  }
  if (schema.getName() == null) {
    schema = Schema.createRecord(RECORD_VALUE_PREFIX, schema.getDoc(), schema.getNamespace(), schema.isError(),
        schema.getFields());
  }
  return schema;
}

代码示例来源：origin: Talend/components

/**
 * Create a new schema by extracting elements from the inputSchema that are *not* present in the keyPaths
 * 
 * @param inputSchema a schema
 * @param keyPaths a list of path to element that will be considered as keys
 * @return a new schema
 */
public static Schema extractValues(Schema inputSchema, List<String> keyPaths) {
  if (inputSchema == null) {
    return AvroUtils.createEmptySchema();
  }
  Schema schema = extractValues(inputSchema, keyPaths, "");
  if (schema == null) {
    return AvroUtils.createEmptySchema();
  }
  if (schema.getName() == null) {
    schema = Schema.createRecord(RECORD_VALUE_PREFIX, schema.getDoc(), schema.getNamespace(), schema.isError(),
        schema.getFields());
  }
  return schema;
}

代码示例来源：origin: Talend/components

return AvroUtils.createEmptySchema();

代码示例来源：origin: org.talend.components/kafka-runtime

s[0] = AvroUtils.createEmptySchema();

代码示例来源：origin: org.talend.components/components-adapter-beam

return AvroUtils.createEmptySchema();

代码示例来源：origin: Talend/components

s[0] = AvroUtils.createEmptySchema();

代码示例来源：origin: Talend/components

/**
 * Create a new schema by extracting elements from the inputSchema that are present in the keyPaths
 * 
 * @param inputSchema a schema
 * @param keyPaths a list of path to element that will be considered as keys
 * @return a new schema
 */
public static Schema extractKeys(Schema inputSchema, List<String> keyPaths) {
  if (inputSchema == null) {
    return AvroUtils.createEmptySchema();
  }
  // Generate the subSchema as a tree
  Map<String, Set<Object>> tree = generateTree(inputSchema, keyPaths);
  // use the generated tree to create an avro Schema
  Schema schema = convertTreeToAvroSchema(tree, TREE_ROOT_DEFAULT_VALUE, inputSchema);
  if (schema.getName() == null) {
    schema = Schema.createRecord(RECORD_KEY_PREFIX, schema.getDoc(), schema.getNamespace(), schema.isError(),
        schema.getFields());
  }
  return schema;
}

代码示例来源：origin: org.talend.components/components-adapter-beam

/**
 * Create a new schema by extracting elements from the inputSchema that are present in the keyPaths
 * 
 * @param inputSchema a schema
 * @param keyPaths a list of path to element that will be considered as keys
 * @return a new schema
 */
public static Schema extractKeys(Schema inputSchema, List<String> keyPaths) {
  if (inputSchema == null) {
    return AvroUtils.createEmptySchema();
  }
  // Generate the subSchema as a tree
  Map<String, Set<Object>> tree = generateTree(inputSchema, keyPaths);
  // use the generated tree to create an avro Schema
  Schema schema = convertTreeToAvroSchema(tree, TREE_ROOT_DEFAULT_VALUE, inputSchema);
  if (schema.getName() == null) {
    schema = Schema.createRecord(RECORD_KEY_PREFIX, schema.getDoc(), schema.getNamespace(), schema.isError(),
        schema.getFields());
  }
  return schema;
}

代码示例来源：origin: org.talend.components/processing-runtime

@Override
public PCollection<IndexedRecord> expand(PCollection<IndexedRecord> indexedRecordPCollection) {
  // Return an empty result if there are no operations in the list. This is normally not a permitted operation.
  if (operationFieldPathList.size() == 0)
    return (PCollection<IndexedRecord>) (PCollection) indexedRecordPCollection.getPipeline().apply(
        Create.empty(AvroCoder.of(AvroUtils.createEmptySchema())));
  PCollection<KV<IndexedRecord, IndexedRecord>> kv = indexedRecordPCollection
      .apply(ParDo.of(new ExtractKVFn(new ArrayList<>(groupByFieldPathList),
          new ArrayList<>(operationFieldPathList))))
      .setCoder(KvCoder.of(LazyAvroCoder.of(), LazyAvroCoder.of()));
  PCollection<KV<IndexedRecord, IndexedRecord>> aggregateResult = kv
      .apply(Combine.<IndexedRecord, IndexedRecord, IndexedRecord> perKey(new AggregateCombineFn(properties)))
      .setCoder(KvCoder.of(LazyAvroCoder.of(), NullableCoder.of(LazyAvroCoder.of())));
  PCollection<IndexedRecord> result = aggregateResult
      .apply(ParDo.of(new DoFn<KV<IndexedRecord, IndexedRecord>, KV<IndexedRecord, IndexedRecord>>() {
        @ProcessElement
        public void processElement(ProcessContext c) {
          /**
           * Filter null value when AggregateCombineFn for nothing, see {@link
           * org.talend.components.processing.runtime.aggregate.AggregateCombineFn#extractOutput(AggregateCombineFn.AggregateAccumulator)}
           */
          if (c.element().getValue() != null) {
            c.output(c.element());
          }
        }
      }))
      .apply(ParDo.of(new MergeKVFn()))
      .setCoder(LazyAvroCoder.of());
  return result;
}

代码示例来源：origin: Talend/components

@Override
public PCollection<IndexedRecord> expand(PCollection<IndexedRecord> indexedRecordPCollection) {
  // Return an empty result if there are no operations in the list. This is normally not a permitted operation.
  if (operationFieldPathList.size() == 0)
    return (PCollection<IndexedRecord>) (PCollection) indexedRecordPCollection.getPipeline().apply(
        Create.empty(AvroCoder.of(AvroUtils.createEmptySchema())));
  PCollection<KV<IndexedRecord, IndexedRecord>> kv = indexedRecordPCollection
      .apply(ParDo.of(new ExtractKVFn(new ArrayList<>(groupByFieldPathList),
          new ArrayList<>(operationFieldPathList))))
      .setCoder(KvCoder.of(LazyAvroCoder.of(), LazyAvroCoder.of()));
  PCollection<KV<IndexedRecord, IndexedRecord>> aggregateResult = kv
      .apply(Combine.<IndexedRecord, IndexedRecord, IndexedRecord> perKey(new AggregateCombineFn(properties)))
      .setCoder(KvCoder.of(LazyAvroCoder.of(), NullableCoder.of(LazyAvroCoder.of())));
  PCollection<IndexedRecord> result = aggregateResult
      .apply(ParDo.of(new DoFn<KV<IndexedRecord, IndexedRecord>, KV<IndexedRecord, IndexedRecord>>() {
        @ProcessElement
        public void processElement(ProcessContext c) {
          /**
           * Filter null value when AggregateCombineFn for nothing, see {@link
           * org.talend.components.processing.runtime.aggregate.AggregateCombineFn#extractOutput(AggregateCombineFn.AggregateAccumulator)}
           */
          if (c.element().getValue() != null) {
            c.output(c.element());
          }
        }
      }))
      .apply(ParDo.of(new MergeKVFn()))
      .setCoder(LazyAvroCoder.of());
  return result;
}

内容来源于网络，如有侵权，请联系作者删除！

AvroUtils

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com