org.apache.gobblin.util.AvroUtils.getDirectorySchema()方法的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(2.9k)|赞(0)|评价(0)|浏览(121)

本文整理了Java中org.apache.gobblin.util.AvroUtils.getDirectorySchema()方法的一些代码示例,展示了AvroUtils.getDirectorySchema()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。AvroUtils.getDirectorySchema()方法的具体详情如下:
包路径:org.apache.gobblin.util.AvroUtils
类名称:AvroUtils
方法名:getDirectorySchema

AvroUtils.getDirectorySchema介绍

[英]Get the latest avro schema for a directory
[中]获取目录的最新avro架构

代码示例

代码示例来源:origin: apache/incubator-gobblin

/**
 * Get schema for a directory using {@link AvroUtils#getDirectorySchema(Path, FileSystem, boolean)}.
 */
protected Schema getDirectorySchema(Path directory) throws IOException {
 return AvroUtils.getDirectorySchema(directory, this.fs, true);
}

代码示例来源:origin: apache/incubator-gobblin

/**
 * Get the latest avro schema for a directory
 * @param directory the input dir that contains avro files
 * @param conf configuration
 * @param latest true to return latest schema, false to return oldest schema
 * @return the latest/oldest schema in the directory
 * @throws IOException
 */
public static Schema getDirectorySchema(Path directory, Configuration conf, boolean latest) throws IOException {
 return getDirectorySchema(directory, FileSystem.get(conf), latest);
}

代码示例来源:origin: apache/incubator-gobblin

@Test
public void testGetDirectorySchema() throws IOException {
 Configuration conf = new Configuration();
 conf.set("fs.default.name", "file:///");
 conf.set("mapred.job.tracker", "local");
 Path mockAvroFilePath = new Path(AVRO_DIR);
 Assert.assertNotNull(AvroUtils.getDirectorySchema(mockAvroFilePath, conf, true));
}

代码示例来源:origin: apache/incubator-gobblin

Schema prevSchema = AvroUtils.getDirectorySchema(originalSchemaPath, conf, false);
Schema mergedSchema = AvroUtils.nullifyFieldsForSchemaMerge(prevSchema, currentAvroSchema);
return mergedSchema;

代码示例来源:origin: apache/incubator-gobblin

Schema schema = AvroUtils.getDirectorySchema(new Path(sd.getLocation()), this.fs, true);

代码示例来源:origin: org.apache.gobblin/gobblin-hive-registration

/**
 * Get schema for a directory using {@link AvroUtils#getDirectorySchema(Path, FileSystem, boolean)}.
 */
protected Schema getDirectorySchema(Path directory) throws IOException {
 return AvroUtils.getDirectorySchema(directory, this.fs, true);
}

代码示例来源:origin: org.apache.gobblin/gobblin-utility

/**
 * Get the latest avro schema for a directory
 * @param directory the input dir that contains avro files
 * @param conf configuration
 * @param latest true to return latest schema, false to return oldest schema
 * @return the latest/oldest schema in the directory
 * @throws IOException
 */
public static Schema getDirectorySchema(Path directory, Configuration conf, boolean latest) throws IOException {
 return getDirectorySchema(directory, FileSystem.get(conf), latest);
}

代码示例来源:origin: org.apache.gobblin/gobblin-data-management

Schema schema = AvroUtils.getDirectorySchema(new Path(sd.getLocation()), this.fs, true);

代码示例来源:origin: org.apache.gobblin/gobblin-core

Schema prevSchema = AvroUtils.getDirectorySchema(originalSchemaPath, conf, false);
Schema mergedSchema = AvroUtils.nullifyFieldsForSchemaMerge(prevSchema, currentAvroSchema);
return mergedSchema;

相关文章