本文整理了Java中org.apache.gobblin.util.AvroUtils.getSchemaFromDataFile()
方法的一些代码示例,展示了AvroUtils.getSchemaFromDataFile()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。AvroUtils.getSchemaFromDataFile()
方法的具体详情如下:
包路径:org.apache.gobblin.util.AvroUtils
类名称:AvroUtils
方法名:getSchemaFromDataFile
[英]Get Avro schema from an Avro data file.
[中]从Avro数据文件获取Avro架构。
代码示例来源:origin: apache/incubator-gobblin
private static Schema getSchema(CombineFileSplit split, TaskAttemptContext cx, Integer idx) throws IOException {
Schema schema = AvroJob.getInputKeySchema(cx.getConfiguration());
if (schema != null) {
return schema;
}
Path path = split.getPath(idx);
FileSystem fs = path.getFileSystem(cx.getConfiguration());
return AvroUtils.getSchemaFromDataFile(path, fs);
}
代码示例来源:origin: apache/incubator-gobblin
public static Schema getNewestSchemaFromSource(Path sourceDir, FileSystem fs) throws IOException {
FileStatus[] files = fs.listStatus(sourceDir);
Arrays.sort(files, new LastModifiedDescComparator());
for (FileStatus status : files) {
if (status.isDirectory()) {
Schema schema = getNewestSchemaFromSource(status.getPath(), fs);
if (schema != null)
return schema;
} else if (FilenameUtils.isExtension(status.getPath().getName(), AVRO)) {
return AvroUtils.getSchemaFromDataFile(status.getPath(), fs);
}
}
return null;
}
代码示例来源:origin: org.apache.gobblin/gobblin-compaction
private static Schema getSchema(CombineFileSplit split, TaskAttemptContext cx, Integer idx) throws IOException {
Schema schema = AvroJob.getInputKeySchema(cx.getConfiguration());
if (schema != null) {
return schema;
}
Path path = split.getPath(idx);
FileSystem fs = path.getFileSystem(cx.getConfiguration());
return AvroUtils.getSchemaFromDataFile(path, fs);
}
代码示例来源:origin: org.apache.gobblin/gobblin-compaction
public static Schema getNewestSchemaFromSource(Path sourceDir, FileSystem fs) throws IOException {
FileStatus[] files = fs.listStatus(sourceDir);
Arrays.sort(files, new LastModifiedDescComparator());
for (FileStatus status : files) {
if (status.isDirectory()) {
Schema schema = getNewestSchemaFromSource(status.getPath(), fs);
if (schema != null)
return schema;
} else if (FilenameUtils.isExtension(status.getPath().getName(), AVRO)) {
return AvroUtils.getSchemaFromDataFile(status.getPath(), fs);
}
}
return null;
}
内容来源于网络,如有侵权,请联系作者删除!