本文整理了Java中parquet.schema.MessageType.getFieldCount()
方法的一些代码示例,展示了MessageType.getFieldCount()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。MessageType.getFieldCount()
方法的具体详情如下:
包路径:parquet.schema.MessageType
类名称:MessageType
方法名:getFieldCount
暂无
代码示例来源:origin: prestodb/presto
public static parquet.schema.Type getParquetType(HiveColumnHandle column, MessageType messageType, boolean useParquetColumnNames)
{
if (useParquetColumnNames) {
return getParquetTypeByName(column.getName(), messageType);
}
if (column.getHiveColumnIndex() < messageType.getFieldCount()) {
return messageType.getType(column.getHiveColumnIndex());
}
return null;
}
}
代码示例来源:origin: com.facebook.presto.hive/hive-apache
/**
* Searchs column names by index on a given Parquet file schema, and returns its corresponded
* Parquet schema types.
*
* @param schema Message schema where to search for column names.
* @param colNames List of column names.
* @param colIndexes List of column indexes.
* @return A MessageType object of the column names found.
*/
private static MessageType getSchemaByIndex(MessageType schema, List<String> colNames, List<Integer> colIndexes) {
List<Type> schemaTypes = new ArrayList<Type>();
for (Integer i : colIndexes) {
if (i < colNames.size()) {
if (i < schema.getFieldCount()) {
schemaTypes.add(schema.getType(i));
} else {
//prefixing with '_mask_' to ensure no conflict with named
//columns in the file schema
schemaTypes.add(Types.optional(PrimitiveTypeName.BINARY).named("_mask_" + colNames.get(i)));
}
}
}
return new MessageType(schema.getName(), schemaTypes);
}
代码示例来源:origin: dbiir/rainbow
this.columnCount = this.fileMetaDataList.get(0).getFileMetaData().getSchema().getFieldCount();
代码示例来源:origin: com.twitter/parquet-cascading
public SchemaIntersection(MessageType fileSchema, Fields requestedFields) {
if(requestedFields == Fields.UNKNOWN)
requestedFields = Fields.ALL;
Fields newFields = Fields.NONE;
List<Type> newSchemaFields = new ArrayList<Type>();
int schemaSize = fileSchema.getFieldCount();
for (int i = 0; i < schemaSize; i++) {
Type type = fileSchema.getType(i);
Fields name = new Fields(type.getName());
if(requestedFields.contains(name)) {
newFields = newFields.append(name);
newSchemaFields.add(type);
}
}
this.sourceFields = newFields;
this.requestedSchema = new MessageType(fileSchema.getName(), newSchemaFields);
}
代码示例来源:origin: uk.co.nichesolutions.presto/presto-hive
public static parquet.schema.Type getParquetType(HiveColumnHandle column, MessageType messageType, boolean useParquetColumnNames)
{
if (useParquetColumnNames) {
return getParquetTypeByName(column.getName(), messageType);
}
if (column.getHiveColumnIndex() < messageType.getFieldCount()) {
return messageType.getType(column.getHiveColumnIndex());
}
return null;
}
代码示例来源:origin: com.facebook.presto.hive/hive-apache
null, fileMetaData.getSchema()));
schemaSize = MessageTypeParser.parseMessageType(readContext.getReadSupportMetadata()
.get(DataWritableReadSupport.HIVE_TABLE_AS_PARQUET_SCHEMA)).getFieldCount();
final List<BlockMetaData> splitGroup = new ArrayList<BlockMetaData>();
final long splitStart = ((FileSplit) oldSplit).getStart();
内容来源于网络,如有侵权,请联系作者删除!