parquet.schema.MessageType.getFieldCount()方法的使用及代码示例

x33g5p2x  于2022-01-25 转载在 其他  
字(3.0k)|赞(0)|评价(0)|浏览(123)

本文整理了Java中parquet.schema.MessageType.getFieldCount()方法的一些代码示例,展示了MessageType.getFieldCount()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。MessageType.getFieldCount()方法的具体详情如下:
包路径:parquet.schema.MessageType
类名称:MessageType
方法名:getFieldCount

MessageType.getFieldCount介绍

暂无

代码示例

代码示例来源:origin: prestodb/presto

public static parquet.schema.Type getParquetType(HiveColumnHandle column, MessageType messageType, boolean useParquetColumnNames)
  {
    if (useParquetColumnNames) {
      return getParquetTypeByName(column.getName(), messageType);
    }

    if (column.getHiveColumnIndex() < messageType.getFieldCount()) {
      return messageType.getType(column.getHiveColumnIndex());
    }
    return null;
  }
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

/**
 * Searchs column names by index on a given Parquet file schema, and returns its corresponded
 * Parquet schema types.
 *
 * @param schema Message schema where to search for column names.
 * @param colNames List of column names.
 * @param colIndexes List of column indexes.
 * @return A MessageType object of the column names found.
 */
private static MessageType getSchemaByIndex(MessageType schema, List<String> colNames, List<Integer> colIndexes) {
 List<Type> schemaTypes = new ArrayList<Type>();
 for (Integer i : colIndexes) {
  if (i < colNames.size()) {
   if (i < schema.getFieldCount()) {
    schemaTypes.add(schema.getType(i));
   } else {
    //prefixing with '_mask_' to ensure no conflict with named
    //columns in the file schema
    schemaTypes.add(Types.optional(PrimitiveTypeName.BINARY).named("_mask_" + colNames.get(i)));
   }
  }
 }
 return new MessageType(schema.getName(), schemaTypes);
}

代码示例来源:origin: dbiir/rainbow

this.columnCount = this.fileMetaDataList.get(0).getFileMetaData().getSchema().getFieldCount();

代码示例来源:origin: com.twitter/parquet-cascading

public SchemaIntersection(MessageType fileSchema, Fields requestedFields) {
 if(requestedFields == Fields.UNKNOWN)
  requestedFields = Fields.ALL;
 Fields newFields = Fields.NONE;
 List<Type> newSchemaFields = new ArrayList<Type>();
 int schemaSize = fileSchema.getFieldCount();
 for (int i = 0; i < schemaSize; i++) {
  Type type = fileSchema.getType(i);
  Fields name = new Fields(type.getName());
  if(requestedFields.contains(name)) {
   newFields = newFields.append(name);
   newSchemaFields.add(type);
  }
 }
 this.sourceFields = newFields;
 this.requestedSchema = new MessageType(fileSchema.getName(), newSchemaFields);
}

代码示例来源:origin: uk.co.nichesolutions.presto/presto-hive

public static parquet.schema.Type getParquetType(HiveColumnHandle column, MessageType messageType, boolean useParquetColumnNames)
{
  if (useParquetColumnNames) {
    return getParquetTypeByName(column.getName(), messageType);
  }
  if (column.getHiveColumnIndex() < messageType.getFieldCount()) {
    return messageType.getType(column.getHiveColumnIndex());
  }
  return null;
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

null, fileMetaData.getSchema()));
schemaSize = MessageTypeParser.parseMessageType(readContext.getReadSupportMetadata()
  .get(DataWritableReadSupport.HIVE_TABLE_AS_PARQUET_SCHEMA)).getFieldCount();
final List<BlockMetaData> splitGroup = new ArrayList<BlockMetaData>();
final long splitStart = ((FileSplit) oldSplit).getStart();

相关文章