org.apache.parquet.schema.MessageType.getFieldIndex()方法的使用及代码示例

x33g5p2x  于2022-01-25 转载在 其他  
字(4.2k)|赞(0)|评价(0)|浏览(70)

本文整理了Java中org.apache.parquet.schema.MessageType.getFieldIndex()方法的一些代码示例,展示了MessageType.getFieldIndex()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。MessageType.getFieldIndex()方法的具体详情如下:
包路径:org.apache.parquet.schema.MessageType
类名称:MessageType
方法名:getFieldIndex

MessageType.getFieldIndex介绍

暂无

代码示例

代码示例来源:origin: io.prestosql/presto-parquet

public static int getFieldIndex(MessageType fileSchema, String name)
{
  try {
    return fileSchema.getFieldIndex(name.toLowerCase(Locale.ENGLISH));
  }
  catch (InvalidRecordException e) {
    for (org.apache.parquet.schema.Type type : fileSchema.getFields()) {
      if (type.getName().equalsIgnoreCase(name)) {
        return fileSchema.getFieldIndex(type.getName());
      }
    }
    return -1;
  }
}

代码示例来源:origin: prestosql/presto

public static int getFieldIndex(MessageType fileSchema, String name)
{
  try {
    return fileSchema.getFieldIndex(name.toLowerCase(Locale.ENGLISH));
  }
  catch (InvalidRecordException e) {
    for (org.apache.parquet.schema.Type type : fileSchema.getFields()) {
      if (type.getName().equalsIgnoreCase(name)) {
        return fileSchema.getFieldIndex(type.getName());
      }
    }
    return -1;
  }
}

代码示例来源:origin: org.apache.apex/malhar-contrib

/**
 * Initializes {@link #activeFieldInfos} by adding fields represented by
 * fieldMapping
 *
 * @param fieldMapping
 *          String representing Parquet field name TO POJO field field name
 *          mapping
 */
private void initialiseActiveFieldInfo(String fieldMapping)
{
 String[] fields = fieldMapping.split(RECORD_SEPARATOR);
 activeFieldInfos = new ArrayList<>(fields.length);
 for (String field : fields) {
  String[] token = field.split(FIELD_SEPARATOR);
  try {
   int fieldIndex = schema.getFieldIndex(token[0]);
   PrimitiveTypeName primitiveTypeName = schema.getType(fieldIndex).asPrimitiveType().getPrimitiveTypeName();
   activeFieldInfos.add(
     new ActiveFieldInfo(getSetter(token[1], SupportType.valueOf(token[2])), primitiveTypeName, fieldIndex));
  } catch (InvalidRecordException e) {
   logger.error("{} not present in schema ", token[0]);
  } catch (UnsupportedOperationException e) {
   logger.error("{} not yet supported ", e.getMessage());
  }
 }
}

代码示例来源:origin: apache/apex-malhar

/**
 * Initializes {@link #activeFieldInfos} by adding fields represented by
 * fieldMapping
 *
 * @param fieldMapping
 *          String representing Parquet field name TO POJO field field name
 *          mapping
 */
private void initialiseActiveFieldInfo(String fieldMapping)
{
 String[] fields = fieldMapping.split(RECORD_SEPARATOR);
 activeFieldInfos = new ArrayList<>(fields.length);
 for (String field : fields) {
  String[] token = field.split(FIELD_SEPARATOR);
  try {
   int fieldIndex = schema.getFieldIndex(token[0]);
   PrimitiveTypeName primitiveTypeName = schema.getType(fieldIndex).asPrimitiveType().getPrimitiveTypeName();
   activeFieldInfos.add(
     new ActiveFieldInfo(getSetter(token[1], SupportType.valueOf(token[2])), primitiveTypeName, fieldIndex));
  } catch (InvalidRecordException e) {
   logger.error("{} not present in schema ", token[0]);
  } catch (UnsupportedOperationException e) {
   logger.error("{} not yet supported ", e.getMessage());
  }
 }
}

代码示例来源:origin: pentaho/pentaho-hadoop-shims

@Override
public ReadContext init( InitContext context ) {
 String schemaStr = context.getConfiguration().get( ParquetConverter.PARQUET_SCHEMA_CONF_KEY );
 if ( schemaStr == null ) {
  throw new RuntimeException( "Schema not defined in the PentahoParquetSchema key" );
 }
 ParquetInputFieldList schema = ParquetInputFieldList.unmarshall( schemaStr );
 converter = new ParquetConverter( schema.getFields() );
 // get all fields from file's schema
 MessageType fileSchema = context.getFileSchema();
 List<Type> newFields = new ArrayList<>();
 // use only required fields
 for ( IParquetInputField f : schema ) {
  Type origField = fileSchema.getFields().get( fileSchema.getFieldIndex( f.getFormatFieldName() ) );
  newFields.add( origField );
 }
 if ( newFields.isEmpty() ) {
  throw new RuntimeException( "Fields should be declared" );
 }
 MessageType newSchema = new MessageType( fileSchema.getName(), newFields );
 return new ReadContext( newSchema, new HashMap<>() );
}

代码示例来源:origin: org.visallo/visallo-web-structured-ingest-parquet

Object val = nameValue.getValue();
if (!(val instanceof SimpleRecord)) {
  Type type = schema.getType(schema.getFieldIndex(name));
  row.put(name, getRecordValue(val, type));

代码示例来源:origin: org.visallo/visallo-web-structured-ingest-parquet

Object val = nameValue.getValue();
if (!(val instanceof SimpleRecord)) {
  Type type = schema.getType(schema.getFieldIndex(name));
  Object recordValue = getRecordValue(val, type);
  String strValue;

相关文章