org.apache.pig.impl.util.Utils类的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(7.1k)|赞(0)|评价(0)|浏览(91)

本文整理了Java中org.apache.pig.impl.util.Utils类的一些代码示例，展示了Utils类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Utils类的具体详情如下：
包路径：org.apache.pig.impl.util.Utils
类名称：Utils

Utils介绍

[英]Class with utility static methods
[中]使用实用程序静态方法初始化

代码示例

代码示例来源：origin: apache/hive

public HCatBaseStorer(String partSpecs, String schema) throws Exception {
 partitionKeys = new ArrayList<String>();
 partitions = new HashMap<String, String>();
 if (partSpecs != null && !partSpecs.trim().isEmpty()) {
  String[] partKVPs = partSpecs.split(",");
  for (String partKVP : partKVPs) {
   String[] partKV = partKVP.split("=");
   if (partKV.length == 2) {
    String partKey = partKV[0].trim();
    partitionKeys.add(partKey);
    partitions.put(partKey, partKV[1].trim());
   } else {
    throw new FrontendException("Invalid partition column specification. " + partSpecs, PigHCatUtil.PIG_EXCEPTION_CODE);
   }
  }
 }
 if (schema != null && !schema.trim().isEmpty()) {
  pigSchema = Utils.getSchemaFromString(schema);
 }
 Properties udfProps = UDFContext.getUDFContext().getUDFProperties(this.getClass(), new String[]{sign});
 onOutOfRange = OOR_VALUE_OPT_VALUES.valueOf(udfProps.getProperty(ON_OORA_VALUE_PROP, getDefaultValue().name()));
}
static OOR_VALUE_OPT_VALUES getDefaultValue() {

代码示例来源：origin: org.apache.pig/pig

public JsonLoader(String schemaString) throws IOException {
  schema = new ResourceSchema(Utils.parseSchema(schemaString));
}

代码示例来源：origin: org.apache.pig/pig

@Override
public boolean equals(Object obj) {
  if(!Utils.checkNullAndClass(this, obj)) {
    return false;
  }
  SortColInfo other = (SortColInfo)obj;
  return Utils.checkNullEquals(this.colName, other.colName, true) &&
  this.colIndex == other.colIndex && 
  this.sortOrder == other.sortOrder;
}

代码示例来源：origin: org.apache.pig/pig

public MapReduceLauncher() {
  super();
  Utils.addShutdownHookWithPriority(new HangingJobKiller(),
      PigImplConstants.SHUTDOWN_HOOK_JOB_KILL_PRIORITY);
}

代码示例来源：origin: org.apache.pig/pig

/**
 * This method is a helper for classes to implement {@link java.lang.Object#equals(java.lang.Object)}
 * The method checks whether the two arguments are both null or both not null and
 * whether they are of the same class
 * @param obj1 first object to compare
 * @param obj2 second object to compare
 * @return true if both objects are null or both are not null
 * and if both are of the same class if not null
 * false otherwise
 */
public static boolean checkNullAndClass(Object obj1, Object obj2) {
  if(checkNullEquals(obj1, obj2, false)) {
    if(obj1 != null) {
      return obj1.getClass() == obj2.getClass();
    } else {
      return true; // both obj1 and obj2 should be null
    }
  } else {
    return false;
  }
}

代码示例来源：origin: com.netflix.metacat/metacat-converters

/**
 * Converts to presto type.
 * @param pigType pig type
 * @param typeManager type manager
 * @return presto type
 */
public Type toType(final String pigType, final TypeManager typeManager) {
  try {
    final LogicalSchema schema = Utils.parseSchema(pigType);
    final LogicalSchema.LogicalFieldSchema field = schema.getField(0);
    return toPrestoType(field);
  } catch (Exception e) {
    LOG.warn("Pig Parsing failed for signature {}", pigType, e);
    throw new IllegalArgumentException(String.format("Bad type signature: '%s'", pigType));
  }
}

代码示例来源：origin: org.apache.pig/pig

@Override
public boolean equals(Object obj) {
  if(!Utils.checkNullAndClass(this, obj)) {
    return false;
  }
  SortInfo other = (SortInfo)obj;
  return (
    isGloballySorted == other.isGloballySorted &&
    Utils.checkNullEquals(sortColInfoList, other.sortColInfoList, true));
}

代码示例来源：origin: org.apache.pig/pig

private PigATSClient() {
  if (executor == null) {
    executor = Executors.newSingleThreadExecutor(
        new ThreadFactoryBuilder().setDaemon(true).setNameFormat("ATS Logger %d").build());
    YarnConfiguration yarnConf = new YarnConfiguration();
    timelineClient = TimelineClient.createTimelineClient();
    timelineClient.init(yarnConf);
    timelineClient.start();
  }
  Utils.addShutdownHookWithPriority(new Runnable() {
    @Override
    public void run() {
      timelineClient.stop();
      executor.shutdownNow();
      executor = null;
    }
  }, PigImplConstants.SHUTDOWN_HOOK_ATS_CLIENT_PRIORITY);
  log.info("Created ATS Hook");
}

代码示例来源：origin: elastic/elasticsearch-hadoop

private ResourceSchema createSchema(String schema) {
  try {
    return new ResourceSchema(Utils.getSchemaFromString(schema));
  } catch (Exception ex) {
    throw new RuntimeException(ex);
  }
}

代码示例来源：origin: org.apache.pig/pig

/**
 * @param schemaString a String representation of the Schema <b>without</b>
 *                     any enclosing curly-braces.<b>Not</b> for use with
 *                     <code>Schema#toString</code>
 * @return Schema instance
 * @throws ParserException
 */
public static Schema getSchemaFromString(String schemaString) throws ParserException {
  LogicalSchema schema = parseSchema(schemaString);
  Schema result = org.apache.pig.newplan.logical.Util.translateSchema(schema);
  Schema.setSchemaDefaultType(result, DataType.BYTEARRAY);
  return result;
}

代码示例来源：origin: elastic/elasticsearch-hadoop

private ResourceSchema createSchema(String schema) {
  try {
    return new ResourceSchema(Utils.getSchemaFromString(schema));
  } catch (Exception ex) {
    throw new RuntimeException(ex);
  }
}

代码示例来源：origin: elastic/elasticsearch-hadoop

private ResourceSchema createSchema(String schema) {
  try {
    return new ResourceSchema(Utils.getSchemaFromString(schema));
  } catch (Exception ex) {
    throw new RuntimeException(ex);
  }
}

代码示例来源：origin: elastic/elasticsearch-hadoop

private ResourceSchema createSchema(String schema) {
  try {
    return new ResourceSchema(Utils.getSchemaFromString(schema));
  } catch (Exception ex) {
    throw new RuntimeException(ex);
  }
}

代码示例来源：origin: elastic/elasticsearch-hadoop

@Test
  public void testProjection() throws Exception {
    String schemaString = "ES_PARENT: {(parent_name: chararray,parent_value: chararray)}";
    Schema schema = Utils.getSchemaFromString(schemaString);
    System.out.println(PigUtils.asProjection(schema, new Properties()));
  }
}

代码示例来源：origin: elastic/elasticsearch-hadoop

@Test(expected = Exception.class)
public void testLoadingOfBagSchema() throws Exception {
  assertNotNull(Utils.getSchemaFromString(Utils.getSchemaFromString("name:bytearray,links:{(missing:chararray)}").toString()));
}

代码示例来源：origin: elastic/elasticsearch-hadoop

@Test
public void testSchemaSerializationPlusBase64() throws Exception {
  Schema schemaFromString = Utils.getSchemaFromString("name:bytearray,links:{(missing:chararray)}");
  Schema schemaSaved = IOUtils.deserializeFromBase64(IOUtils.serializeToBase64(schemaFromString));
  assertEquals(schemaFromString.toString(), schemaSaved.toString());
}

代码示例来源：origin: org.apache.pig/pig

/**
 * @param schema
 * @return the schema represented by the string
 * @throws ParserException if the schema is invalid
 */
public static Schema schema(String schema) throws ParserException {
  return Utils.getSchemaFromString(schema);
}

代码示例来源：origin: com.twitter.elephantbird/elephant-bird-pig

@Override
public Schema outputSchema(Schema input) {
 try {
  return Utils.getSchemaFromString("json: [chararray]");
 } catch (ParserException e) {
  throw new RuntimeException(e);
 }
}

代码示例来源：origin: ShifuML/shifu

public Schema outputSchema(Schema input) {
  try {
    return Utils
      .getSchemaFromString("PopulationInfo:Tuple(columnId : int, population : chararray, unitstats : chararray)");
  } catch (ParserException e) {
    log.debug("Error when generating output schema.", e);
    // just ignore
    return null;
  }
}

代码示例来源：origin: ShifuML/shifu

public Schema outputSchema(Schema input) {
    try {
      return Utils.getSchemaFromString("BinningDataInfo:Tuple(columnId : int, binningDataInfo : chararray)");
    } catch (ParserException e) {
      log.debug("Error when generating output schema.", e);
      // just ignore
      return null;
    }
  }
}

内容来源于网络，如有侵权，请联系作者删除！

Utils

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com