org.apache.hadoop.yarn.util.Apps类的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(7.9k)|赞(0)|评价(0)|浏览(438)

本文整理了Java中org.apache.hadoop.yarn.util.Apps类的一些代码示例，展示了Apps类的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Apps类的具体详情如下：
包路径：org.apache.hadoop.yarn.util.Apps
类名称：Apps

Apps介绍

[英]YARN internal application-related utilities
[中]纱线内部应用程序相关实用程序

代码示例

代码示例来源：origin: apache/ignite

/**
   * @param envs Environment variables.
   * @param conf Yarn configuration.
   */
  private static void setupAppMasterEnv(Map<String, String> envs, YarnConfiguration conf) {
    for (String c : conf.getStrings(YarnConfiguration.YARN_APPLICATION_CLASSPATH,
      YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH))
      Apps.addToEnvironment(envs, Environment.CLASSPATH.name(),
          c.trim(), File.pathSeparator);

    Apps.addToEnvironment(envs,
        Environment.CLASSPATH.name(),
        Environment.PWD.$() + File.separator + "*",
        File.pathSeparator);
  }
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

public static ApplicationId toAppID(String aid) {
 Iterator<String> it = _split(aid).iterator();
 return toAppID(APP, aid, it);
}

代码示例来源：origin: apache/oozie

@Override
protected void addActionSpecificEnvVars(Map<String, String> env) {
  Apps.setEnvFromInputString(env, "PATH=.:$PATH", File.pathSeparator);
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-common

public static ApplicationId toAppID(String prefix, String s, Iterator<String> it) {
 if (!it.hasNext() || !it.next().equals(prefix)) {
  throwParseException(sjoin(prefix, ID), s);
 }
 shouldHaveNext(prefix, s, it);
 ApplicationId appId = ApplicationId.newInstance(Long.parseLong(it.next()),
   Integer.parseInt(it.next()));
 return appId;
}

代码示例来源：origin: org.apache.hadoop/hadoop-mapreduce-client-common

public static void setEnvFromInputProperty(Map<String, String> env,
  String propName, String defaultPropValue, Configuration conf) {
 String classPathSeparator =
   conf.getBoolean(MRConfig.MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM,
     MRConfig.DEFAULT_MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM)
     ? ApplicationConstants.CLASS_PATH_SEPARATOR : File.pathSeparator;
 Apps.setEnvFromInputProperty(env, propName, defaultPropValue, conf,
   classPathSeparator);
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-nodemanager

Apps.setEnvFromInputProperty(environment,
  YarnConfiguration.NM_ADMIN_USER_ENV, defEnvStr, conf,
  File.pathSeparator);
nmVars.addAll(Apps.getEnvVarsFromInputProperty(
  YarnConfiguration.NM_ADMIN_USER_ENV, defEnvStr, conf));

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

public static void shouldHaveNext(String prefix, String s, Iterator<String> it) {
 if (!it.hasNext()) {
  throwParseException(sjoin(prefix, ID), s);
 }
}

代码示例来源：origin: harbby/sylph

private YarnJobContainer(YarnClient yarnClient, String jobInfo, Callable<Optional<ApplicationId>> runnable)
{
  this.runnable = runnable;
  this.yarnClient = yarnClient;
  if (jobInfo != null) {
    this.yarnAppId = Apps.toAppID(jobInfo);
    this.setStatus(RUNNING);
  }
}

代码示例来源：origin: org.apache.oozie/oozie-core

@Override
protected void addActionSpecificEnvVars(Map<String, String> env) {
  Apps.setEnvFromInputString(env, "PATH=.:$PATH", File.pathSeparator);
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-common

public static ApplicationId toAppID(String prefix, String s, Iterator<String> it) {
 if (!it.hasNext() || !it.next().equals(prefix)) {
  throwParseException(sjoin(prefix, ID), s);
 }
 shouldHaveNext(prefix, s, it);
 ApplicationId appId = ApplicationId.newInstance(Long.parseLong(it.next()),
   Integer.parseInt(it.next()));
 return appId;
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

@Test
 public void testSetEnvFromInputPropertyNull() {
  Configuration conf = new Configuration(false);
  Map<String, String> env = new HashMap<>();
  String propName = "mapreduce.map.env";
  String defaultPropName = "mapreduce.child.env";
  // Setup environment input properties
  conf.set(propName, "env1=env1_val,env2=env2_val,env3=env3_val");
  conf.set(propName + ".env4", "env4_val");
  conf.set(propName + ".env2", "new_env2_val");
  // Setup some default values - we shouldn't see these values
  conf.set(defaultPropName, "env1=def1_val,env2=def2_val,env3=def3_val");
  String defaultPropValue = conf.get(defaultPropName);
  // These should never be referenced.
  conf.set(defaultPropName + ".env4", "def4_val");
  conf.set(defaultPropName + ".env2", "new_def2_val");
  // Try with null inputs
  Apps.setEnvFromInputProperty(env, "bogus1", null, conf, File.pathSeparator);
  assertTrue(env.isEmpty());
 }
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-common

public static void shouldHaveNext(String prefix, String s, Iterator<String> it) {
 if (!it.hasNext()) {
  throwParseException(sjoin(prefix, ID), s);
 }
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-common

public static ApplicationId toAppID(String aid) {
 Iterator<String> it = _split(aid).iterator();
 return toAppID(APP, aid, it);
}

代码示例来源：origin: org.apache.hadoop/hadoop-mapreduce-client-common

public static void setEnvFromInputString(Map<String, String> env,
  String envString, Configuration conf) {
 String classPathSeparator =
   conf.getBoolean(MRConfig.MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM,
    MRConfig.DEFAULT_MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM)
     ? ApplicationConstants.CLASS_PATH_SEPARATOR : File.pathSeparator;
 Apps.setEnvFromInputString(env, envString, classPathSeparator);
}

代码示例来源：origin: apache/incubator-gobblin

Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.JAVA_HOME.key(),
   System.getenv(ApplicationConstants.Environment.JAVA_HOME.key()));
Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(),
  ApplicationConstants.Environment.PWD.$());
Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(),
  ApplicationConstants.Environment.PWD.$() + File.separator + "*");
if (classpaths != null) {
 for (String classpath : classpaths) {
  Apps.addToEnvironment(
    environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(), classpath.trim());

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

public static ApplicationId toAppID(String prefix, String s, Iterator<String> it) {
 if (!it.hasNext() || !it.next().equals(prefix)) {
  throwParseException(sjoin(prefix, ID), s);
 }
 shouldHaveNext(prefix, s, it);
 ApplicationId appId = ApplicationId.newInstance(Long.parseLong(it.next()),
   Integer.parseInt(it.next()));
 return appId;
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

@Test
public void testSetEnvFromInputPropertyCommas() {
 Configuration conf = new Configuration(false);
 Map<String, String> env = new HashMap<>();
 String propName = "mapreduce.reduce.env";
 conf.set(propName, "env1=env1_val,env2=env2_val,env3=env3_val");
 conf.set(propName + ".env2", "new2_val1,new2_val2,new2_val3");
 conf.set(propName + ".env4", "new4_valwith=equals");
 // Setup some default values - we shouldn't see these values
 String defaultPropName = "mapreduce.child.env";
 conf.set(defaultPropName, "env1=def1_val,env2=def2_val,env3=def3_val");
 String defaultPropValue = conf.get(defaultPropName);
 Apps.setEnvFromInputProperty(env, propName, defaultPropValue, conf,
   File.pathSeparator);
 // Check values from string
 assertEquals("env1_val", env.get("env1"));
 assertEquals("env3_val", env.get("env3"));
 // Check individual value
 assertEquals("new4_valwith=equals", env.get("env4"));
 // Check individual value that eclipses one in string
 assertEquals("new2_val1,new2_val2,new2_val3", env.get("env2"));
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-common

public static void shouldHaveNext(String prefix, String s, Iterator<String> it) {
 if (!it.hasNext()) {
  throwParseException(sjoin(prefix, ID), s);
 }
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-common

public static ApplicationId toAppID(String aid) {
 Iterator<String> it = _split(aid).iterator();
 return toAppID(APP, aid, it);
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-common

/**
 * This older version of this method is kept around for compatibility
 * because downstream frameworks like Spark and Tez have been using it.
 * Downstream frameworks are expected to move off of it.
 */
@Deprecated
public static void setEnvFromInputString(Map<String, String> env,
  String envString) {
 setEnvFromInputString(env, envString, File.pathSeparator);
}

内容来源于网络，如有侵权，请联系作者删除！

Apps

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com