org.apache.hadoop.yarn.util.Apps.addToEnvironment()方法的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(9.8k)|赞(0)|评价(0)|浏览(79)

本文整理了Java中org.apache.hadoop.yarn.util.Apps.addToEnvironment()方法的一些代码示例，展示了Apps.addToEnvironment()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Apps.addToEnvironment()方法的具体详情如下：
包路径：org.apache.hadoop.yarn.util.Apps
类名称：Apps
方法名：addToEnvironment

Apps.addToEnvironment介绍

[英]This older version of this method is kept around for compatibility because downstream frameworks like Spark and Tez have been using it. Downstream frameworks are expected to move off of it.
[中]由于Spark和Tez等下游框架一直在使用该方法，因此保留该方法的旧版本是为了兼容性。下游框架有望脱离it。

代码示例

代码示例来源：origin: apache/ignite

/**
   * @param envs Environment variables.
   * @param conf Yarn configuration.
   */
  private static void setupAppMasterEnv(Map<String, String> envs, YarnConfiguration conf) {
    for (String c : conf.getStrings(YarnConfiguration.YARN_APPLICATION_CLASSPATH,
      YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH))
      Apps.addToEnvironment(envs, Environment.CLASSPATH.name(),
          c.trim(), File.pathSeparator);

    Apps.addToEnvironment(envs,
        Environment.CLASSPATH.name(),
        Environment.PWD.$() + File.separator + "*",
        File.pathSeparator);
  }
}

代码示例来源：origin: apache/incubator-gobblin

Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.JAVA_HOME.key(),
   System.getenv(ApplicationConstants.Environment.JAVA_HOME.key()));
Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(),
  ApplicationConstants.Environment.PWD.$());
Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(),
  ApplicationConstants.Environment.PWD.$() + File.separator + "*");
if (classpaths != null) {
 for (String classpath : classpaths) {
  Apps.addToEnvironment(
    environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(), classpath.trim());

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-common

/**
 * This older version of this method is kept around for compatibility
 * because downstream frameworks like Spark and Tez have been using it.
 * Downstream frameworks are expected to move off of it.
 */
@Deprecated
public static void addToEnvironment(
  Map<String, String> environment,
  String variable, String value) {
 addToEnvironment(environment, variable, value, File.pathSeparator);
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

/**
 * This older version of this method is kept around for compatibility
 * because downstream frameworks like Spark and Tez have been using it.
 * Downstream frameworks are expected to move off of it.
 */
@Deprecated
public static void addToEnvironment(
  Map<String, String> environment,
  String variable, String value) {
 addToEnvironment(environment, variable, value, File.pathSeparator);
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-common

/**
 * This older version of this method is kept around for compatibility
 * because downstream frameworks like Spark and Tez have been using it.
 * Downstream frameworks are expected to move off of it.
 */
@Deprecated
public static void addToEnvironment(
  Map<String, String> environment,
  String variable, String value) {
 addToEnvironment(environment, variable, value, File.pathSeparator);
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

/**
 * Set environment from string without doing any variable substitution.
 * Used internally to avoid double expansion.
 * @param env environment to set
 * @param envString comma-separated k=v pairs.
 * @param classPathSeparator Separator to use when appending to an existing
 *                           environment variable.
 */
private static void setEnvFromInputStringNoExpand(Map<String, String> env,
  String envString,  String classPathSeparator) {
 if (envString != null && envString.length() > 0) {
  Matcher varValMatcher = VARVAL_SPLITTER.matcher(envString);
  while (varValMatcher.find()) {
   String envVar = varValMatcher.group(1);
   String varString = varValMatcher.group(2);
   addToEnvironment(env, envVar, varString, classPathSeparator);
  }
 }
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-common

private static void setEnvFromString(Map<String, String> env,
  String envVar, String varString, String classPathSeparator) {
 Matcher m = VAR_SUBBER.matcher(varString);
 StringBuffer sb = new StringBuffer();
 while (m.find()) {
  String var = m.group(1);
  // do variable substitution of $var from passed in environment or from
  // system environment and default to empty string if undefined in both.
  String replace = env.get(var);
  if (replace == null) {
   replace = System.getenv(var);
  }
  if (replace == null) {
   replace = "";
  }
  m.appendReplacement(sb, Matcher.quoteReplacement(replace));
 }
 m.appendTail(sb);
 addToEnvironment(env, envVar, sb.toString(), classPathSeparator);
}

代码示例来源：origin: hortonworks/simple-yarn-app

private void setupAppMasterEnv(Map<String, String> appMasterEnv) {
 for (String c : conf.getStrings(
   YarnConfiguration.YARN_APPLICATION_CLASSPATH,
   YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH)) {
  Apps.addToEnvironment(appMasterEnv, Environment.CLASSPATH.name(),
    c.trim());
 }
 Apps.addToEnvironment(appMasterEnv,
   Environment.CLASSPATH.name(),
   Environment.PWD.$() + File.separator + "*");
}

代码示例来源：origin: org.apache.hadoop/hadoop-mapreduce-client-common

@Public
@Unstable
public static void addToEnvironment(Map<String, String> environment,
  String variable, String value, Configuration conf) {
 String classPathSeparator =
   conf.getBoolean(MRConfig.MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM,
    MRConfig.DEFAULT_MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM)
     ? ApplicationConstants.CLASS_PATH_SEPARATOR : File.pathSeparator;
 Apps.addToEnvironment(environment, variable, value, classPathSeparator);
}

代码示例来源：origin: ch.cern.hadoop/hadoop-mapreduce-client-common

@Public
@Unstable
public static void addToEnvironment(Map<String, String> environment,
  String variable, String value, Configuration conf) {
 String classPathSeparator =
   conf.getBoolean(MRConfig.MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM,
    MRConfig.DEFAULT_MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM)
     ? ApplicationConstants.CLASS_PATH_SEPARATOR : File.pathSeparator;
 Apps.addToEnvironment(environment, variable, value, classPathSeparator);
}

代码示例来源：origin: io.hops/hadoop-mapreduce-client-common

@Public
@Unstable
public static void addToEnvironment(Map<String, String> environment,
  String variable, String value, Configuration conf) {
 String classPathSeparator =
   conf.getBoolean(MRConfig.MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM,
    MRConfig.DEFAULT_MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM)
     ? ApplicationConstants.CLASS_PATH_SEPARATOR : File.pathSeparator;
 Apps.addToEnvironment(environment, variable, value, classPathSeparator);
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-mapreduce-client-common

@Public
@Unstable
public static void addToEnvironment(Map<String, String> environment,
  String variable, String value, Configuration conf) {
 String classPathSeparator =
   conf.getBoolean(MRConfig.MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM,
    MRConfig.DEFAULT_MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM)
     ? ApplicationConstants.CLASS_PATH_SEPARATOR : File.pathSeparator;
 Apps.addToEnvironment(environment, variable, value, classPathSeparator);
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-common

public static void setEnvFromInputString(Map<String, String> env,
  String envString,  String classPathSeparator) {
 if (envString != null && envString.length() > 0) {
  String childEnvs[] = envString.split(",");
  Pattern p = Pattern.compile(Shell.getEnvironmentVariableRegex());
  for (String cEnv : childEnvs) {
   String[] parts = cEnv.split("="); // split on '='
   Matcher m = p.matcher(parts[1]);
   StringBuffer sb = new StringBuffer();
   while (m.find()) {
    String var = m.group(1);
    // replace $env with the child's env constructed by tt's
    String replace = env.get(var);
    // if this key is not configured by the tt for the child .. get it
    // from the tt's env
    if (replace == null)
     replace = System.getenv(var);
    // the env key is note present anywhere .. simply set it
    if (replace == null)
     replace = "";
    m.appendReplacement(sb, Matcher.quoteReplacement(replace));
   }
   m.appendTail(sb);
   addToEnvironment(env, parts[0], sb.toString(), classPathSeparator);
  }
 }
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-common

public static void setEnvFromInputString(Map<String, String> env,
  String envString,  String classPathSeparator) {
 if (envString != null && envString.length() > 0) {
  String childEnvs[] = envString.split(",");
  Pattern p = Pattern.compile(Shell.getEnvironmentVariableRegex());
  for (String cEnv : childEnvs) {
   String[] parts = cEnv.split("="); // split on '='
   Matcher m = p.matcher(parts[1]);
   StringBuffer sb = new StringBuffer();
   while (m.find()) {
    String var = m.group(1);
    // replace $env with the child's env constructed by tt's
    String replace = env.get(var);
    // if this key is not configured by the tt for the child .. get it
    // from the tt's env
    if (replace == null)
     replace = System.getenv(var);
    // the env key is note present anywhere .. simply set it
    if (replace == null)
     replace = "";
    m.appendReplacement(sb, Matcher.quoteReplacement(replace));
   }
   m.appendTail(sb);
   addToEnvironment(env, parts[0], sb.toString(), classPathSeparator);
  }
 }
}

代码示例来源：origin: org.apache.gobblin/gobblin-yarn

Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.JAVA_HOME.key(),
   System.getenv(ApplicationConstants.Environment.JAVA_HOME.key()));
Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(),
  ApplicationConstants.Environment.PWD.$());
Apps.addToEnvironment(environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(),
  ApplicationConstants.Environment.PWD.$() + File.separator + "*");
if (classpaths != null) {
 for (String classpath : classpaths) {
  Apps.addToEnvironment(
    environmentVariableMap, ApplicationConstants.Environment.CLASSPATH.key(), classpath.trim());

代码示例来源：origin: yahoo/storm-yarn

Map<String, String> env = new HashMap<String, String>();
Apps.addToEnvironment(env, Environment.CLASSPATH.name(), "./conf",":");
Apps.addToEnvironment(env, Environment.CLASSPATH.name(), "./AppMaster.jar",":");
proc.waitFor();
reader.close();
Apps.addToEnvironment(env, Environment.CLASSPATH.name(), yarn_class_path);
Apps.addToEnvironment(env, Environment.CLASSPATH.name(), yarn_class_path,":");
String stormHomeInZip = Util.getStormHomeInZip(fs, zip, stormVersion.version());
Apps.addToEnvironment(env, Environment.CLASSPATH.name(), "./storm/" + stormHomeInZip + "/*",":");
Apps.addToEnvironment(env, Environment.CLASSPATH.name(), "./storm/" + stormHomeInZip + "/lib/*",":");

内容来源于网络，如有侵权，请联系作者删除！

Apps

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com