org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.QueueCapacities.getAbsoluteMaximumCapacity()方法的使用及代码示例

x33g5p2x  于2022-01-28 转载在 其他  
字(16.9k)|赞(0)|评价(0)|浏览(228)

本文整理了Java中org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.QueueCapacities.getAbsoluteMaximumCapacity方法的一些代码示例，展示了QueueCapacities.getAbsoluteMaximumCapacity的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。QueueCapacities.getAbsoluteMaximumCapacity方法的具体详情如下：
包路径：org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.QueueCapacities
类名称：QueueCapacities
方法名：getAbsoluteMaximumCapacity

QueueCapacities.getAbsoluteMaximumCapacity介绍

暂无

代码示例

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

@Override
public float getAbsoluteMaximumCapacity() {
 return queueCapacities.getAbsoluteMaximumCapacity();
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

@Override
public float getAbsoluteMaximumCapacity() {
 return queueCapacities.getAbsoluteMaximumCapacity();
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

@Override
public float getAbsoluteMaximumCapacity() {
 return queueCapacities.getAbsoluteMaximumCapacity();
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

/**
 * Check sanity of capacities:
 * - capacity <= maxCapacity
 * - absCapacity <= absMaximumCapacity
 */
private static void capacitiesSanityCheck(String queueName,
  QueueCapacities queueCapacities) {
 for (String label : queueCapacities.getExistingNodeLabels()) {
  // The only thing we should care about is absolute capacity <=
  // absolute max capacity otherwise the absolute max capacity is
  // no longer an absolute maximum.
  float absCapacity = queueCapacities.getAbsoluteCapacity(label);
  float absMaxCapacity = queueCapacities.getAbsoluteMaximumCapacity(label);
  if (absCapacity > absMaxCapacity) {
   throw new IllegalArgumentException("Illegal queue capacity setting, "
     + "(abs-capacity=" + absCapacity + ") > (abs-maximum-capacity="
     + absMaxCapacity + "). When label=[" + label + "]");
  }
 }
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

synchronized void setupQueueConfigs(Resource clusterResource)
  throws IOException {
 super.setupQueueConfigs(clusterResource);
 StringBuilder aclsString = new StringBuilder();
 for (Map.Entry<AccessType, AccessControlList> e : acls.entrySet()) {
  aclsString.append(e.getKey() + ":" + e.getValue().getAclString());
 }
 StringBuilder labelStrBuilder = new StringBuilder(); 
 if (accessibleLabels != null) {
  for (String s : accessibleLabels) {
   labelStrBuilder.append(s);
   labelStrBuilder.append(",");
  }
 }
 LOG.info(queueName +
   ", capacity=" + this.queueCapacities.getCapacity() +
   ", asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +
   ", maxCapacity=" + this.queueCapacities.getMaximumCapacity() +
   ", asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() + 
   ", state=" + state +
   ", acls=" + aclsString + 
   ", labels=" + labelStrBuilder.toString() + "\n" +
   ", reservationsContinueLooking=" + reservationsContinueLooking);
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

synchronized void setupQueueConfigs(Resource clusterResource)
  throws IOException {
 super.setupQueueConfigs(clusterResource);
 StringBuilder aclsString = new StringBuilder();
 for (Map.Entry<AccessType, AccessControlList> e : acls.entrySet()) {
  aclsString.append(e.getKey() + ":" + e.getValue().getAclString());
 }
 StringBuilder labelStrBuilder = new StringBuilder(); 
 if (accessibleLabels != null) {
  for (String s : accessibleLabels) {
   labelStrBuilder.append(s);
   labelStrBuilder.append(",");
  }
 }
 LOG.info(queueName +
   ", capacity=" + this.queueCapacities.getCapacity() +
   ", asboluteCapacity=" + this.queueCapacities.getAbsoluteCapacity() +
   ", maxCapacity=" + this.queueCapacities.getMaximumCapacity() +
   ", asboluteMaxCapacity=" + this.queueCapacities.getAbsoluteMaximumCapacity() + 
   ", state=" + state +
   ", acls=" + aclsString + 
   ", labels=" + labelStrBuilder.toString() + "\n" +
   ", reservationsContinueLooking=" + reservationsContinueLooking);
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

private Resource getCurrentLimitResource(String nodeLabel,
  Resource clusterResource, ResourceLimits currentResourceLimits) {
 /*
  * Current limit resource: For labeled resource: limit = queue-max-resource
  * (TODO, this part need update when we support labeled-limit) For
  * non-labeled resource: limit = min(queue-max-resource,
  * limit-set-by-parent)
  */
 Resource queueMaxResource =
   Resources.multiplyAndNormalizeDown(resourceCalculator,
     labelManager.getResourceByLabel(nodeLabel, clusterResource),
     queueCapacities.getAbsoluteMaximumCapacity(nodeLabel), minimumAllocation);
 if (nodeLabel.equals(RMNodeLabelsManager.NO_LABEL)) {
  return Resources.min(resourceCalculator, clusterResource,
    queueMaxResource, currentResourceLimits.getLimit());
 }
 return queueMaxResource;
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

private Resource getCurrentLimitResource(String nodeLabel,
  Resource clusterResource, ResourceLimits currentResourceLimits) {
 /*
  * Current limit resource: For labeled resource: limit = queue-max-resource
  * (TODO, this part need update when we support labeled-limit) For
  * non-labeled resource: limit = min(queue-max-resource,
  * limit-set-by-parent)
  */
 Resource queueMaxResource =
   Resources.multiplyAndNormalizeDown(resourceCalculator,
     labelManager.getResourceByLabel(nodeLabel, clusterResource),
     queueCapacities.getAbsoluteMaximumCapacity(nodeLabel), minimumAllocation);
 if (nodeLabel.equals(RMNodeLabelsManager.NO_LABEL)) {
  return Resources.min(resourceCalculator, clusterResource,
    queueMaxResource, currentResourceLimits.getLimit());
 }
 return queueMaxResource;
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

/**
 * Check sanity of capacities:
 * - capacity <= maxCapacity
 * - absCapacity <= absMaximumCapacity
 */
private static void capacitiesSanityCheck(String queueName,
  QueueCapacities queueCapacities) {
 for (String label : queueCapacities.getExistingNodeLabels()) {
  float capacity = queueCapacities.getCapacity(label);
  float maximumCapacity = queueCapacities.getMaximumCapacity(label);
  if (capacity > maximumCapacity) {
   throw new IllegalArgumentException("Illegal queue capacity setting, "
     + "(capacity=" + capacity + ") > (maximum-capacity="
     + maximumCapacity + "). When label=[" + label + "]");
  }
    // Actually, this may not needed since we have verified capacity <=
  // maximumCapacity. And the way we compute absolute capacity (abs(x) =
  // cap(x) * cap(x.parent) * ...) is a monotone increasing function. But
  // just keep it here to make sure our compute abs capacity method works
  // correctly. 
  float absCapacity = queueCapacities.getAbsoluteCapacity(label);
  float absMaxCapacity = queueCapacities.getAbsoluteMaximumCapacity(label);
  if (absCapacity > absMaxCapacity) {
   throw new IllegalArgumentException("Illegal queue capacity setting, "
     + "(abs-capacity=" + absCapacity + ") > (abs-maximum-capacity="
     + absMaxCapacity + "). When label=[" + label + "]");
  }
 }
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

/**
 * Check sanity of capacities:
 * - capacity <= maxCapacity
 * - absCapacity <= absMaximumCapacity
 */
private static void capacitiesSanityCheck(String queueName,
  QueueCapacities queueCapacities) {
 for (String label : queueCapacities.getExistingNodeLabels()) {
  float capacity = queueCapacities.getCapacity(label);
  float maximumCapacity = queueCapacities.getMaximumCapacity(label);
  if (capacity > maximumCapacity) {
   throw new IllegalArgumentException("Illegal queue capacity setting, "
     + "(capacity=" + capacity + ") > (maximum-capacity="
     + maximumCapacity + "). When label=[" + label + "]");
  }
    // Actually, this may not needed since we have verified capacity <=
  // maximumCapacity. And the way we compute absolute capacity (abs(x) =
  // cap(x) * cap(x.parent) * ...) is a monotone increasing function. But
  // just keep it here to make sure our compute abs capacity method works
  // correctly. 
  float absCapacity = queueCapacities.getAbsoluteCapacity(label);
  float absMaxCapacity = queueCapacities.getAbsoluteMaximumCapacity(label);
  if (absCapacity > absMaxCapacity) {
   throw new IllegalArgumentException("Illegal queue capacity setting, "
     + "(abs-capacity=" + absCapacity + ") > (abs-maximum-capacity="
     + absMaxCapacity + "). When label=[" + label + "]");
  }
 }
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

private void updateCurrentResourceLimits(
  ResourceLimits currentResourceLimits, Resource clusterResource) {
 // TODO: need consider non-empty node labels when resource limits supports
 // node labels
 // Even if ParentQueue will set limits respect child's max queue capacity,
 // but when allocating reserved container, CapacityScheduler doesn't do
 // this. So need cap limits by queue's max capacity here.
 this.cachedResourceLimitsForHeadroom = new ResourceLimits(currentResourceLimits.getLimit());
 Resource queueMaxResource =
   Resources.multiplyAndNormalizeDown(resourceCalculator, labelManager
     .getResourceByLabel(RMNodeLabelsManager.NO_LABEL, clusterResource),
     queueCapacities
       .getAbsoluteMaximumCapacity(RMNodeLabelsManager.NO_LABEL),
     minimumAllocation);
 this.cachedResourceLimitsForHeadroom.setLimit(Resources.min(resourceCalculator,
   clusterResource, queueMaxResource, currentResourceLimits.getLimit()));
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

private static void updateAbsoluteCapacitiesByNodeLabels(
  QueueCapacities queueCapacities, QueueCapacities parentQueueCapacities) {
 for (String label : queueCapacities.getExistingNodeLabels()) {
  float capacity = queueCapacities.getCapacity(label);
  if (capacity > 0f) {
   queueCapacities.setAbsoluteCapacity(
     label,
     capacity
       * (parentQueueCapacities == null ? 1 : parentQueueCapacities
         .getAbsoluteCapacity(label)));
  }
  float maxCapacity = queueCapacities.getMaximumCapacity(label);
  if (maxCapacity > 0f) {
   queueCapacities.setAbsoluteMaximumCapacity(
     label,
     maxCapacity
       * (parentQueueCapacities == null ? 1 : parentQueueCapacities
         .getAbsoluteMaximumCapacity(label)));
  }
 }
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

private static void updateAbsoluteCapacitiesByNodeLabels(
  QueueCapacities queueCapacities, QueueCapacities parentQueueCapacities) {
 for (String label : queueCapacities.getExistingNodeLabels()) {
  float capacity = queueCapacities.getCapacity(label);
  if (capacity > 0f) {
   queueCapacities.setAbsoluteCapacity(
     label,
     capacity
       * (parentQueueCapacities == null ? 1 : parentQueueCapacities
         .getAbsoluteCapacity(label)));
  }
  float maxCapacity = queueCapacities.getMaximumCapacity(label);
  if (maxCapacity > 0f) {
   queueCapacities.setAbsoluteMaximumCapacity(
     label,
     maxCapacity
       * (parentQueueCapacities == null ? 1 : parentQueueCapacities
         .getAbsoluteMaximumCapacity(label)));
  }
 }
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private static void updateAbsoluteCapacitiesByNodeLabels(
  QueueCapacities queueCapacities, QueueCapacities parentQueueCapacities) {
 for (String label : queueCapacities.getExistingNodeLabels()) {
  float capacity = queueCapacities.getCapacity(label);
  if (capacity > 0f) {
   queueCapacities.setAbsoluteCapacity(
     label,
     capacity
       * (parentQueueCapacities == null ? 1 : parentQueueCapacities
         .getAbsoluteCapacity(label)));
  }
  float maxCapacity = queueCapacities.getMaximumCapacity(label);
  if (maxCapacity > 0f) {
   queueCapacities.setAbsoluteMaximumCapacity(
     label,
     maxCapacity
       * (parentQueueCapacities == null ? 1 : parentQueueCapacities
         .getAbsoluteMaximumCapacity(label)));
  }
 }
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

private void updateCurrentResourceLimits(
  ResourceLimits currentResourceLimits, Resource clusterResource) {
 // TODO: need consider non-empty node labels when resource limits supports
 // node labels
 // Even if ParentQueue will set limits respect child's max queue capacity,
 // but when allocating reserved container, CapacityScheduler doesn't do
 // this. So need cap limits by queue's max capacity here.
 this.cachedResourceLimitsForHeadroom = new ResourceLimits(currentResourceLimits.getLimit());
 Resource queueMaxResource =
   Resources.multiplyAndNormalizeDown(resourceCalculator, labelManager
     .getResourceByLabel(RMNodeLabelsManager.NO_LABEL, clusterResource),
     queueCapacities
       .getAbsoluteMaximumCapacity(RMNodeLabelsManager.NO_LABEL),
     minimumAllocation);
 this.cachedResourceLimitsForHeadroom.setLimit(Resources.min(resourceCalculator,
   clusterResource, queueMaxResource, currentResourceLimits.getLimit()));
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private void mergeCapacities(QueueCapacities capacities) {
 for ( String nodeLabel : capacities.getExistingNodeLabels()) {
  queueCapacities.setCapacity(nodeLabel,
    capacities.getCapacity(nodeLabel));
  queueCapacities.setAbsoluteCapacity(nodeLabel, capacities
    .getAbsoluteCapacity(nodeLabel));
  queueCapacities.setMaximumCapacity(nodeLabel, capacities
    .getMaximumCapacity(nodeLabel));
  queueCapacities.setAbsoluteMaximumCapacity(nodeLabel, capacities
    .getAbsoluteMaximumCapacity(nodeLabel));
  Resource resourceByLabel = labelManager.getResourceByLabel(nodeLabel,
    csContext.getClusterResource());
  getQueueResourceQuotas().setEffectiveMinResource(nodeLabel,
    Resources.multiply(resourceByLabel,
      queueCapacities.getAbsoluteCapacity(nodeLabel)));
  getQueueResourceQuotas().setEffectiveMaxResource(nodeLabel,
    Resources.multiply(resourceByLabel, queueCapacities
      .getAbsoluteMaximumCapacity(nodeLabel)));
 }
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

public void checkAbsCapacities(CSQueue queue, String partition,
  float guaranteed, float max, float used) {
 QueueCapacities qc = queue.getQueueCapacities();
 Assert.assertEquals(guaranteed, qc.getAbsoluteCapacity(partition), 1e-3);
 Assert.assertEquals(max, qc.getAbsoluteMaximumCapacity(partition), 1e-3);
 Assert.assertEquals(used, qc.getAbsoluteUsedCapacity(partition), 1e-3);
}

代码示例来源：origin: com.github.jiayuhan-it/hadoop-yarn-server-resourcemanager

CapacitySchedulerQueueInfo(final CSQueue q, final String nodeLabel) {
 QueueCapacities qCapacities = q.getQueueCapacities();
 ResourceUsage queueResourceUsage = q.getQueueResourceUsage();
 queuePath = q.getQueuePath();
 capacity = qCapacities.getCapacity(nodeLabel) * 100;
 usedCapacity = q.getUsedCapacity(nodeLabel) * 100;
 maxCapacity = qCapacities.getMaximumCapacity(nodeLabel);
 if (maxCapacity < EPSILON || maxCapacity > 1f)
  maxCapacity = 1f;
 maxCapacity *= 100;
 absoluteCapacity =
   cap(qCapacities.getAbsoluteCapacity(nodeLabel), 0f, 1f) * 100;
 absoluteMaxCapacity =
   cap(qCapacities.getAbsoluteMaximumCapacity(nodeLabel), 0f, 1f) * 100;
 absoluteUsedCapacity = q.getAbsoluteUsedCapacity(nodeLabel) * 100;
 numApplications = q.getNumApplications();
 queueName = q.getQueueName();
 state = q.getState();
 resourcesUsed = new ResourceInfo(queueResourceUsage.getUsed(nodeLabel));
 if (q instanceof PlanQueue && !((PlanQueue) q).showReservationsAsQueues()) {
  hideReservationQueues = true;
 }
 // add labels
 Set<String> labelSet = q.getAccessibleNodeLabels();
 if (labelSet != null) {
  nodeLabels.addAll(labelSet);
  Collections.sort(nodeLabels);
 }
}

代码示例来源：origin: ch.cern.hadoop/hadoop-yarn-server-resourcemanager

CapacitySchedulerQueueInfo(final CSQueue q, final String nodeLabel) {
 QueueCapacities qCapacities = q.getQueueCapacities();
 ResourceUsage queueResourceUsage = q.getQueueResourceUsage();
 queuePath = q.getQueuePath();
 capacity = qCapacities.getCapacity(nodeLabel) * 100;
 usedCapacity = q.getUsedCapacity(nodeLabel) * 100;
 maxCapacity = qCapacities.getMaximumCapacity(nodeLabel);
 if (maxCapacity < EPSILON || maxCapacity > 1f)
  maxCapacity = 1f;
 maxCapacity *= 100;
 absoluteCapacity =
   cap(qCapacities.getAbsoluteCapacity(nodeLabel), 0f, 1f) * 100;
 absoluteMaxCapacity =
   cap(qCapacities.getAbsoluteMaximumCapacity(nodeLabel), 0f, 1f) * 100;
 absoluteUsedCapacity = q.getAbsoluteUsedCapacity(nodeLabel) * 100;
 numApplications = q.getNumApplications();
 queueName = q.getQueueName();
 state = q.getState();
 resourcesUsed = new ResourceInfo(queueResourceUsage.getUsed(nodeLabel));
 if (q instanceof PlanQueue && !((PlanQueue) q).showReservationsAsQueues()) {
  hideReservationQueues = true;
 }
 // add labels
 Set<String> labelSet = q.getAccessibleNodeLabels();
 if (labelSet != null) {
  nodeLabels.addAll(labelSet);
  Collections.sort(nodeLabels);
 }
}

代码示例来源：origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

public Map<String, QueueConfigurations> getQueueConfigurations() {
 Map<String, QueueConfigurations> queueConfigurations = new HashMap<>();
 Set<String> nodeLabels = getNodeLabelsForQueue();
 for (String nodeLabel : nodeLabels) {
  QueueConfigurations queueConfiguration =
    recordFactory.newRecordInstance(QueueConfigurations.class);
  float capacity = queueCapacities.getCapacity(nodeLabel);
  float absoluteCapacity = queueCapacities.getAbsoluteCapacity(nodeLabel);
  float maxCapacity = queueCapacities.getMaximumCapacity(nodeLabel);
  float absMaxCapacity =
    queueCapacities.getAbsoluteMaximumCapacity(nodeLabel);
  float maxAMPercentage =
    queueCapacities.getMaxAMResourcePercentage(nodeLabel);
  queueConfiguration.setCapacity(capacity);
  queueConfiguration.setAbsoluteCapacity(absoluteCapacity);
  queueConfiguration.setMaxCapacity(maxCapacity);
  queueConfiguration.setAbsoluteMaxCapacity(absMaxCapacity);
  queueConfiguration.setMaxAMPercentage(maxAMPercentage);
  queueConfiguration.setConfiguredMinCapacity(
    queueResourceQuotas.getConfiguredMinResource(nodeLabel));
  queueConfiguration.setConfiguredMaxCapacity(
    queueResourceQuotas.getConfiguredMaxResource(nodeLabel));
  queueConfiguration.setEffectiveMinCapacity(
    queueResourceQuotas.getEffectiveMinResource(nodeLabel));
  queueConfiguration.setEffectiveMaxCapacity(
    queueResourceQuotas.getEffectiveMaxResource(nodeLabel));
  queueConfigurations.put(nodeLabel, queueConfiguration);
 }
 return queueConfigurations;
}

内容来源于网络，如有侵权，请联系作者删除！

QueueCapacities

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com