org.apache.hadoop.yarn.api.records.Resource.getMemorySize()方法的使用及代码示例

x33g5p2x  于2022-01-28 转载在 其他  
字(6.5k)|赞(0)|评价(0)|浏览(203)

本文整理了Java中org.apache.hadoop.yarn.api.records.Resource.getMemorySize方法的一些代码示例,展示了Resource.getMemorySize的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Resource.getMemorySize方法的具体详情如下:
包路径:org.apache.hadoop.yarn.api.records.Resource
类名称:Resource
方法名:getMemorySize

Resource.getMemorySize介绍

[英]Get memory of the resource. Note - while memory has never had a unit specified, all YARN configurations have specified memory in MB. The assumption has been that the daemons and applications are always using the same units. With the introduction of the ResourceInformation class we have support for units - so this function will continue to return memory but in the units of MB
[中]获取资源的“内存”。注意——虽然内存从未指定过单位,但所有纱线配置都以MB为单位指定了内存。假设守护进程和应用程序总是使用相同的单元。随着ResourceInformation类的引入,我们支持单位,所以这个函数将继续以MB为单位返回内存

代码示例

代码示例来源:origin: apache/hive

memoryPerInstance = si.getResource().getMemorySize() * 1024L * 1024L;

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

public boolean isInvalidDivisor(Resource r) {
 if (r.getMemorySize() == 0.0f) {
  return true;
 }
 return false;
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Override
 public boolean isAnyMajorResourceAboveZero(Resource resource) {
  return resource.getMemorySize() > 0;
 }
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Override
public Resource roundDown(Resource r, Resource stepFactor) {
 return Resources.createResource(
   roundDown(r.getMemorySize(), stepFactor.getMemorySize()));
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Override
public Resource multiplyAndNormalizeUp(Resource r, double[] by,
  Resource stepFactor) {
 return Resources.createResource(
   roundUp((long) (r.getMemorySize() * by[0] + 0.5),
     stepFactor.getMemorySize()));
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Override
public Resource normalizeDown(Resource r, Resource stepFactor) {
 return Resources.createResource(
   roundDown((r.getMemorySize()), stepFactor.getMemorySize()));
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private float computeQueueAbsoluteUsedCapacity(CSQueue queue, 
  int expectedMemory, Resource clusterResource) {
 return (
   ((float)expectedMemory / (float)clusterResource.getMemorySize())
  );
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Override
public Resource divideAndCeil(Resource numerator, float denominator) {
 return Resources.createResource(
   divideAndCeil(numerator.getMemorySize(), denominator));
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

/**
 * Get the fair share assigned to the appAttemptId.
 * @param appAttemptId
 * @return The fair share assigned to the appAttemptId,
 * <code>FairSchedulerInfo#INVALID_FAIR_SHARE</code> if the scheduler does
 * not know about this application attempt.
 */
public long getAppFairShare(ApplicationAttemptId appAttemptId) {
 FSAppAttempt fsAppAttempt = scheduler.getSchedulerApp(appAttemptId);
 return fsAppAttempt == null ?
   INVALID_FAIR_SHARE :  fsAppAttempt.getFairShare().getMemorySize();
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-nodemanager

public void addResource(Resource res) {
 availableMB = availableMB + res.getMemorySize();
 availableGB.set((int)Math.floor(availableMB/1024d));
 availableVCores.incr(res.getVirtualCores());
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

public static Resource newAvailResource(Resource total, Resource used) {
 Resource rs = recordFactory.newRecordInstance(Resource.class);
 rs.setMemorySize(total.getMemorySize() - used.getMemorySize());
 return rs;
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private float computeQueueUsedCapacity(CSQueue queue,
  int expectedMemory, Resource clusterResource) {
 return (expectedMemory / 
   (clusterResource.getMemorySize() * queue.getAbsoluteCapacity()));
}

代码示例来源:origin: org.apache.hadoop/hadoop-mapreduce-client-app

@Test
public void testReducerMemoryRequestDefaultMemory() {
 EventHandler eventHandler = mock(EventHandler.class);
 Clock clock = SystemClock.getInstance();
 TaskAttemptImpl taImpl =
   createReduceTaskAttemptImplForTest(eventHandler, clock, new JobConf());
 long memorySize =
   getResourceInfoFromContainerRequest(taImpl, eventHandler).
   getMemorySize();
 assertEquals(MRJobConfig.DEFAULT_REDUCE_MEMORY_MB, memorySize);
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

public void reserveResource(String user, Resource res) {
 reservedContainers.incr();
 reservedMB.incr(res.getMemorySize());
 reservedVCores.incr(res.getVirtualCores());
 QueueMetrics userMetrics = getUserMetrics(user);
 if (userMetrics != null) {
  userMetrics.reserveResource(user, res);
 }
 if (parent != null) {
  parent.reserveResource(user, res);
 }
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private void checkNodeLabelInfo(List<RMNodeLabel> infos, String labelName, int activeNMs, int memory) {
 for (RMNodeLabel info : infos) {
  if (info.getLabelName().equals(labelName)) {
   Assert.assertEquals(activeNMs, info.getNumActiveNMs());
   Assert.assertEquals(memory, info.getResource().getMemorySize());
   return;
  }
 }
 Assert.fail("Failed to find info has label=" + labelName);
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Test
public void testCreateSimpleResourceWithSameLongValue() {
 Resource res = Resources.createResourceWithSameValue(11L);
 assertEquals(11L, res.getMemorySize());
 assertEquals(11, res.getVirtualCores());
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-server-resourcemanager

private void checkQueueResourceUsage(String partition, String queueName, MockRM rm, int expectedMemUsage) {
 CapacityScheduler cs = (CapacityScheduler) rm.getResourceScheduler();
 CSQueue queue = cs.getQueue(queueName);
 Assert.assertEquals(expectedMemUsage, queue.getQueueResourceUsage()
   .getUsed(partition).getMemorySize());
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Test
@SuppressWarnings("deprecation")
public void testGetMemory() {
 Resource res = new ResourcePBImpl();
 long memorySize = Integer.MAX_VALUE + 1L;
 res.setMemorySize(memorySize);
 assertEquals("No need to cast if both are long", memorySize,
   res.getMemorySize());
 assertEquals("Cast to Integer.MAX_VALUE if the long is greater than "
     + "Integer.MAX_VALUE", Integer.MAX_VALUE, res.getMemory());
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Test
public void testCreateResourceWithSameLongValue() throws Exception {
 unsetExtraResourceType();
 setupExtraResourceType();
 Resource res = Resources.createResourceWithSameValue(11L);
 assertEquals(11L, res.getMemorySize());
 assertEquals(11, res.getVirtualCores());
 assertEquals(11L, res.getResourceInformation(EXTRA_RESOURCE_TYPE).getValue());
}

代码示例来源:origin: org.apache.hadoop/hadoop-yarn-common

@Test
public void testCreateResourceWithSameIntValue() throws Exception {
 unsetExtraResourceType();
 setupExtraResourceType();
 Resource res = Resources.createResourceWithSameValue(11);
 assertEquals(11, res.getMemorySize());
 assertEquals(11, res.getVirtualCores());
 assertEquals(11, res.getResourceInformation(EXTRA_RESOURCE_TYPE).getValue());
}

相关文章