org.apache.spark.executor.TaskMetrics.inputMetrics()方法的使用及代码示例

x33g5p2x  于2022-01-30 转载在 其他  
字(3.5k)|赞(0)|评价(0)|浏览(95)

本文整理了Java中org.apache.spark.executor.TaskMetrics.inputMetrics()方法的一些代码示例,展示了TaskMetrics.inputMetrics()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。TaskMetrics.inputMetrics()方法的具体详情如下:
包路径:org.apache.spark.executor.TaskMetrics
类名称:TaskMetrics
方法名:inputMetrics

TaskMetrics.inputMetrics介绍

暂无

代码示例

代码示例来源:origin: apache/hive

public InputMetrics(TaskMetrics metrics) {
 this(metrics.inputMetrics().bytesRead(),
  metrics.inputMetrics().recordsRead());
}

代码示例来源:origin: apache/hive

private static InputMetrics optionalInputMetric(TaskMetrics metrics) {
 return (metrics.inputMetrics() != null) ? new InputMetrics(metrics) : null;
}

代码示例来源:origin: org.spark-project.hive/spark-client

public InputMetrics(TaskMetrics metrics) {
 this(DataReadMethod.valueOf(metrics.inputMetrics().get().readMethod().toString()),
  metrics.inputMetrics().get().bytesRead());
}

代码示例来源:origin: com.github.hyukjinkwon/spark-client

public InputMetrics(TaskMetrics metrics) {
 this(DataReadMethod.valueOf(metrics.inputMetrics().get().readMethod().toString()),
  metrics.inputMetrics().get().bytesRead());
}

代码示例来源:origin: com.github.hyukjinkwon/spark-client

private static InputMetrics optionalInputMetric(TaskMetrics metrics) {
 return metrics.inputMetrics().isDefined() ? new InputMetrics(metrics) : null;
}

代码示例来源:origin: org.spark-project.hive/spark-client

private static InputMetrics optionalInputMetric(TaskMetrics metrics) {
 return metrics.inputMetrics().isDefined() ? new InputMetrics(metrics) : null;
}

代码示例来源:origin: com.facebook.presto.hive/hive-apache

memoryBytesSpilled += taskMetrics.memoryBytesSpilled();
diskBytesSpilled += taskMetrics.diskBytesSpilled();
if (!taskMetrics.inputMetrics().isEmpty()) {
 inputMetricExist = true;
 bytesRead += taskMetrics.inputMetrics().get().bytesRead();

代码示例来源:origin: org.apache.pig/pig

memoryBytesSpilled += taskMetrics.memoryBytesSpilled();
diskBytesSpilled += taskMetrics.diskBytesSpilled();
if (!taskMetrics.inputMetrics().isEmpty()) {
  inputMetricExist = true;
  bytesRead += taskMetrics.inputMetrics().get().bytesRead();

代码示例来源:origin: com.criteo.java/garmadon-frameworks-spark

tryToSet(() -> taskEventBuilder.setShuffleWriteShuffleTime(taskEnd.taskMetrics().shuffleWriteMetrics().shuffleWriteTime()));
tryToSet(() -> taskEventBuilder.setShuffleWriteShuffleBytes(taskEnd.taskMetrics().shuffleWriteMetrics().shuffleBytesWritten()));
tryToSet(() -> taskEventBuilder.setInputRecords(taskEnd.taskMetrics().inputMetrics().recordsRead()));
tryToSet(() -> taskEventBuilder.setInputBytes(taskEnd.taskMetrics().inputMetrics().bytesRead()));
tryToSet(() -> taskEventBuilder.setOutputRecords(taskEnd.taskMetrics().outputMetrics().recordsWritten()));
tryToSet(() -> taskEventBuilder.setOutputBytes(taskEnd.taskMetrics().outputMetrics().bytesWritten()));

代码示例来源:origin: com.criteo.java/garmadon-frameworks-spark

tryToSet(() -> stageEventBuilder.setShuffleWriteShuffleTime(stageCompleted.stageInfo().taskMetrics().shuffleWriteMetrics().shuffleWriteTime()));
tryToSet(() -> stageEventBuilder.setShuffleWriteShuffleBytes(stageCompleted.stageInfo().taskMetrics().shuffleWriteMetrics().shuffleBytesWritten()));
tryToSet(() -> stageEventBuilder.setInputRecords(stageCompleted.stageInfo().taskMetrics().inputMetrics().recordsRead()));
tryToSet(() -> stageEventBuilder.setInputBytes(stageCompleted.stageInfo().taskMetrics().inputMetrics().bytesRead()));
tryToSet(() -> stageEventBuilder.setOutputRecords(stageCompleted.stageInfo().taskMetrics().outputMetrics().recordsWritten()));
tryToSet(() -> stageEventBuilder.setOutputBytes(stageCompleted.stageInfo().taskMetrics().outputMetrics().bytesWritten()));

相关文章