本文整理了Java中org.apache.spark.executor.TaskMetrics.inputMetrics()
方法的一些代码示例,展示了TaskMetrics.inputMetrics()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。TaskMetrics.inputMetrics()
方法的具体详情如下:
包路径:org.apache.spark.executor.TaskMetrics
类名称:TaskMetrics
方法名:inputMetrics
暂无
代码示例来源:origin: apache/hive
public InputMetrics(TaskMetrics metrics) {
this(metrics.inputMetrics().bytesRead(),
metrics.inputMetrics().recordsRead());
}
代码示例来源:origin: apache/hive
private static InputMetrics optionalInputMetric(TaskMetrics metrics) {
return (metrics.inputMetrics() != null) ? new InputMetrics(metrics) : null;
}
代码示例来源:origin: org.spark-project.hive/spark-client
public InputMetrics(TaskMetrics metrics) {
this(DataReadMethod.valueOf(metrics.inputMetrics().get().readMethod().toString()),
metrics.inputMetrics().get().bytesRead());
}
代码示例来源:origin: com.github.hyukjinkwon/spark-client
public InputMetrics(TaskMetrics metrics) {
this(DataReadMethod.valueOf(metrics.inputMetrics().get().readMethod().toString()),
metrics.inputMetrics().get().bytesRead());
}
代码示例来源:origin: com.github.hyukjinkwon/spark-client
private static InputMetrics optionalInputMetric(TaskMetrics metrics) {
return metrics.inputMetrics().isDefined() ? new InputMetrics(metrics) : null;
}
代码示例来源:origin: org.spark-project.hive/spark-client
private static InputMetrics optionalInputMetric(TaskMetrics metrics) {
return metrics.inputMetrics().isDefined() ? new InputMetrics(metrics) : null;
}
代码示例来源:origin: com.facebook.presto.hive/hive-apache
memoryBytesSpilled += taskMetrics.memoryBytesSpilled();
diskBytesSpilled += taskMetrics.diskBytesSpilled();
if (!taskMetrics.inputMetrics().isEmpty()) {
inputMetricExist = true;
bytesRead += taskMetrics.inputMetrics().get().bytesRead();
代码示例来源:origin: org.apache.pig/pig
memoryBytesSpilled += taskMetrics.memoryBytesSpilled();
diskBytesSpilled += taskMetrics.diskBytesSpilled();
if (!taskMetrics.inputMetrics().isEmpty()) {
inputMetricExist = true;
bytesRead += taskMetrics.inputMetrics().get().bytesRead();
代码示例来源:origin: com.criteo.java/garmadon-frameworks-spark
tryToSet(() -> taskEventBuilder.setShuffleWriteShuffleTime(taskEnd.taskMetrics().shuffleWriteMetrics().shuffleWriteTime()));
tryToSet(() -> taskEventBuilder.setShuffleWriteShuffleBytes(taskEnd.taskMetrics().shuffleWriteMetrics().shuffleBytesWritten()));
tryToSet(() -> taskEventBuilder.setInputRecords(taskEnd.taskMetrics().inputMetrics().recordsRead()));
tryToSet(() -> taskEventBuilder.setInputBytes(taskEnd.taskMetrics().inputMetrics().bytesRead()));
tryToSet(() -> taskEventBuilder.setOutputRecords(taskEnd.taskMetrics().outputMetrics().recordsWritten()));
tryToSet(() -> taskEventBuilder.setOutputBytes(taskEnd.taskMetrics().outputMetrics().bytesWritten()));
代码示例来源:origin: com.criteo.java/garmadon-frameworks-spark
tryToSet(() -> stageEventBuilder.setShuffleWriteShuffleTime(stageCompleted.stageInfo().taskMetrics().shuffleWriteMetrics().shuffleWriteTime()));
tryToSet(() -> stageEventBuilder.setShuffleWriteShuffleBytes(stageCompleted.stageInfo().taskMetrics().shuffleWriteMetrics().shuffleBytesWritten()));
tryToSet(() -> stageEventBuilder.setInputRecords(stageCompleted.stageInfo().taskMetrics().inputMetrics().recordsRead()));
tryToSet(() -> stageEventBuilder.setInputBytes(stageCompleted.stageInfo().taskMetrics().inputMetrics().bytesRead()));
tryToSet(() -> stageEventBuilder.setOutputRecords(stageCompleted.stageInfo().taskMetrics().outputMetrics().recordsWritten()));
tryToSet(() -> stageEventBuilder.setOutputBytes(stageCompleted.stageInfo().taskMetrics().outputMetrics().bytesWritten()));
内容来源于网络,如有侵权,请联系作者删除!