本文整理了Java中org.apache.spark.api.java.JavaRDD.checkpoint()
方法的一些代码示例,展示了JavaRDD.checkpoint()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。JavaRDD.checkpoint()
方法的具体详情如下:
包路径:org.apache.spark.api.java.JavaRDD
类名称:JavaRDD
方法名:checkpoint
暂无
代码示例来源:origin: org.apache.spark/spark-core_2.11
@Test
public void checkpointAndComputation() {
JavaRDD<Integer> rdd = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5));
sc.setCheckpointDir(tempDir.getAbsolutePath());
assertFalse(rdd.isCheckpointed());
rdd.checkpoint();
rdd.count(); // Forces the DAG to cause a checkpoint
assertTrue(rdd.isCheckpointed());
assertEquals(Arrays.asList(1, 2, 3, 4, 5), rdd.collect());
}
代码示例来源:origin: org.apache.spark/spark-core_2.10
@Test
public void checkpointAndComputation() {
JavaRDD<Integer> rdd = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5));
sc.setCheckpointDir(tempDir.getAbsolutePath());
assertFalse(rdd.isCheckpointed());
rdd.checkpoint();
rdd.count(); // Forces the DAG to cause a checkpoint
assertTrue(rdd.isCheckpointed());
assertEquals(Arrays.asList(1, 2, 3, 4, 5), rdd.collect());
}
代码示例来源:origin: org.apache.spark/spark-core
@Test
public void checkpointAndComputation() {
JavaRDD<Integer> rdd = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5));
sc.setCheckpointDir(tempDir.getAbsolutePath());
assertFalse(rdd.isCheckpointed());
rdd.checkpoint();
rdd.count(); // Forces the DAG to cause a checkpoint
assertTrue(rdd.isCheckpointed());
assertEquals(Arrays.asList(1, 2, 3, 4, 5), rdd.collect());
}
代码示例来源:origin: org.apache.spark/spark-core_2.11
@Test
public void checkpointAndRestore() {
JavaRDD<Integer> rdd = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5));
sc.setCheckpointDir(tempDir.getAbsolutePath());
assertFalse(rdd.isCheckpointed());
rdd.checkpoint();
rdd.count(); // Forces the DAG to cause a checkpoint
assertTrue(rdd.isCheckpointed());
assertTrue(rdd.getCheckpointFile().isPresent());
JavaRDD<Integer> recovered = sc.checkpointFile(rdd.getCheckpointFile().get());
assertEquals(Arrays.asList(1, 2, 3, 4, 5), recovered.collect());
}
代码示例来源:origin: org.apache.spark/spark-core_2.10
@Test
public void checkpointAndRestore() {
JavaRDD<Integer> rdd = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5));
sc.setCheckpointDir(tempDir.getAbsolutePath());
assertFalse(rdd.isCheckpointed());
rdd.checkpoint();
rdd.count(); // Forces the DAG to cause a checkpoint
assertTrue(rdd.isCheckpointed());
assertTrue(rdd.getCheckpointFile().isPresent());
JavaRDD<Integer> recovered = sc.checkpointFile(rdd.getCheckpointFile().get());
assertEquals(Arrays.asList(1, 2, 3, 4, 5), recovered.collect());
}
代码示例来源:origin: org.apache.spark/spark-core
@Test
public void checkpointAndRestore() {
JavaRDD<Integer> rdd = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5));
sc.setCheckpointDir(tempDir.getAbsolutePath());
assertFalse(rdd.isCheckpointed());
rdd.checkpoint();
rdd.count(); // Forces the DAG to cause a checkpoint
assertTrue(rdd.isCheckpointed());
assertTrue(rdd.getCheckpointFile().isPresent());
JavaRDD<Integer> recovered = sc.checkpointFile(rdd.getCheckpointFile().get());
assertEquals(Arrays.asList(1, 2, 3, 4, 5), recovered.collect());
}
内容来源于网络,如有侵权,请联系作者删除!