org.apache.spark.api.java.JavaPairRDD.sampleByKeyExact()方法的使用及代码示例

x33g5p2x  于2022-01-21 转载在 其他  
字(3.0k)|赞(0)|评价(0)|浏览(76)

本文整理了Java中org.apache.spark.api.java.JavaPairRDD.sampleByKeyExact()方法的一些代码示例,展示了JavaPairRDD.sampleByKeyExact()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。JavaPairRDD.sampleByKeyExact()方法的具体详情如下:
包路径:org.apache.spark.api.java.JavaPairRDD
类名称:JavaPairRDD
方法名:sampleByKeyExact

JavaPairRDD.sampleByKeyExact介绍

暂无

代码示例

代码示例来源:origin: org.apache.spark/spark-core

@Test
@SuppressWarnings("unchecked")
public void sampleByKeyExact() {
 JavaRDD<Integer> rdd1 = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8), 3);
 JavaPairRDD<Integer, Integer> rdd2 = rdd1.mapToPair(i -> new Tuple2<>(i % 2, 1));
 Map<Integer, Double> fractions = new HashMap<>();
 fractions.put(0, 0.5);
 fractions.put(1, 1.0);
 JavaPairRDD<Integer, Integer> wrExact = rdd2.sampleByKeyExact(true, fractions, 1L);
 Map<Integer, Long> wrExactCounts = wrExact.countByKey();
 assertEquals(2, wrExactCounts.size());
 assertTrue(wrExactCounts.get(0) == 2);
 assertTrue(wrExactCounts.get(1) == 4);
 JavaPairRDD<Integer, Integer> worExact = rdd2.sampleByKeyExact(false, fractions, 1L);
 Map<Integer, Long> worExactCounts = worExact.countByKey();
 assertEquals(2, worExactCounts.size());
 assertTrue(worExactCounts.get(0) == 2);
 assertTrue(worExactCounts.get(1) == 4);
}

代码示例来源:origin: org.apache.spark/spark-core_2.11

@Test
@SuppressWarnings("unchecked")
public void sampleByKeyExact() {
 JavaRDD<Integer> rdd1 = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8), 3);
 JavaPairRDD<Integer, Integer> rdd2 = rdd1.mapToPair(i -> new Tuple2<>(i % 2, 1));
 Map<Integer, Double> fractions = new HashMap<>();
 fractions.put(0, 0.5);
 fractions.put(1, 1.0);
 JavaPairRDD<Integer, Integer> wrExact = rdd2.sampleByKeyExact(true, fractions, 1L);
 Map<Integer, Long> wrExactCounts = wrExact.countByKey();
 assertEquals(2, wrExactCounts.size());
 assertTrue(wrExactCounts.get(0) == 2);
 assertTrue(wrExactCounts.get(1) == 4);
 JavaPairRDD<Integer, Integer> worExact = rdd2.sampleByKeyExact(false, fractions, 1L);
 Map<Integer, Long> worExactCounts = worExact.countByKey();
 assertEquals(2, worExactCounts.size());
 assertTrue(worExactCounts.get(0) == 2);
 assertTrue(worExactCounts.get(1) == 4);
}

代码示例来源:origin: org.apache.spark/spark-core_2.10

@Test
@SuppressWarnings("unchecked")
public void sampleByKeyExact() {
 JavaRDD<Integer> rdd1 = sc.parallelize(Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8), 3);
 JavaPairRDD<Integer, Integer> rdd2 = rdd1.mapToPair(i -> new Tuple2<>(i % 2, 1));
 Map<Integer, Double> fractions = new HashMap<>();
 fractions.put(0, 0.5);
 fractions.put(1, 1.0);
 JavaPairRDD<Integer, Integer> wrExact = rdd2.sampleByKeyExact(true, fractions, 1L);
 Map<Integer, Long> wrExactCounts = wrExact.countByKey();
 assertEquals(2, wrExactCounts.size());
 assertTrue(wrExactCounts.get(0) == 2);
 assertTrue(wrExactCounts.get(1) == 4);
 JavaPairRDD<Integer, Integer> worExact = rdd2.sampleByKeyExact(false, fractions, 1L);
 Map<Integer, Long> worExactCounts = worExact.countByKey();
 assertEquals(2, worExactCounts.size());
 assertTrue(worExactCounts.get(0) == 2);
 assertTrue(worExactCounts.get(1) == 4);
}

相关文章

微信公众号

最新文章

更多

JavaPairRDD类方法