本文整理了Java中org.apache.spark.api.java.JavaPairRDD.countApproxDistinctByKey()
方法的一些代码示例,展示了JavaPairRDD.countApproxDistinctByKey()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。JavaPairRDD.countApproxDistinctByKey()
方法的具体详情如下:
包路径:org.apache.spark.api.java.JavaPairRDD
类名称:JavaPairRDD
方法名:countApproxDistinctByKey
暂无
代码示例来源:origin: org.apache.spark/spark-core_2.10
@Test
public void countApproxDistinctByKey() {
List<Tuple2<Integer, Integer>> arrayData = new ArrayList<>();
for (int i = 10; i < 100; i++) {
for (int j = 0; j < i; j++) {
arrayData.add(new Tuple2<>(i, j));
}
}
double relativeSD = 0.001;
JavaPairRDD<Integer, Integer> pairRdd = sc.parallelizePairs(arrayData);
List<Tuple2<Integer, Long>> res = pairRdd.countApproxDistinctByKey(relativeSD, 8).collect();
for (Tuple2<Integer, Long> resItem : res) {
double count = resItem._1();
long resCount = resItem._2();
double error = Math.abs((resCount - count) / count);
assertTrue(error < 0.1);
}
}
代码示例来源:origin: org.apache.spark/spark-core_2.11
@Test
public void countApproxDistinctByKey() {
List<Tuple2<Integer, Integer>> arrayData = new ArrayList<>();
for (int i = 10; i < 100; i++) {
for (int j = 0; j < i; j++) {
arrayData.add(new Tuple2<>(i, j));
}
}
double relativeSD = 0.001;
JavaPairRDD<Integer, Integer> pairRdd = sc.parallelizePairs(arrayData);
List<Tuple2<Integer, Long>> res = pairRdd.countApproxDistinctByKey(relativeSD, 8).collect();
for (Tuple2<Integer, Long> resItem : res) {
double count = resItem._1();
long resCount = resItem._2();
double error = Math.abs((resCount - count) / count);
assertTrue(error < 0.1);
}
}
代码示例来源:origin: org.apache.spark/spark-core
@Test
public void countApproxDistinctByKey() {
List<Tuple2<Integer, Integer>> arrayData = new ArrayList<>();
for (int i = 10; i < 100; i++) {
for (int j = 0; j < i; j++) {
arrayData.add(new Tuple2<>(i, j));
}
}
double relativeSD = 0.001;
JavaPairRDD<Integer, Integer> pairRdd = sc.parallelizePairs(arrayData);
List<Tuple2<Integer, Long>> res = pairRdd.countApproxDistinctByKey(relativeSD, 8).collect();
for (Tuple2<Integer, Long> resItem : res) {
double count = resItem._1();
long resCount = resItem._2();
double error = Math.abs((resCount - count) / count);
assertTrue(error < 0.1);
}
}
内容来源于网络,如有侵权,请联系作者删除!