org.apache.spark.api.java.JavaPairRDD.countApproxDistinctByKey()方法的使用及代码示例

x33g5p2x  于2022-01-21 转载在 其他  
字(2.2k)|赞(0)|评价(0)|浏览(78)

本文整理了Java中org.apache.spark.api.java.JavaPairRDD.countApproxDistinctByKey()方法的一些代码示例,展示了JavaPairRDD.countApproxDistinctByKey()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。JavaPairRDD.countApproxDistinctByKey()方法的具体详情如下:
包路径:org.apache.spark.api.java.JavaPairRDD
类名称:JavaPairRDD
方法名:countApproxDistinctByKey

JavaPairRDD.countApproxDistinctByKey介绍

暂无

代码示例

代码示例来源:origin: org.apache.spark/spark-core_2.10

@Test
public void countApproxDistinctByKey() {
 List<Tuple2<Integer, Integer>> arrayData = new ArrayList<>();
 for (int i = 10; i < 100; i++) {
  for (int j = 0; j < i; j++) {
   arrayData.add(new Tuple2<>(i, j));
  }
 }
 double relativeSD = 0.001;
 JavaPairRDD<Integer, Integer> pairRdd = sc.parallelizePairs(arrayData);
 List<Tuple2<Integer, Long>> res =  pairRdd.countApproxDistinctByKey(relativeSD, 8).collect();
 for (Tuple2<Integer, Long> resItem : res) {
  double count = resItem._1();
  long resCount = resItem._2();
  double error = Math.abs((resCount - count) / count);
  assertTrue(error < 0.1);
 }
}

代码示例来源:origin: org.apache.spark/spark-core_2.11

@Test
public void countApproxDistinctByKey() {
 List<Tuple2<Integer, Integer>> arrayData = new ArrayList<>();
 for (int i = 10; i < 100; i++) {
  for (int j = 0; j < i; j++) {
   arrayData.add(new Tuple2<>(i, j));
  }
 }
 double relativeSD = 0.001;
 JavaPairRDD<Integer, Integer> pairRdd = sc.parallelizePairs(arrayData);
 List<Tuple2<Integer, Long>> res =  pairRdd.countApproxDistinctByKey(relativeSD, 8).collect();
 for (Tuple2<Integer, Long> resItem : res) {
  double count = resItem._1();
  long resCount = resItem._2();
  double error = Math.abs((resCount - count) / count);
  assertTrue(error < 0.1);
 }
}

代码示例来源:origin: org.apache.spark/spark-core

@Test
public void countApproxDistinctByKey() {
 List<Tuple2<Integer, Integer>> arrayData = new ArrayList<>();
 for (int i = 10; i < 100; i++) {
  for (int j = 0; j < i; j++) {
   arrayData.add(new Tuple2<>(i, j));
  }
 }
 double relativeSD = 0.001;
 JavaPairRDD<Integer, Integer> pairRdd = sc.parallelizePairs(arrayData);
 List<Tuple2<Integer, Long>> res =  pairRdd.countApproxDistinctByKey(relativeSD, 8).collect();
 for (Tuple2<Integer, Long> resItem : res) {
  double count = resItem._1();
  long resCount = resItem._2();
  double error = Math.abs((resCount - count) / count);
  assertTrue(error < 0.1);
 }
}

相关文章

微信公众号

最新文章

更多

JavaPairRDD类方法