org.apache.spark.sql.DataFrame.toJavaRDD()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(2.7k)|赞(0)|评价(0)|浏览(189)

本文整理了Java中org.apache.spark.sql.DataFrame.toJavaRDD()方法的一些代码示例,展示了DataFrame.toJavaRDD()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。DataFrame.toJavaRDD()方法的具体详情如下:
包路径:org.apache.spark.sql.DataFrame
类名称:DataFrame
方法名:toJavaRDD

DataFrame.toJavaRDD介绍

暂无

代码示例

代码示例来源:origin: databricks/learning-spark

public static void main(String[] args) throws Exception {
    if (args.length != 3) {
   throw new Exception("Usage LoadHive sparkMaster tbl");
    }
  String master = args[0];
  String tbl = args[1];

    JavaSparkContext sc = new JavaSparkContext(
   master, "loadhive", System.getenv("SPARK_HOME"), System.getenv("JARS"));
  SQLContext sqlCtx = new SQLContext(sc);
  DataFrame rdd = sqlCtx.sql("SELECT key, value FROM src");
  JavaRDD<Integer> squaredKeys = rdd.toJavaRDD().map(new SquareKey());
  List<Integer> result = squaredKeys.collect();
  for (Integer elem : result) {
   System.out.println(elem);
  }
  }
}

代码示例来源:origin: databricks/learning-spark

System.out.println(row.get(0));
JavaRDD<String> topTweetText = topTweets.toJavaRDD().map(new Function<Row, String>() {
  public String call(Row row) {
   return row.getString(0);

代码示例来源:origin: stackoverflow.com

// SQL can be run over RDDs that have been registered as tables.
DataFrame teenagers = sqlContext.sql("SELECT name FROM people WHERE age >= 13 AND age <= 19");

List<String> teenagerNames = teenagers.toJavaRDD().map(
    new Function<Row, String>() {
   public String call(Row row) {
    return "Name: " + row.getString(0);
   }
  }).collect();

代码示例来源:origin: stackoverflow.com

DataFrame greaterThan25 = resultingDataFrame.where(resultingDataFrame.col("AGE").geq(25));
DataFrame lessThanGreaterThan = greaterThan25.where(resultingDataFrame.col("AGE").leq(50));
JavaRDD<Row> resultFromQuery = lessThanGreaterThan.toJavaRDD();

代码示例来源:origin: stackoverflow.com

DataFrame predictions = cvModel.transform(testingFrame);
JavaRDD<Tuple2<Object, Object>> scoreAndLabels = predictions.toJavaRDD().map(
    new Function<Row, Tuple2<Object, Object>>() {
      @Override
      public Tuple2<Object, Object> call(Row r) {
        Double score = r.getDouble(4);
        Double label = r.getDouble(1);
        return new Tuple2<Object, Object>(score, label);
      }
    });

BinaryClassificationMetrics metrics
    = new BinaryClassificationMetrics(JavaRDD.toRDD(scoreAndLabels));

代码示例来源:origin: stackoverflow.com

DataFrame df;
SQLContext sqlContext;
Long start;
Long end;

JavaPairRDD<Row, Long> indexedRDD = df.toJavaRDD().zipWithIndex();
JavaRDD filteredRDD = indexedRDD.filter((Tuple2<Row,Long> v1) -> v1._2 >= start && v1._2 < end);
DataFrame filteredDataFrame = sqlContext.createDataFrame(filteredRDD, df.schema());

代码示例来源:origin: phuonglh/vn.vitk

this.numLabels = (int)row.getDouble(0) + 1;
JavaRDD<Row> rows = df.sqlContext().sql("SELECT label, features FROM dft").toJavaRDD();

相关文章