本文整理了Java中org.apache.spark.sql.functions.expr()
方法的一些代码示例,展示了functions.expr()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。functions.expr()
方法的具体详情如下:
包路径:org.apache.spark.sql.functions
类名称:functions
方法名:expr
暂无
代码示例来源:origin: org.apache.spark/spark-sql
@Test
public void testSelect() {
List<Integer> data = Arrays.asList(2, 6);
Dataset<Integer> ds = spark.createDataset(data, Encoders.INT());
Dataset<Tuple2<Integer, String>> selected = ds.select(
expr("value + 1"),
col("value").cast("string")).as(Encoders.tuple(Encoders.INT(), Encoders.STRING()));
Assert.assertEquals(
Arrays.asList(tuple2(3, "2"), tuple2(7, "6")),
selected.collectAsList());
}
代码示例来源:origin: org.apache.spark/spark-sql_2.11
@Test
public void testSelect() {
List<Integer> data = Arrays.asList(2, 6);
Dataset<Integer> ds = spark.createDataset(data, Encoders.INT());
Dataset<Tuple2<Integer, String>> selected = ds.select(
expr("value + 1"),
col("value").cast("string")).as(Encoders.tuple(Encoders.INT(), Encoders.STRING()));
Assert.assertEquals(
Arrays.asList(tuple2(3, "2"), tuple2(7, "6")),
selected.collectAsList());
}
代码示例来源:origin: org.apache.spark/spark-sql_2.10
@Test
public void testSelect() {
List<Integer> data = Arrays.asList(2, 6);
Dataset<Integer> ds = spark.createDataset(data, Encoders.INT());
Dataset<Tuple2<Integer, String>> selected = ds.select(
expr("value + 1"),
col("value").cast("string")).as(Encoders.tuple(Encoders.INT(), Encoders.STRING()));
Assert.assertEquals(
Arrays.asList(tuple2(3, "2"), tuple2(7, "6")),
selected.collectAsList());
}
代码示例来源:origin: org.apache.spark/spark-sql
@Test
public void testTupleEncoderSchema() {
Encoder<Tuple2<String, Tuple2<String,String>>> encoder =
Encoders.tuple(Encoders.STRING(), Encoders.tuple(Encoders.STRING(), Encoders.STRING()));
List<Tuple2<String, Tuple2<String, String>>> data = Arrays.asList(tuple2("1", tuple2("a", "b")),
tuple2("2", tuple2("c", "d")));
Dataset<Row> ds1 = spark.createDataset(data, encoder).toDF("value1", "value2");
JavaPairRDD<String, Tuple2<String, String>> pairRDD = jsc.parallelizePairs(data);
Dataset<Row> ds2 = spark.createDataset(JavaPairRDD.toRDD(pairRDD), encoder)
.toDF("value1", "value2");
Assert.assertEquals(ds1.schema(), ds2.schema());
Assert.assertEquals(ds1.select(expr("value2._1")).collectAsList(),
ds2.select(expr("value2._1")).collectAsList());
}
代码示例来源:origin: org.apache.spark/spark-sql_2.11
@Test
public void testTupleEncoderSchema() {
Encoder<Tuple2<String, Tuple2<String,String>>> encoder =
Encoders.tuple(Encoders.STRING(), Encoders.tuple(Encoders.STRING(), Encoders.STRING()));
List<Tuple2<String, Tuple2<String, String>>> data = Arrays.asList(tuple2("1", tuple2("a", "b")),
tuple2("2", tuple2("c", "d")));
Dataset<Row> ds1 = spark.createDataset(data, encoder).toDF("value1", "value2");
JavaPairRDD<String, Tuple2<String, String>> pairRDD = jsc.parallelizePairs(data);
Dataset<Row> ds2 = spark.createDataset(JavaPairRDD.toRDD(pairRDD), encoder)
.toDF("value1", "value2");
Assert.assertEquals(ds1.schema(), ds2.schema());
Assert.assertEquals(ds1.select(expr("value2._1")).collectAsList(),
ds2.select(expr("value2._1")).collectAsList());
}
内容来源于网络,如有侵权,请联系作者删除!