本文整理了Java中org.apache.spark.api.java.JavaRDD.fromRDD()
方法的一些代码示例,展示了JavaRDD.fromRDD()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。JavaRDD.fromRDD()
方法的具体详情如下:
包路径:org.apache.spark.api.java.JavaRDD
类名称:JavaRDD
方法名:fromRDD
暂无
代码示例来源:origin: spirom/learning-spark-with-java
JavaRDD.fromRDD(combinedRDD.rdd(), combinedRDD.classTag());
JavaRDD<Tuple2<String, Double>> tupleDividedRDD = tupleCombinedRDD.map(keyAndsumAndCount ->
new Tuple2<>(keyAndsumAndCount._1(), (double) keyAndsumAndCount._2()._1() / keyAndsumAndCount._2()._2()));
代码示例来源:origin: sujee81/SparkApps
public static void main(String[] args) {
DbConnection dbConnection = new DbConnection(MYSQL_DRIVER, MYSQL_CONNECTION_URL, MYSQL_USERNAME, MYSQL_PWD);
// Load data from MySQL
JdbcRDD<Object[]> jdbcRDD =
new JdbcRDD<>(sc.sc(), dbConnection, "select * from employees where emp_no >= ? and emp_no <= ?", 10001,
499999, 10, new MapResult(), ClassManifestFactory$.MODULE$.fromClass(Object[].class));
// Convert to JavaRDD
JavaRDD<Object[]> javaRDD = JavaRDD.fromRDD(jdbcRDD, ClassManifestFactory$.MODULE$.fromClass(Object[].class));
// Join first name and last name
List<String> employeeFullNameList = javaRDD.map(new Function<Object[], String>() {
@Override
public String call(final Object[] record) throws Exception {
return record[2] + " " + record[3];
}
}).collect();
for (String fullName : employeeFullNameList) {
LOGGER.info(fullName);
}
}
代码示例来源:origin: org.apache.pig/pig
} else {
for (RDD<Tuple> rdd : predecessors) {
JavaRDD<Tuple> jrdd = JavaRDD.fromRDD(rdd, SparkUtil.getManifest(Tuple.class));
JavaRDD<Tuple2<IndexedKey, Tuple>> rddPair = jrdd.map(new ToKeyValueFunction());
rddPairs.add(rddPair.rdd());
内容来源于网络,如有侵权,请联系作者删除!