本文整理了Java中org.apache.spark.api.java.JavaPairRDD.isEmpty()
方法的一些代码示例,展示了JavaPairRDD.isEmpty()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。JavaPairRDD.isEmpty()
方法的具体详情如下:
包路径:org.apache.spark.api.java.JavaPairRDD
类名称:JavaPairRDD
方法名:isEmpty
暂无
代码示例来源:origin: OryxProject/oryx
@Override
public void call(JavaPairRDD<K,M> newData) throws IOException {
if (newData.isEmpty()) {
log.debug("RDD was empty");
} else {
Iterable<U> updates = modelManager.buildUpdates(newData);
if (updates != null) {
try (TopicProducer<String, U> producer = new TopicProducerImpl<>(updateBroker, updateTopic, true)) {
updates.forEach(update -> producer.send("UP", update));
}
}
}
}
代码示例来源:origin: OryxProject/oryx
@Override
public void call(JavaPairRDD<K,M> rdd, Time time) throws IOException {
if (rdd.isEmpty()) {
log.info("RDD was empty, not saving to HDFS");
} else {
String file = prefix + "-" + time.milliseconds() + "." + suffix;
Path path = new Path(file);
FileSystem fs = FileSystem.get(path.toUri(), hadoopConf);
if (fs.exists(path)) {
log.warn("Saved data already existed, possibly from a failed job. Deleting {}", path);
fs.delete(path, true);
}
log.info("Saving RDD to HDFS at {}", file);
rdd.mapToPair(
new ValueToWritableFunction<>(keyClass, messageClass, keyWritableClass, messageWritableClass)
).saveAsNewAPIHadoopFile(
file,
keyWritableClass,
messageWritableClass,
SequenceFileOutputFormat.class,
hadoopConf);
}
}
}
代码示例来源:origin: OryxProject/oryx
throws IOException, InterruptedException {
if (newData.isEmpty()) {
log.info("No data in current generation's RDD; nothing to do");
return;
代码示例来源:origin: com.davidbracewell/mango
@Override
public boolean isEmpty() {
return rdd.isEmpty();
}
代码示例来源:origin: com.cloudera.oryx/oryx-lambda
@Override
public void call(JavaPairRDD<K,M> newData) throws IOException {
if (newData.isEmpty()) {
log.debug("RDD was empty");
} else {
Iterable<U> updates = modelManager.buildUpdates(newData);
if (updates != null) {
try (TopicProducer<String, U> producer = new TopicProducerImpl<>(updateBroker, updateTopic, true)) {
updates.forEach(update -> producer.send("UP", update));
}
}
}
}
代码示例来源:origin: Stratio/Decision
@Override
public Void call(JavaPairRDD<StreamAction, Iterable<StratioStreamingMessage>> rdd) throws Exception {
if (!rdd.isEmpty()) {
rdd.mapPartitions(
new FlatMapFunction<Iterator<Tuple2<StreamAction, Iterable<StratioStreamingMessage>>>, Object>() {
@Override public Iterable<Object> call(
Iterator<Tuple2<StreamAction, Iterable<StratioStreamingMessage>>> tuple2Iterator)
throws Exception {
while (tuple2Iterator.hasNext()) {
process(tuple2Iterator.next()._2());
}
return new ArrayList<Object>();
}
}).count();
}
return null;
}
代码示例来源:origin: com.cloudera.oryx/oryx-lambda
@Override
public void call(JavaPairRDD<K,M> rdd, Time time) throws IOException {
if (rdd.isEmpty()) {
log.info("RDD was empty, not saving to HDFS");
} else {
String file = prefix + "-" + time.milliseconds() + "." + suffix;
Path path = new Path(file);
FileSystem fs = FileSystem.get(path.toUri(), hadoopConf);
if (fs.exists(path)) {
log.warn("Saved data already existed, possibly from a failed job. Deleting {}", path);
fs.delete(path, true);
}
log.info("Saving RDD to HDFS at {}", file);
rdd.mapToPair(
new ValueToWritableFunction<>(keyClass, messageClass, keyWritableClass, messageWritableClass)
).saveAsNewAPIHadoopFile(
file,
keyWritableClass,
messageWritableClass,
SequenceFileOutputFormat.class,
hadoopConf);
}
}
}
代码示例来源:origin: com.cloudera.oryx/oryx-lambda
throws IOException, InterruptedException {
if (newData.isEmpty()) {
log.info("No data in current generation's RDD; nothing to do");
return;
代码示例来源:origin: locationtech/geowave
combinedResults.isEmpty();
内容来源于网络,如有侵权,请联系作者删除!