org.apache.spark.rdd.RDD.getStorageLevel()方法的使用及代码示例

x33g5p2x  于2022-01-28 转载在 其他  
字(2.1k)|赞(0)|评价(0)|浏览(177)

本文整理了Java中org.apache.spark.rdd.RDD.getStorageLevel方法的一些代码示例,展示了RDD.getStorageLevel的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。RDD.getStorageLevel方法的具体详情如下:
包路径:org.apache.spark.rdd.RDD
类名称:RDD
方法名:getStorageLevel

RDD.getStorageLevel介绍

暂无

代码示例

代码示例来源:origin: apache/drill

private static void rddToString(RDD rdd, StringBuilder sb, String offset) {
 sb.append(offset).append(rdd.getClass().getCanonicalName()).append("[").append(rdd.hashCode()).append("]");
 if (rdd.getStorageLevel().useMemory()) {
  sb.append("(cached)");
 }
 sb.append("\n");
 Collection<Dependency> dependencies = JavaConversions.asJavaCollection(rdd.dependencies());
 if (dependencies != null) {
  offset += "\t";
  for (Dependency dependency : dependencies) {
   RDD parentRdd = dependency.rdd();
   rddToString(parentRdd, sb, offset);
  }
 } else if (rdd instanceof UnionRDD) {
  UnionRDD unionRDD = (UnionRDD) rdd;
  offset += "\t";
  Collection<RDD> parentRdds = JavaConversions.asJavaCollection(unionRDD.rdds());
  for (RDD parentRdd : parentRdds) {
   rddToString(parentRdd, sb, offset);
  }
 }
}

代码示例来源:origin: apache/tinkerpop

@Override
public List<String> ls(final String location) {
  final List<String> rdds = new ArrayList<>();
  final String wildCardLocation = (location.endsWith("*") ? location : location + "*").replace('\\', '/').replace(".", "\\.").replace("*", ".*");
  for (final RDD<?> rdd : Spark.getRDDs()) {
    if (rdd.name().replace('\\', '/').matches(wildCardLocation))
      rdds.add(rdd.name() + " [" + rdd.getStorageLevel().description() + "]");
  }
  return rdds;
}

代码示例来源:origin: org.apache.tinkerpop/spark-gremlin

@Override
public List<String> ls(final String location) {
  final List<String> rdds = new ArrayList<>();
  final String wildCardLocation = (location.endsWith("*") ? location : location + "*").replace('\\', '/').replace(".", "\\.").replace("*", ".*");
  for (final RDD<?> rdd : Spark.getRDDs()) {
    if (rdd.name().replace('\\', '/').matches(wildCardLocation))
      rdds.add(rdd.name() + " [" + rdd.getStorageLevel().description() + "]");
  }
  return rdds;
}

相关文章