cz.seznam.euphoria.core.client.dataset.Dataset.getSource()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(2.8k)|赞(0)|评价(0)|浏览(116)

本文整理了Java中cz.seznam.euphoria.core.client.dataset.Dataset.getSource()方法的一些代码示例,展示了Dataset.getSource()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Dataset.getSource()方法的具体详情如下:
包路径:cz.seznam.euphoria.core.client.dataset.Dataset
类名称:Dataset
方法名:getSource

Dataset.getSource介绍

[英]Retrieve source of data associated with this dataset. This might be null, if this dataset has no explicit source, it is calculated. If this method returns null, getProducer returns non null and vice versa.
[中]检索与此数据集关联的数据源。这可能为null,如果此数据集没有显式源,则会对其进行计算。如果此方法返回null,则getProducer返回非null,反之亦然。

代码示例

代码示例来源:origin: seznam/euphoria

DataSource<?> raw = op.output().getSource();
if (raw.isBounded()) {
 BoundedDataSource<?> source = raw.asBounded();

代码示例来源:origin: seznam/euphoria

output = createStream(op.output().getSource());
} else if (op instanceof FlatMap) {
 output = execMap((Node) node, context);

代码示例来源:origin: seznam/euphoria

output = createStream(op.output().getSource());
} else if (op instanceof FlatMap) {
 output = execMap((Node) node, context);

代码示例来源:origin: seznam/euphoria

@Override
 public DataStream<?> translate(FlinkOperator<FlowUnfolder.InputOperator> operator,
                 StreamingExecutorContext context)
 {
  // get original datasource from operator
  DataSource<?> ds = operator.output().getSource();

  return context.getExecutionEnvironment()
      .addSource(new DataSourceWrapper<>("input::" + operator.getName(), ds))
      .setParallelism(operator.getParallelism());
 }
}

代码示例来源:origin: seznam/euphoria

@Override
 public DataSet translate(
   FlinkOperator<FlowUnfolder.InputOperator> operator,
   BatchExecutorContext context) {

  // get original datasource from operator
  BoundedDataSource<?> ds = operator.output().getSource().asBounded();

  int envParallel = context.getExecutionEnvironment().getParallelism();
  DataSourceWrapper<?> wrapper = new DataSourceWrapper<>(
    ds, splitAssignerFactory, envParallel);
  return context
    .getExecutionEnvironment()
    .createInput(wrapper)
    .setParallelism(Math.min(envParallel, wrapper.getParallelism()));
 }
}

代码示例来源:origin: seznam/euphoria

@Override
 public JavaRDD<?> translate(FlowUnfolder.InputOperator operator, SparkExecutorContext context) {

  // get original datasource from operator
  DataSource<?> ds = operator.output().getSource();

  try {
   final long desiredSplitSize = context.getSettings()
     .getLong(DESIRED_SPLIT_SIZE, DEFAULT_DESIRED_SPLIT_SIZE);
   final Configuration conf = DataSourceInputFormat.configure(
     new Configuration(), ds, desiredSplitSize);

   @SuppressWarnings("unchecked")
   JavaPairRDD<Object, Object> pairs =
     context
       .getExecutionEnvironment()
       .newAPIHadoopRDD(conf, DataSourceInputFormat.class, Object.class, Object.class)
       .setName(operator.getName() + "::read");

   // map values to WindowedElement
   return pairs
     .values()
     .map(v -> new SparkElement<>(GlobalWindowing.Window.get(), 0L,  v))
     .setName(operator.getName() + "::windowed-element");

  } catch (IOException e) {
   throw new RuntimeException(e);
  }
 }
}

相关文章