本文整理了Java中cz.seznam.euphoria.core.client.dataset.Dataset.getSource()
方法的一些代码示例,展示了Dataset.getSource()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Dataset.getSource()
方法的具体详情如下:
包路径:cz.seznam.euphoria.core.client.dataset.Dataset
类名称:Dataset
方法名:getSource
[英]Retrieve source of data associated with this dataset. This might be null, if this dataset has no explicit source, it is calculated. If this method returns null, getProducer returns non null and vice versa.
[中]检索与此数据集关联的数据源。这可能为null,如果此数据集没有显式源,则会对其进行计算。如果此方法返回null,则getProducer返回非null,反之亦然。
代码示例来源:origin: seznam/euphoria
DataSource<?> raw = op.output().getSource();
if (raw.isBounded()) {
BoundedDataSource<?> source = raw.asBounded();
代码示例来源:origin: seznam/euphoria
output = createStream(op.output().getSource());
} else if (op instanceof FlatMap) {
output = execMap((Node) node, context);
代码示例来源:origin: seznam/euphoria
output = createStream(op.output().getSource());
} else if (op instanceof FlatMap) {
output = execMap((Node) node, context);
代码示例来源:origin: seznam/euphoria
@Override
public DataStream<?> translate(FlinkOperator<FlowUnfolder.InputOperator> operator,
StreamingExecutorContext context)
{
// get original datasource from operator
DataSource<?> ds = operator.output().getSource();
return context.getExecutionEnvironment()
.addSource(new DataSourceWrapper<>("input::" + operator.getName(), ds))
.setParallelism(operator.getParallelism());
}
}
代码示例来源:origin: seznam/euphoria
@Override
public DataSet translate(
FlinkOperator<FlowUnfolder.InputOperator> operator,
BatchExecutorContext context) {
// get original datasource from operator
BoundedDataSource<?> ds = operator.output().getSource().asBounded();
int envParallel = context.getExecutionEnvironment().getParallelism();
DataSourceWrapper<?> wrapper = new DataSourceWrapper<>(
ds, splitAssignerFactory, envParallel);
return context
.getExecutionEnvironment()
.createInput(wrapper)
.setParallelism(Math.min(envParallel, wrapper.getParallelism()));
}
}
代码示例来源:origin: seznam/euphoria
@Override
public JavaRDD<?> translate(FlowUnfolder.InputOperator operator, SparkExecutorContext context) {
// get original datasource from operator
DataSource<?> ds = operator.output().getSource();
try {
final long desiredSplitSize = context.getSettings()
.getLong(DESIRED_SPLIT_SIZE, DEFAULT_DESIRED_SPLIT_SIZE);
final Configuration conf = DataSourceInputFormat.configure(
new Configuration(), ds, desiredSplitSize);
@SuppressWarnings("unchecked")
JavaPairRDD<Object, Object> pairs =
context
.getExecutionEnvironment()
.newAPIHadoopRDD(conf, DataSourceInputFormat.class, Object.class, Object.class)
.setName(operator.getName() + "::read");
// map values to WindowedElement
return pairs
.values()
.map(v -> new SparkElement<>(GlobalWindowing.Window.get(), 0L, v))
.setName(operator.getName() + "::windowed-element");
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
内容来源于网络,如有侵权,请联系作者删除!