本文整理了Java中cz.seznam.euphoria.core.client.flow.Flow.createInput()
方法的一些代码示例,展示了Flow.createInput()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Flow.createInput()
方法的具体详情如下:
包路径:cz.seznam.euphoria.core.client.flow.Flow
类名称:Flow
方法名:createInput
[英]Creates new input dataset.
[中]创建新的输入数据集。
代码示例来源:origin: seznam/euphoria
public <T> Dataset<T> read(DataSource<T> src) {
return new Dataset<>(wrap.createInput(src));
}
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3),
Arrays.asList(4, 5, 6)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(7, 8, 9),
Arrays.asList(10, 11, 12)));
final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(13, 14, 15),
Arrays.asList(16, 17, 18)));
return Union.of(first, second, third).output();
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3),
Arrays.asList(4, 5, 6)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(7, 8, 9),
Arrays.asList(10, 11, 12)));
final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(13, 14, 15),
Arrays.asList(16, 17, 18)));
return Union.of(first, second, third).output();
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(4, 5, 6),
Arrays.asList(7, 8, 9)));
final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(10, 11, 12),
Arrays.asList(13, 14, 15),
Arrays.asList(16, 17, 18)));
return Union.of(first, second, third).output();
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(4, 5, 6)));
final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(7, 8, 9)));
final Dataset<Integer> fourth = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(10, 11, 12)));
final Dataset<Integer> fifth = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(13, 14, 15)));
return Union.of(first, second, third, fourth, fifth).output();
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(4, 5, 6)));
final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(7, 8, 9)));
final Dataset<Integer> fourth = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(10, 11, 12)));
final Dataset<Integer> fifth = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(13, 14, 15)));
return Union.of(first, second, third, fourth, fifth).output();
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(4, 5, 6),
Arrays.asList(7, 8, 9)));
final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(10, 11, 12),
Arrays.asList(13, 14, 15),
Arrays.asList(16, 17, 18)));
return Union.of(first, second, third).output();
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3),
Arrays.asList(4, 5, 6)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(7, 8, 9),
Arrays.asList(10, 11, 12)));
return Union.of(first, second).output();
}
代码示例来源:origin: seznam/euphoria
@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(1, 2, 3),
Arrays.asList(4, 5, 6)));
final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
Arrays.asList(7, 8, 9),
Arrays.asList(10, 11, 12)));
return Union.of(first, second).output();
}
代码示例来源:origin: seznam/euphoria
static Dataset<Pair<Long, String>> getTestInput(Flow flow) {
ListDataSource<Pair<Long, String>> source = ListDataSource.bounded(
Benchmarks.testInput(Pair::of));
return flow.createInput(source);
}
代码示例来源:origin: seznam/euphoria
/**
* A convenience method to create a data set from the given source and
* assign the elements event time using the user defined function.
*
* @param <T> the type of elements of the created input data set
*
* @param source the data source to represent as a data set
* @param evtTimeFn the user defined event time extraction function
*
* @return a data set representing the specified source of data with assigned
* event time assigned
*/
public <T> Dataset<T> createInput(DataSource<T> source, ExtractEventTime<T> evtTimeFn) {
Dataset<T> input = createInput(source);
return AssignEventTime.of(input).using(Objects.requireNonNull(evtTimeFn)).output();
}
代码示例来源:origin: seznam/euphoria
protected abstract Dataset<OUT> getOutput(Dataset<LEFT> left, Dataset<RIGHT> right);
protected abstract List<LEFT> getLeftInput();
代码示例来源:origin: seznam/euphoria
protected abstract Dataset<OUT> getOutput(Dataset<LEFT> left, Dataset<RIGHT> right);
protected abstract List<LEFT> getLeftInput();
代码示例来源:origin: seznam/euphoria
public static <T> Dataset<T> createMockDataset(Flow flow, int numPartitions) {
@SuppressWarnings("unchecked")
List<T>[] partitions = new List[numPartitions];
for (int i = 0; i < numPartitions; i++) {
partitions[i] = new ArrayList<>();
}
return flow.createInput(ListDataSource.bounded(partitions));
}
代码示例来源:origin: seznam/euphoria
protected abstract Dataset<O> getOutput(Dataset<I> input);
代码示例来源:origin: seznam/euphoria
protected abstract Dataset<O> getOutput(Dataset<I> input);
代码示例来源:origin: seznam/euphoria
private void run() {
Flow flow = Flow.create();
Dataset<Pair<ImmutableBytesWritable, Result>> ds = flow.createInput(
Utils.getHBaseSource(input, conf.get()));
FlatMap.of(ds)
.using((Pair<ImmutableBytesWritable, Result> p, Collector<byte[]> c) -> {
writeCellsAsBytes(p.getSecond(), c);
})
.output()
.persist(Utils.getSink(output, conf.get()));
LOG.info("Starting flow reading from {} and persisting to {}", input, output);
executor.submit(flow).join();
}
代码示例来源:origin: seznam/euphoria
@Override
protected Dataset<String> buildFlow(Flow flow) {
return SimpleWordCount.buildFlow(
flow.createInput(ListDataSource.bounded(getInput())));
}
}
代码示例来源:origin: seznam/euphoria
@Test
public void testDatasetConsumers() throws Exception {
Dataset<Object> input = flow.createInput(new MockStreamDataSource<>());
Dataset<Object> transformed = MapElements.of(input).using(e -> e).output();
Dataset<Object> transformed2 = Filter.of(transformed).by(e -> false).output();
Dataset<Object> union = Union.of(transformed, transformed2).output();
assertEquals(1, input.getConsumers().size());
assertEquals(2, transformed.getConsumers().size());
assertEquals(1, transformed2.getConsumers().size());
assertEquals(0, union.getConsumers().size());
// the 'transformed' data set is consumed by Filter and Union operators
assertEquals(toSet(Arrays.asList(Filter.class, Union.class)),
toSet(transformed.getConsumers().stream().map(Object::getClass)));
}
代码示例来源:origin: seznam/euphoria
@Test
public void testTestSimpleOutput() throws IOException {
List<String> data = Arrays.asList("a", "b", "c");
ListDataSource<String> source = ListDataSource.unbounded(data);
Dataset<String> input = flow.createInput(source);
MapElements.of(input)
.using(HBaseTestCase::put)
.output()
.persist(sink);
new LocalExecutor().submit(flow).join();
for (String v : data) {
assertArrayEquals(b(v), get(v));
}
}
内容来源于网络,如有侵权,请联系作者删除!