cz.seznam.euphoria.core.client.flow.Flow.createInput()方法的使用及代码示例

x33g5p2x  于2022-01-19 转载在 其他  
字(8.2k)|赞(0)|评价(0)|浏览(76)

本文整理了Java中cz.seznam.euphoria.core.client.flow.Flow.createInput()方法的一些代码示例,展示了Flow.createInput()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Flow.createInput()方法的具体详情如下:
包路径:cz.seznam.euphoria.core.client.flow.Flow
类名称:Flow
方法名:createInput

Flow.createInput介绍

[英]Creates new input dataset.
[中]创建新的输入数据集。

代码示例

代码示例来源:origin: seznam/euphoria

public <T> Dataset<T> read(DataSource<T> src) {
  return new Dataset<>(wrap.createInput(src));
 }
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3),
   Arrays.asList(4, 5, 6)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(7, 8, 9),
   Arrays.asList(10, 11, 12)));
 final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(13, 14, 15),
   Arrays.asList(16, 17, 18)));
 return Union.of(first, second, third).output();
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3),
   Arrays.asList(4, 5, 6)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(7, 8, 9),
   Arrays.asList(10, 11, 12)));
 final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(13, 14, 15),
   Arrays.asList(16, 17, 18)));
 return Union.of(first, second, third).output();
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(4, 5, 6),
   Arrays.asList(7, 8, 9)));
 final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(10, 11, 12),
   Arrays.asList(13, 14, 15),
   Arrays.asList(16, 17, 18)));
 return Union.of(first, second, third).output();
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(4, 5, 6)));
 final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(7, 8, 9)));
 final Dataset<Integer> fourth = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(10, 11, 12)));
 final Dataset<Integer> fifth = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(13, 14, 15)));
 return Union.of(first, second, third, fourth, fifth).output();
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(4, 5, 6)));
 final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(7, 8, 9)));
 final Dataset<Integer> fourth = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(10, 11, 12)));
 final Dataset<Integer> fifth = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(13, 14, 15)));
 return Union.of(first, second, third, fourth, fifth).output();
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(4, 5, 6),
   Arrays.asList(7, 8, 9)));
 final Dataset<Integer> third = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(10, 11, 12),
   Arrays.asList(13, 14, 15),
   Arrays.asList(16, 17, 18)));
 return Union.of(first, second, third).output();
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3),
   Arrays.asList(4, 5, 6)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(7, 8, 9),
   Arrays.asList(10, 11, 12)));
 return Union.of(first, second).output();
}

代码示例来源:origin: seznam/euphoria

@Override
public Dataset<Integer> getOutput(Flow flow, boolean bounded) {
 final Dataset<Integer> first = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(1, 2, 3),
   Arrays.asList(4, 5, 6)));
 final Dataset<Integer> second = flow.createInput(ListDataSource.of(bounded,
   Arrays.asList(7, 8, 9),
   Arrays.asList(10, 11, 12)));
 return Union.of(first, second).output();
}

代码示例来源:origin: seznam/euphoria

static Dataset<Pair<Long, String>> getTestInput(Flow flow) {
 ListDataSource<Pair<Long, String>> source = ListDataSource.bounded(
   Benchmarks.testInput(Pair::of));
 return flow.createInput(source);
}

代码示例来源:origin: seznam/euphoria

/**
 * A convenience method to create a data set from the given source and
 * assign the elements event time using the user defined function.
 *
 * @param <T> the type of elements of the created input data set
 *
 * @param source the data source to represent as a data set
 * @param evtTimeFn the user defined event time extraction function
 *
 * @return a data set representing the specified source of data with assigned
 *          event time assigned
 */
public <T> Dataset<T> createInput(DataSource<T> source, ExtractEventTime<T> evtTimeFn) {
 Dataset<T> input = createInput(source);
 return AssignEventTime.of(input).using(Objects.requireNonNull(evtTimeFn)).output();
}

代码示例来源:origin: seznam/euphoria

protected abstract Dataset<OUT> getOutput(Dataset<LEFT> left, Dataset<RIGHT> right);
protected abstract List<LEFT> getLeftInput();

代码示例来源:origin: seznam/euphoria

protected abstract Dataset<OUT> getOutput(Dataset<LEFT> left, Dataset<RIGHT> right);
protected abstract List<LEFT> getLeftInput();

代码示例来源:origin: seznam/euphoria

public static <T> Dataset<T> createMockDataset(Flow flow, int numPartitions) {
 @SuppressWarnings("unchecked")
 List<T>[] partitions = new List[numPartitions];
 for (int i = 0; i < numPartitions; i++) {
  partitions[i] = new ArrayList<>();
 }
 return flow.createInput(ListDataSource.bounded(partitions));
}

代码示例来源:origin: seznam/euphoria

protected abstract Dataset<O> getOutput(Dataset<I> input);

代码示例来源:origin: seznam/euphoria

protected abstract Dataset<O> getOutput(Dataset<I> input);

代码示例来源:origin: seznam/euphoria

private void run() {
 Flow flow = Flow.create();
 Dataset<Pair<ImmutableBytesWritable, Result>> ds = flow.createInput(
   Utils.getHBaseSource(input, conf.get()));
 FlatMap.of(ds)
   .using((Pair<ImmutableBytesWritable, Result> p, Collector<byte[]> c) -> {
    writeCellsAsBytes(p.getSecond(), c);
   })
   .output()
   .persist(Utils.getSink(output, conf.get()));
 LOG.info("Starting flow reading from {} and persisting to {}", input, output);
 executor.submit(flow).join();
}

代码示例来源:origin: seznam/euphoria

@Override
 protected Dataset<String> buildFlow(Flow flow) {
  return SimpleWordCount.buildFlow(
    flow.createInput(ListDataSource.bounded(getInput())));
 }
}

代码示例来源:origin: seznam/euphoria

@Test
public void testDatasetConsumers() throws Exception {
 Dataset<Object> input = flow.createInput(new MockStreamDataSource<>());
 Dataset<Object> transformed = MapElements.of(input).using(e -> e).output();
 Dataset<Object> transformed2 = Filter.of(transformed).by(e -> false).output();
 Dataset<Object> union = Union.of(transformed, transformed2).output();
 assertEquals(1, input.getConsumers().size());
 assertEquals(2, transformed.getConsumers().size());
 assertEquals(1, transformed2.getConsumers().size());
 assertEquals(0, union.getConsumers().size());
 // the 'transformed' data set is consumed by Filter and Union operators
 assertEquals(toSet(Arrays.asList(Filter.class, Union.class)),
   toSet(transformed.getConsumers().stream().map(Object::getClass)));
}

代码示例来源:origin: seznam/euphoria

@Test
public void testTestSimpleOutput() throws IOException {
 List<String> data = Arrays.asList("a", "b", "c");
 ListDataSource<String> source = ListDataSource.unbounded(data);
 Dataset<String> input = flow.createInput(source);
 MapElements.of(input)
   .using(HBaseTestCase::put)
   .output()
   .persist(sink);
 new LocalExecutor().submit(flow).join();
 for (String v : data) {
  assertArrayEquals(b(v), get(v));
 }
}

相关文章