org.apache.flink.api.java.operators.GroupReduceOperator.withForwardedFields()方法的使用及代码示例

x33g5p2x  于2022-01-20 转载在 其他  
字(10.0k)|赞(0)|评价(0)|浏览(92)

本文整理了Java中org.apache.flink.api.java.operators.GroupReduceOperator.withForwardedFields()方法的一些代码示例，展示了GroupReduceOperator.withForwardedFields()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。GroupReduceOperator.withForwardedFields()方法的具体详情如下：
包路径：org.apache.flink.api.java.operators.GroupReduceOperator
类名称：GroupReduceOperator
方法名：withForwardedFields

GroupReduceOperator.withForwardedFields介绍

暂无

代码示例

代码示例来源：origin: apache/flink

out.collect(values.iterator().next());
}).withForwardedFields("0;1");

代码示例来源：origin: apache/flink

.withForwardedFields("0")

代码示例来源：origin: apache/flink

.groupBy(1)
.sortGroup(0, Order.ASCENDING)
.reduceGroup(new IdentityGroupReducerCombinable<Tuple3<Long,Long,Long>>()).withForwardedFields("0", "1");

代码示例来源：origin: apache/flink

.groupBy(1)
.sortGroup(0, Order.ASCENDING)
.reduceGroup(new IdentityGroupReducerCombinable<Tuple3<Long,Long,Long>>()).withForwardedFields("0", "1");

代码示例来源：origin: apache/flink

.reduceGroup(new IdentityGroupReducerCombinable<Tuple3<Long,Long,Long>>()).withForwardedFields("0", "1", "2")

代码示例来源：origin: apache/flink

.withForwardedFields("*").setParallelism(p).name("Map1")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
.withForwardedFields("*").setParallelism(p).name("Reduce1")
.map(new IdentityMapper<Long>())
.withForwardedFields("*").setParallelism(p * 2).name("Map2")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
.withForwardedFields("*").setParallelism(p * 2).name("Reduce2")
.output(new DiscardingOutputFormat<Long>()).setParallelism(p * 2).name("Sink");

代码示例来源：origin: apache/flink

.withForwardedFields("*").setParallelism(p).name("Map1")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
.withForwardedFields("*").setParallelism(p).name("Reduce1")
.map(new IdentityMapper<Long>())
.withForwardedFields("*").setParallelism(p).name("Map2")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
.withForwardedFields("*").setParallelism(p * 2).name("Reduce2")
.output(new DiscardingOutputFormat<Long>()).setParallelism(p * 2).name("Sink");

代码示例来源：origin: apache/flink

.withForwardedFields("*").setParallelism(p).name("Map1")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
  .withForwardedFields("*").setParallelism(p).name("Reduce1")
.map(new IdentityMapper<Long>())
  .withForwardedFields("*").setParallelism(p * 2).name("Map2")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
  .withForwardedFields("*").setParallelism(p * 2).name("Reduce2")
.output(new DiscardingOutputFormat<Long>()).setParallelism(p * 2).name("Sink");

代码示例来源：origin: apache/flink

.withForwardedFields("*").setParallelism(p * 2).name("Map1")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
  .withForwardedFields("*").setParallelism(p * 2).name("Reduce1")
.map(new IdentityMapper<Long>())
  .withForwardedFields("*").setParallelism(p).name("Map2")
.groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
  .withForwardedFields("*").setParallelism(p).name("Reduce2")
.output(new DiscardingOutputFormat<Long>()).setParallelism(p).name("Sink");

代码示例来源：origin: apache/flink

.withForwardedFields("*").setParallelism(5);
DataSet<Long> reduce2 = set2
    .groupBy("*").reduceGroup(new IdentityGroupReducer<Long>())
      .withForwardedFields("*").setParallelism(7);

代码示例来源：origin: apache/flink

.reduceGroup(new IdentityGroupReducerCombinable<Tuple3<Long,Long,Long>>()).withForwardedFields("0", "1", "2")

代码示例来源：origin: apache/flink

.withForwardedFields("*").name(NEXT_WORKSET_REDUCER_NAME);

代码示例来源：origin: apache/flink

@Test
public void testSemanticPropsWithKeySelector5() {
  final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
  DataSet<Tuple5<Integer, Long, String, Long, Integer>> tupleDs = env.fromCollection(emptyTupleData, tupleTypeInfo);
  GroupReduceOperator<Tuple5<Integer, Long, String, Long, Integer>, Tuple5<Integer, Long, String, Long, Integer>> reduceOp =
      tupleDs.groupBy(new DummyTestKeySelector())
          .reduceGroup(new DummyGroupReduceFunction3())
          .withForwardedFields("4->0;3;3->1;2");
  SemanticProperties semProps = reduceOp.getSemanticProperties();
  assertTrue(semProps.getForwardingTargetFields(0, 0).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 1).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 2).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 3).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 4).size() == 1);
  assertTrue(semProps.getForwardingTargetFields(0, 4).contains(2));
  assertTrue(semProps.getForwardingTargetFields(0, 5).size() == 2);
  assertTrue(semProps.getForwardingTargetFields(0, 5).contains(1));
  assertTrue(semProps.getForwardingTargetFields(0, 5).contains(3));
  assertTrue(semProps.getForwardingTargetFields(0, 6).size() == 1);
  assertTrue(semProps.getForwardingTargetFields(0, 6).contains(0));
  assertTrue(semProps.getForwardingSourceField(0, 0) == 6);
  assertTrue(semProps.getForwardingSourceField(0, 1) == 5);
  assertTrue(semProps.getForwardingSourceField(0, 2) == 4);
  assertTrue(semProps.getForwardingSourceField(0, 3) == 5);
  assertTrue(semProps.getForwardingSourceField(0, 4) < 0);
  assertTrue(semProps.getReadFields(0) == null);
}

代码示例来源：origin: apache/flink

@Test
public void testSemanticPropsWithKeySelector6() {
  final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
  DataSet<Tuple5<Integer, Long, String, Long, Integer>> tupleDs = env.fromCollection(emptyTupleData, tupleTypeInfo);
  GroupReduceOperator<Tuple5<Integer, Long, String, Long, Integer>, Tuple5<Integer, Long, String, Long, Integer>> reduceOp =
      tupleDs.groupBy(new DummyTestKeySelector())
          .sortGroup(new DummyTestKeySelector(), Order.ASCENDING)
          .reduceGroup(new DummyGroupReduceFunction3())
          .withForwardedFields("4->0;3;3->1;2");
  SemanticProperties semProps = reduceOp.getSemanticProperties();
  assertTrue(semProps.getForwardingTargetFields(0, 0).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 1).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 2).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 3).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 4).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 5).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 6).size() == 1);
  assertTrue(semProps.getForwardingTargetFields(0, 6).contains(2));
  assertTrue(semProps.getForwardingTargetFields(0, 7).size() == 2);
  assertTrue(semProps.getForwardingTargetFields(0, 7).contains(1));
  assertTrue(semProps.getForwardingTargetFields(0, 7).contains(3));
  assertTrue(semProps.getForwardingTargetFields(0, 8).size() == 1);
  assertTrue(semProps.getForwardingTargetFields(0, 8).contains(0));
  assertTrue(semProps.getForwardingSourceField(0, 0) == 8);
  assertTrue(semProps.getForwardingSourceField(0, 1) == 7);
  assertTrue(semProps.getForwardingSourceField(0, 2) == 6);
  assertTrue(semProps.getForwardingSourceField(0, 3) == 7);
  assertTrue(semProps.getForwardingSourceField(0, 4) < 0);
  assertTrue(semProps.getReadFields(0) == null);
}

代码示例来源：origin: apache/flink

@Test
public void testSemanticPropsWithKeySelector3() {
  final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
  DataSet<Tuple5<Integer, Long, String, Long, Integer>> tupleDs = env.fromCollection(emptyTupleData, tupleTypeInfo);
  GroupReduceOperator<Tuple5<Integer, Long, String, Long, Integer>, Tuple5<Integer, Long, String, Long, Integer>> reduceOp =
      tupleDs.groupBy(new DummyTestKeySelector())
          .reduceGroup(new DummyGroupReduceFunction2())
            .withForwardedFields("0->4;1;1->3;2");
  SemanticProperties semProps = reduceOp.getSemanticProperties();
  assertTrue(semProps.getForwardingTargetFields(0, 0).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 1).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 2).size() == 1);
  assertTrue(semProps.getForwardingTargetFields(0, 2).contains(4));
  assertTrue(semProps.getForwardingTargetFields(0, 3).size() == 2);
  assertTrue(semProps.getForwardingTargetFields(0, 3).contains(1));
  assertTrue(semProps.getForwardingTargetFields(0, 3).contains(3));
  assertTrue(semProps.getForwardingTargetFields(0, 4).size() == 1);
  assertTrue(semProps.getForwardingTargetFields(0, 4).contains(2));
  assertTrue(semProps.getForwardingTargetFields(0, 5).size() == 0);
  assertTrue(semProps.getForwardingTargetFields(0, 6).size() == 0);
  assertTrue(semProps.getForwardingSourceField(0, 0) < 0);
  assertTrue(semProps.getForwardingSourceField(0, 1) == 3);
  assertTrue(semProps.getForwardingSourceField(0, 2) == 4);
  assertTrue(semProps.getForwardingSourceField(0, 3) == 3);
  assertTrue(semProps.getForwardingSourceField(0, 4) == 2);
  assertTrue(semProps.getReadFields(0).size() == 3);
  assertTrue(semProps.getReadFields(0).contains(2));
  assertTrue(semProps.getReadFields(0).contains(5));
  assertTrue(semProps.getReadFields(0).contains(6));
}

代码示例来源：origin: apache/flink

.sortGroup(new DummyTestKeySelector(), Order.ASCENDING)
.reduceGroup(new DummyGroupReduceFunction2())
  .withForwardedFields("0->4;1;1->3;2");

代码示例来源：origin: apache/flink

})
.name(NEXT_WORKSET_REDUCER_NAME)
.withForwardedFields("1->1","2->2","0->0");

代码示例来源：origin: org.apache.flink/flink-examples-batch

out.collect(values.iterator().next());
}).withForwardedFields("0;1");

代码示例来源：origin: org.apache.flink/flink-examples-batch_2.10

out.collect(values.iterator().next());
}).withForwardedFields("0;1");

代码示例来源：origin: org.apache.flink/flink-java-examples

out.collect(values.iterator().next());
}).withForwardedFields("0;1");

内容来源于网络，如有侵权，请联系作者删除！

GroupReduceOperator

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com