org.apache.flink.api.common.operators.Operator.getParameters()方法的使用及代码示例

x33g5p2x  于2022-01-26 转载在 其他  
字(18.0k)|赞(0)|评价(0)|浏览(97)

本文整理了Java中org.apache.flink.api.common.operators.Operator.getParameters()方法的一些代码示例，展示了Operator.getParameters()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Operator.getParameters()方法的具体详情如下：
包路径：org.apache.flink.api.common.operators.Operator
类名称：Operator
方法名：getParameters

Operator.getParameters介绍

[英]Gets the stub parameters of this contract. The stub parameters are a map that maps string keys to string or integer values. The map is accessible by the user code at runtime. Parameters that the user code needs to access at runtime to configure its behavior are typically stored in that configuration object.
[中]获取此合约的存根参数。存根参数是将字符串键映射到字符串或整数值的映射。用户代码可以在运行时访问映射。用户代码在运行时需要访问以配置其行为的参数通常存储在该配置对象中。

代码示例

代码示例来源：origin: apache/flink

private <I, O> org.apache.flink.api.common.operators.Operator<O> translateSingleInputOperator(SingleInputOperator<?, ?, ?> op) {
  @SuppressWarnings("unchecked")
  SingleInputOperator<I, O, ?> typedOp = (SingleInputOperator<I, O, ?>) op;
  @SuppressWarnings("unchecked")
  DataSet<I> typedInput = (DataSet<I>) op.getInput();
  Operator<I> input = translate(typedInput);
  org.apache.flink.api.common.operators.Operator<O> dataFlowOp = typedOp.translateToDataFlow(input);
  if (op instanceof UdfOperator<?>) {
    @SuppressWarnings("unchecked")
    SingleInputUdfOperator<I, O, ?> udfOp = (SingleInputUdfOperator<I, O, ?>) op;
    // set configuration parameters
    Configuration opParams = udfOp.getParameters();
    if (opParams != null) {
      dataFlowOp.getParameters().addAll(opParams);
    }
    if (dataFlowOp instanceof org.apache.flink.api.common.operators.SingleInputOperator) {
      org.apache.flink.api.common.operators.SingleInputOperator<?, O, ?> unaryOp =
          (org.apache.flink.api.common.operators.SingleInputOperator<?, O, ?>) dataFlowOp;
      // set the semantic properties
      unaryOp.setSemanticProperties(udfOp.getSemanticProperties());
    }
  }
  return dataFlowOp;
}

代码示例来源：origin: apache/flink

private <I1, I2, O> org.apache.flink.api.common.operators.Operator<O> translateTwoInputOperator(TwoInputOperator<?, ?, ?, ?> op) {
  @SuppressWarnings("unchecked")
  TwoInputOperator<I1, I2, O, ?> typedOp = (TwoInputOperator<I1, I2, O, ?>) op;
  @SuppressWarnings("unchecked")
  DataSet<I1> typedInput1 = (DataSet<I1>) op.getInput1();
  @SuppressWarnings("unchecked")
  DataSet<I2> typedInput2 = (DataSet<I2>) op.getInput2();
  Operator<I1> input1 = translate(typedInput1);
  Operator<I2> input2 = translate(typedInput2);
  org.apache.flink.api.common.operators.Operator<O> dataFlowOp = typedOp.translateToDataFlow(input1, input2);
  if (op instanceof UdfOperator<?>) {
    @SuppressWarnings("unchecked")
    TwoInputUdfOperator<I1, I2, O, ?> udfOp = (TwoInputUdfOperator<I1, I2, O, ?>) op;
    // set configuration parameters
    Configuration opParams = udfOp.getParameters();
    if (opParams != null) {
      dataFlowOp.getParameters().addAll(opParams);
    }
    if (dataFlowOp instanceof org.apache.flink.api.common.operators.DualInputOperator) {
      org.apache.flink.api.common.operators.DualInputOperator<?, ?,  O, ?> binaryOp =
          (org.apache.flink.api.common.operators.DualInputOperator<?, ?, O, ?>) dataFlowOp;
      // set the semantic properties
      binaryOp.setSemanticProperties(udfOp.getSemanticProperties());
    }
  }
  return dataFlowOp;
}

代码示例来源：origin: apache/flink

private JobVertex createDataSinkVertex(SinkPlanNode node) throws CompilerException {
  final OutputFormatVertex vertex = new OutputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSinkTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  return vertex;
}

代码示例来源：origin: apache/flink

private InputFormatVertex createDataSourceVertex(SourcePlanNode node) throws CompilerException {
  final InputFormatVertex vertex = new InputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSourceTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  config.setOutputSerializer(node.getSerializer());
  return vertex;
}

代码示例来源：origin: apache/flink

config.setStubParameters(node.getProgramOperator().getParameters());

代码示例来源：origin: apache/flink

private JobVertex createDualInputVertex(DualInputPlanNode node) throws CompilerException {
  final String taskName = node.getNodeName();
  final DriverStrategy ds = node.getDriverStrategy();
  final JobVertex vertex = new JobVertex(taskName);
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass( (this.currentIteration != null && node.isOnDynamicPath()) ? IterationIntermediateTask.class : BatchTask.class);
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  
  // set the driver strategy
  config.setDriver(ds.getDriverClass());
  config.setDriverStrategy(ds);
  if (node.getComparator1() != null) {
    config.setDriverComparator(node.getComparator1(), 0);
  }
  if (node.getComparator2() != null) {
    config.setDriverComparator(node.getComparator2(), 1);
  }
  if (node.getPairComparator() != null) {
    config.setDriverPairComparator(node.getPairComparator());
  }
  
  // assign memory, file-handles, etc.
  assignDriverResources(node, config);
  return vertex;
}

代码示例来源：origin: com.alibaba.blink/flink-java

private <I, O> org.apache.flink.api.common.operators.Operator<O> translateSingleInputOperator(SingleInputOperator<?, ?, ?> op) {
  @SuppressWarnings("unchecked")
  SingleInputOperator<I, O, ?> typedOp = (SingleInputOperator<I, O, ?>) op;
  @SuppressWarnings("unchecked")
  DataSet<I> typedInput = (DataSet<I>) op.getInput();
  Operator<I> input = translate(typedInput);
  org.apache.flink.api.common.operators.Operator<O> dataFlowOp = typedOp.translateToDataFlow(input);
  if (op instanceof UdfOperator<?>) {
    @SuppressWarnings("unchecked")
    SingleInputUdfOperator<I, O, ?> udfOp = (SingleInputUdfOperator<I, O, ?>) op;
    // set configuration parameters
    Configuration opParams = udfOp.getParameters();
    if (opParams != null) {
      dataFlowOp.getParameters().addAll(opParams);
    }
    if (dataFlowOp instanceof org.apache.flink.api.common.operators.SingleInputOperator) {
      org.apache.flink.api.common.operators.SingleInputOperator<?, O, ?> unaryOp =
          (org.apache.flink.api.common.operators.SingleInputOperator<?, O, ?>) dataFlowOp;
      // set the semantic properties
      unaryOp.setSemanticProperties(udfOp.getSemanticProperties());
    }
  }
  return dataFlowOp;
}

代码示例来源：origin: org.apache.flink/flink-java

private <I, O> org.apache.flink.api.common.operators.Operator<O> translateSingleInputOperator(SingleInputOperator<?, ?, ?> op) {
  @SuppressWarnings("unchecked")
  SingleInputOperator<I, O, ?> typedOp = (SingleInputOperator<I, O, ?>) op;
  @SuppressWarnings("unchecked")
  DataSet<I> typedInput = (DataSet<I>) op.getInput();
  Operator<I> input = translate(typedInput);
  org.apache.flink.api.common.operators.Operator<O> dataFlowOp = typedOp.translateToDataFlow(input);
  if (op instanceof UdfOperator<?>) {
    @SuppressWarnings("unchecked")
    SingleInputUdfOperator<I, O, ?> udfOp = (SingleInputUdfOperator<I, O, ?>) op;
    // set configuration parameters
    Configuration opParams = udfOp.getParameters();
    if (opParams != null) {
      dataFlowOp.getParameters().addAll(opParams);
    }
    if (dataFlowOp instanceof org.apache.flink.api.common.operators.SingleInputOperator) {
      org.apache.flink.api.common.operators.SingleInputOperator<?, O, ?> unaryOp =
          (org.apache.flink.api.common.operators.SingleInputOperator<?, O, ?>) dataFlowOp;
      // set the semantic properties
      unaryOp.setSemanticProperties(udfOp.getSemanticProperties());
    }
  }
  return dataFlowOp;
}

代码示例来源：origin: org.apache.flink/flink-optimizer

private JobVertex createDataSinkVertex(SinkPlanNode node) throws CompilerException {
  final OutputFormatVertex vertex = new OutputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSinkTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer_2.11

private JobVertex createDataSinkVertex(SinkPlanNode node) throws CompilerException {
  final OutputFormatVertex vertex = new OutputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSinkTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer_2.10

private JobVertex createDataSinkVertex(SinkPlanNode node) throws CompilerException {
  final OutputFormatVertex vertex = new OutputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSinkTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  return vertex;
}

代码示例来源：origin: com.alibaba.blink/flink-optimizer

private JobVertex createDataSinkVertex(SinkPlanNode node) throws CompilerException {
  final OutputFormatVertex vertex = new OutputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSinkTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer_2.10

private InputFormatVertex createDataSourceVertex(SourcePlanNode node) throws CompilerException {
  final InputFormatVertex vertex = new InputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSourceTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  config.setOutputSerializer(node.getSerializer());
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer_2.11

private InputFormatVertex createDataSourceVertex(SourcePlanNode node) throws CompilerException {
  final InputFormatVertex vertex = new InputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSourceTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  config.setOutputSerializer(node.getSerializer());
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer

private InputFormatVertex createDataSourceVertex(SourcePlanNode node) throws CompilerException {
  final InputFormatVertex vertex = new InputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSourceTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  config.setOutputSerializer(node.getSerializer());
  return vertex;
}

代码示例来源：origin: com.alibaba.blink/flink-optimizer

private InputFormatVertex createDataSourceVertex(SourcePlanNode node) throws CompilerException {
  final InputFormatVertex vertex = new InputFormatVertex(node.getNodeName());
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass(DataSourceTask.class);
  vertex.setFormatDescription(getDescriptionForUserCode(node.getProgramOperator().getUserCodeWrapper()));
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  config.setOutputSerializer(node.getSerializer());
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer

private JobVertex createDualInputVertex(DualInputPlanNode node) throws CompilerException {
  final String taskName = node.getNodeName();
  final DriverStrategy ds = node.getDriverStrategy();
  final JobVertex vertex = new JobVertex(taskName);
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass( (this.currentIteration != null && node.isOnDynamicPath()) ? IterationIntermediateTask.class : BatchTask.class);
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  
  // set the driver strategy
  config.setDriver(ds.getDriverClass());
  config.setDriverStrategy(ds);
  if (node.getComparator1() != null) {
    config.setDriverComparator(node.getComparator1(), 0);
  }
  if (node.getComparator2() != null) {
    config.setDriverComparator(node.getComparator2(), 1);
  }
  if (node.getPairComparator() != null) {
    config.setDriverPairComparator(node.getPairComparator());
  }
  
  // assign memory, file-handles, etc.
  assignDriverResources(node, config);
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer_2.11

private JobVertex createDualInputVertex(DualInputPlanNode node) throws CompilerException {
  final String taskName = node.getNodeName();
  final DriverStrategy ds = node.getDriverStrategy();
  final JobVertex vertex = new JobVertex(taskName);
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass( (this.currentIteration != null && node.isOnDynamicPath()) ? IterationIntermediateTask.class : BatchTask.class);
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  
  // set the driver strategy
  config.setDriver(ds.getDriverClass());
  config.setDriverStrategy(ds);
  if (node.getComparator1() != null) {
    config.setDriverComparator(node.getComparator1(), 0);
  }
  if (node.getComparator2() != null) {
    config.setDriverComparator(node.getComparator2(), 1);
  }
  if (node.getPairComparator() != null) {
    config.setDriverPairComparator(node.getPairComparator());
  }
  
  // assign memory, file-handles, etc.
  assignDriverResources(node, config);
  return vertex;
}

代码示例来源：origin: com.alibaba.blink/flink-optimizer

private JobVertex createDualInputVertex(DualInputPlanNode node) throws CompilerException {
  final String taskName = node.getNodeName();
  final DriverStrategy ds = node.getDriverStrategy();
  final JobVertex vertex = new JobVertex(taskName);
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass( (this.currentIteration != null && node.isOnDynamicPath()) ? IterationIntermediateTask.class : BatchTask.class);
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  
  // set the driver strategy
  config.setDriver(ds.getDriverClass());
  config.setDriverStrategy(ds);
  if (node.getComparator1() != null) {
    config.setDriverComparator(node.getComparator1(), 0);
  }
  if (node.getComparator2() != null) {
    config.setDriverComparator(node.getComparator2(), 1);
  }
  if (node.getPairComparator() != null) {
    config.setDriverPairComparator(node.getPairComparator());
  }
  
  // assign memory, file-handles, etc.
  assignDriverResources(node, config);
  return vertex;
}

代码示例来源：origin: org.apache.flink/flink-optimizer_2.10

private JobVertex createDualInputVertex(DualInputPlanNode node) throws CompilerException {
  final String taskName = node.getNodeName();
  final DriverStrategy ds = node.getDriverStrategy();
  final JobVertex vertex = new JobVertex(taskName);
  final TaskConfig config = new TaskConfig(vertex.getConfiguration());
  vertex.setResources(node.getMinResources(), node.getPreferredResources());
  vertex.setInvokableClass( (this.currentIteration != null && node.isOnDynamicPath()) ? IterationIntermediateTask.class : BatchTask.class);
  
  // set user code
  config.setStubWrapper(node.getProgramOperator().getUserCodeWrapper());
  config.setStubParameters(node.getProgramOperator().getParameters());
  
  // set the driver strategy
  config.setDriver(ds.getDriverClass());
  config.setDriverStrategy(ds);
  if (node.getComparator1() != null) {
    config.setDriverComparator(node.getComparator1(), 0);
  }
  if (node.getComparator2() != null) {
    config.setDriverComparator(node.getComparator2(), 1);
  }
  if (node.getPairComparator() != null) {
    config.setDriverPairComparator(node.getPairComparator());
  }
  
  // assign memory, file-handles, etc.
  assignDriverResources(node, config);
  return vertex;
}

内容来源于网络，如有侵权，请联系作者删除！

Operator

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com