cascading.flow.Flow.getStats()方法的使用及代码示例

x33g5p2x  于2022-01-19 转载在 其他  
字(4.5k)|赞(0)|评价(0)|浏览(124)

本文整理了Java中cascading.flow.Flow.getStats()方法的一些代码示例,展示了Flow.getStats()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Flow.getStats()方法的具体详情如下:
包路径:cascading.flow.Flow
类名称:Flow
方法名:getStats

Flow.getStats介绍

暂无

代码示例

代码示例来源:origin: com.hotels/plunger

FlowStats getStats() {
 return flow == null ? null : flow.getStats();
}

代码示例来源:origin: cascading/lingual-core

@Override
public void onCompleted( Flow flow )
 {
 if( !flow.getStats().isSuccessful() )
  return;
 platformBroker.addResultToSchema( flow.getSink(), lingualConnection.get() );
 }

代码示例来源:origin: cwensel/cascading

@Test
public void testCombinedPartitionTap() throws Exception
 {
 getPlatform().copyFromLocal( inputFileLower );
 Tap source = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), " ", inputFileLower );
 Tap partitionTap = getPlatform().getDelimitedFile( new Fields( "lower" ), "+", getOutputPath( "/combinedpartition/partitioned" ), SinkMode.REPLACE );
 Partition partition = new DelimitedPartition( new Fields( "number" ) );
 partitionTap = getPlatform().getPartitionTap( partitionTap, partition, 1 );
 Flow firstFlow = getPlatform().getFlowConnector().connect( source, partitionTap, new Pipe( "partition" ) );
 firstFlow.complete();
 // Configure combine inputs for reading from the partition tap
 Map<Object, Object> properties = getProperties();
 HfsProps.setUseCombinedInput( properties, true );
 //set to lots of bytes so the test will combine all input files
 HfsProps.setCombinedInputMaxSize( properties, 100000000L );
 Tap sink = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), "+", getOutputPath( "/combinedpartition/final" ), SinkMode.REPLACE );
 Flow secondFlow = getPlatform().getFlowConnector( properties ).connect( partitionTap, sink, new Pipe( "copy" ) );
 secondFlow.complete();
 //Asserting we combined all partition files into one mapper
 if( getPlatform().isUseCluster() )
  assertEquals( 1, secondFlow.getStats().getCounterValue( JobInProgress.Counter.TOTAL_LAUNCHED_MAPS ) );
 List<Tuple> values = getSinkAsList( secondFlow );
 assertEquals( 5, values.size() );
 assertTrue( values.contains( new Tuple( "1", "a" ) ) );
 assertTrue( values.contains( new Tuple( "2", "b" ) ) );
 assertTrue( values.contains( new Tuple( "3", "c" ) ) );
 assertTrue( values.contains( new Tuple( "4", "d" ) ) );
 assertTrue( values.contains( new Tuple( "5", "e" ) ) );
 }

代码示例来源:origin: cascading/cascading-hadoop2-mr1

@Test
public void testCombinedPartitionTap() throws Exception
 {
 getPlatform().copyFromLocal( inputFileLower );
 Tap source = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), " ", inputFileLower );
 Tap partitionTap = getPlatform().getDelimitedFile( new Fields( "lower" ), "+", getOutputPath( "/combinedpartition/partitioned" ), SinkMode.REPLACE );
 Partition partition = new DelimitedPartition( new Fields( "number" ) );
 partitionTap = getPlatform().getPartitionTap( partitionTap, partition, 1 );
 Flow firstFlow = getPlatform().getFlowConnector().connect( source, partitionTap, new Pipe( "partition" ) );
 firstFlow.complete();
 // Configure combine inputs for reading from the partition tap
 Map<Object, Object> properties = getProperties();
 HfsProps.setUseCombinedInput( properties, true );
 //set to lots of bytes so the test will combine all input files
 HfsProps.setCombinedInputMaxSize( properties, 100000000L );
 Tap sink = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), "+", getOutputPath( "/combinedpartition/final" ), SinkMode.REPLACE );
 Flow secondFlow = getPlatform().getFlowConnector( properties ).connect( partitionTap, sink, new Pipe( "copy" ) );
 secondFlow.complete();
 //Asserting we combined all partition files into one mapper
 if( getPlatform().isUseCluster() )
  assertEquals( 1, secondFlow.getStats().getCounterValue( JobInProgress.Counter.TOTAL_LAUNCHED_MAPS ) );
 List<Tuple> values = getSinkAsList( secondFlow );
 assertEquals( 5, values.size() );
 assertTrue( values.contains( new Tuple( "1", "a" ) ) );
 assertTrue( values.contains( new Tuple( "2", "b" ) ) );
 assertTrue( values.contains( new Tuple( "3", "c" ) ) );
 assertTrue( values.contains( new Tuple( "4", "d" ) ) );
 assertTrue( values.contains( new Tuple( "5", "e" ) ) );
 }

代码示例来源:origin: cascading/cascading-platform

assertTrue( secondFlow.getStats().isSkipped() );
assertFalse( thirdFlow.getStats().isSkipped() );

代码示例来源:origin: cwensel/cascading

assertTrue( secondFlow.getStats().isSkipped() );
assertFalse( thirdFlow.getStats().isSkipped() );

相关文章