本文整理了Java中cascading.flow.Flow.getStats()
方法的一些代码示例,展示了Flow.getStats()
的具体用法。这些代码示例主要来源于Github
/Stackoverflow
/Maven
等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Flow.getStats()
方法的具体详情如下:
包路径:cascading.flow.Flow
类名称:Flow
方法名:getStats
暂无
代码示例来源:origin: com.hotels/plunger
FlowStats getStats() {
return flow == null ? null : flow.getStats();
}
代码示例来源:origin: cascading/lingual-core
@Override
public void onCompleted( Flow flow )
{
if( !flow.getStats().isSuccessful() )
return;
platformBroker.addResultToSchema( flow.getSink(), lingualConnection.get() );
}
代码示例来源:origin: cwensel/cascading
@Test
public void testCombinedPartitionTap() throws Exception
{
getPlatform().copyFromLocal( inputFileLower );
Tap source = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), " ", inputFileLower );
Tap partitionTap = getPlatform().getDelimitedFile( new Fields( "lower" ), "+", getOutputPath( "/combinedpartition/partitioned" ), SinkMode.REPLACE );
Partition partition = new DelimitedPartition( new Fields( "number" ) );
partitionTap = getPlatform().getPartitionTap( partitionTap, partition, 1 );
Flow firstFlow = getPlatform().getFlowConnector().connect( source, partitionTap, new Pipe( "partition" ) );
firstFlow.complete();
// Configure combine inputs for reading from the partition tap
Map<Object, Object> properties = getProperties();
HfsProps.setUseCombinedInput( properties, true );
//set to lots of bytes so the test will combine all input files
HfsProps.setCombinedInputMaxSize( properties, 100000000L );
Tap sink = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), "+", getOutputPath( "/combinedpartition/final" ), SinkMode.REPLACE );
Flow secondFlow = getPlatform().getFlowConnector( properties ).connect( partitionTap, sink, new Pipe( "copy" ) );
secondFlow.complete();
//Asserting we combined all partition files into one mapper
if( getPlatform().isUseCluster() )
assertEquals( 1, secondFlow.getStats().getCounterValue( JobInProgress.Counter.TOTAL_LAUNCHED_MAPS ) );
List<Tuple> values = getSinkAsList( secondFlow );
assertEquals( 5, values.size() );
assertTrue( values.contains( new Tuple( "1", "a" ) ) );
assertTrue( values.contains( new Tuple( "2", "b" ) ) );
assertTrue( values.contains( new Tuple( "3", "c" ) ) );
assertTrue( values.contains( new Tuple( "4", "d" ) ) );
assertTrue( values.contains( new Tuple( "5", "e" ) ) );
}
代码示例来源:origin: cascading/cascading-hadoop2-mr1
@Test
public void testCombinedPartitionTap() throws Exception
{
getPlatform().copyFromLocal( inputFileLower );
Tap source = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), " ", inputFileLower );
Tap partitionTap = getPlatform().getDelimitedFile( new Fields( "lower" ), "+", getOutputPath( "/combinedpartition/partitioned" ), SinkMode.REPLACE );
Partition partition = new DelimitedPartition( new Fields( "number" ) );
partitionTap = getPlatform().getPartitionTap( partitionTap, partition, 1 );
Flow firstFlow = getPlatform().getFlowConnector().connect( source, partitionTap, new Pipe( "partition" ) );
firstFlow.complete();
// Configure combine inputs for reading from the partition tap
Map<Object, Object> properties = getProperties();
HfsProps.setUseCombinedInput( properties, true );
//set to lots of bytes so the test will combine all input files
HfsProps.setCombinedInputMaxSize( properties, 100000000L );
Tap sink = getPlatform().getDelimitedFile( new Fields( "number", "lower" ), "+", getOutputPath( "/combinedpartition/final" ), SinkMode.REPLACE );
Flow secondFlow = getPlatform().getFlowConnector( properties ).connect( partitionTap, sink, new Pipe( "copy" ) );
secondFlow.complete();
//Asserting we combined all partition files into one mapper
if( getPlatform().isUseCluster() )
assertEquals( 1, secondFlow.getStats().getCounterValue( JobInProgress.Counter.TOTAL_LAUNCHED_MAPS ) );
List<Tuple> values = getSinkAsList( secondFlow );
assertEquals( 5, values.size() );
assertTrue( values.contains( new Tuple( "1", "a" ) ) );
assertTrue( values.contains( new Tuple( "2", "b" ) ) );
assertTrue( values.contains( new Tuple( "3", "c" ) ) );
assertTrue( values.contains( new Tuple( "4", "d" ) ) );
assertTrue( values.contains( new Tuple( "5", "e" ) ) );
}
代码示例来源:origin: cascading/cascading-platform
assertTrue( secondFlow.getStats().isSkipped() );
assertFalse( thirdFlow.getStats().isSkipped() );
代码示例来源:origin: cwensel/cascading
assertTrue( secondFlow.getStats().isSkipped() );
assertFalse( thirdFlow.getStats().isSkipped() );
内容来源于网络,如有侵权,请联系作者删除!