org.apache.flink.runtime.io.network.partition.consumer

Class SingleInputGate

java.lang.Object

org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate

All Implemented Interfaces:

InputGate

public class SingleInputGate
extends Object
implements InputGate

An input gate consumes one or more partitions of a single produced intermediate result.

Each intermediate result is partitioned over its producing parallel subtasks; each of these partitions is furthermore partitioned into one or more subpartitions.

As an example, consider a map-reduce program, where the map operator produces data and the reduce operator consumes the produced data.

 +-----+              +---------------------+              +--------+
 | Map | = produce => | Intermediate Result | <= consume = | Reduce |
 +-----+              +---------------------+              +--------+
 

When deploying such a program in parallel, the intermediate result will be partitioned over its producing parallel subtasks; each of these partitions is furthermore partitioned into one or more subpartitions.

                            Intermediate result
               +-----------------------------------------+
               |                      +----------------+ |              +-----------------------+
 +-------+     | +-------------+  +=> | Subpartition 1 | | <=======+=== | Input Gate | Reduce 1 |
 | Map 1 | ==> | | Partition 1 | =|   +----------------+ |         |    +-----------------------+
 +-------+     | +-------------+  +=> | Subpartition 2 | | <==+    |
               |                      +----------------+ |    |    | Subpartition request
               |                                         |    |    |
               |                      +----------------+ |    |    |
 +-------+     | +-------------+  +=> | Subpartition 1 | | <==+====+
 | Map 2 | ==> | | Partition 2 | =|   +----------------+ |    |         +-----------------------+
 +-------+     | +-------------+  +=> | Subpartition 2 | | <==+======== | Input Gate | Reduce 2 |
               |                      +----------------+ |              +-----------------------+
               +-----------------------------------------+
 

In the above example, two map subtasks produce the intermediate result in parallel, resulting in two partitions (Partition 1 and 2). Each of these partitions is further partitioned into two subpartitions -- one for each parallel reduce subtask.

Constructor Summary

Constructors

Constructor and Description
SingleInputGate(String owningTaskName, org.apache.flink.api.common.JobID jobId, ExecutionAttemptID executionId, IntermediateDataSetID consumedResultId, int consumedSubpartitionIndex, int numberOfInputChannels, PartitionStateChecker partitionStateChecker, IOMetricGroup metrics)

Method Summary

Modifier and Type	Method and Description
static SingleInputGate	create(String owningTaskName, org.apache.flink.api.common.JobID jobId, ExecutionAttemptID executionId, InputGateDeploymentDescriptor igdd, NetworkEnvironment networkEnvironment, IOMetricGroup metrics) Creates an input gate and all of its input channels.
IntermediateDataSetID	getConsumedResultId()
BufferOrEvent	getNextBufferOrEvent()
int	getNumberOfInputChannels()
int	getPageSize()
boolean	isFinished()
void	onAvailableBuffer(InputChannel channel)
void	registerListener(EventListener<InputGate> listener)
void	releaseAllResources()
void	requestPartitions()
void	retriggerPartitionRequest(IntermediateResultPartitionID partitionId) Retriggers a partition request.
void	sendTaskEvent(TaskEvent event)
void	setBufferPool(BufferPool bufferPool)
void	setInputChannel(IntermediateResultPartitionID partitionId, InputChannel inputChannel)
void	updateInputChannel(InputChannelDeploymentDescriptor icdd)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SingleInputGate

public SingleInputGate(String owningTaskName,
                       org.apache.flink.api.common.JobID jobId,
                       ExecutionAttemptID executionId,
                       IntermediateDataSetID consumedResultId,
                       int consumedSubpartitionIndex,
                       int numberOfInputChannels,
                       PartitionStateChecker partitionStateChecker,
                       IOMetricGroup metrics)

Method Detail

getNumberOfInputChannels

public int getNumberOfInputChannels()

Specified by:

getNumberOfInputChannels in interface InputGate

getConsumedResultId

public IntermediateDataSetID getConsumedResultId()

getPageSize

public int getPageSize()

Specified by:

getPageSize in interface InputGate

setBufferPool

public void setBufferPool(BufferPool bufferPool)

setInputChannel

public void setInputChannel(IntermediateResultPartitionID partitionId,
                            InputChannel inputChannel)

updateInputChannel

public void updateInputChannel(InputChannelDeploymentDescriptor icdd)
                        throws IOException,
                               InterruptedException

Throws:

IOException

InterruptedException

retriggerPartitionRequest

public void retriggerPartitionRequest(IntermediateResultPartitionID partitionId)
                               throws IOException,
                                      InterruptedException

Retriggers a partition request.

Throws:

IOException

InterruptedException

releaseAllResources

public void releaseAllResources()
                         throws IOException

Throws:

IOException

isFinished

public boolean isFinished()

Specified by:

isFinished in interface InputGate

requestPartitions

public void requestPartitions()
                       throws IOException,
                              InterruptedException

Specified by:

requestPartitions in interface InputGate

Throws:

IOException

InterruptedException

getNextBufferOrEvent

public BufferOrEvent getNextBufferOrEvent()
                                   throws IOException,
                                          InterruptedException

Specified by:

getNextBufferOrEvent in interface InputGate

Throws:

IOException

InterruptedException

sendTaskEvent

public void sendTaskEvent(TaskEvent event)
                   throws IOException

Specified by:

sendTaskEvent in interface InputGate

Throws:

IOException

registerListener

public void registerListener(EventListener<InputGate> listener)

Specified by:

registerListener in interface InputGate

onAvailableBuffer

public void onAvailableBuffer(InputChannel channel)

create

public static SingleInputGate create(String owningTaskName,
                                     org.apache.flink.api.common.JobID jobId,
                                     ExecutionAttemptID executionId,
                                     InputGateDeploymentDescriptor igdd,
                                     NetworkEnvironment networkEnvironment,
                                     IOMetricGroup metrics)

Creates an input gate and all of its input channels.