| Modifier and Type | Method and Description |
|---|---|
IOManager |
Environment.getIOManager()
Returns the current
IOManager. |
| Constructor and Description |
|---|
ChannelReaderInputViewIterator(IOManager ioAccess,
FileIOChannel.ID channel,
LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment> returnQueue,
List<org.apache.flink.core.memory.MemorySegment> segments,
List<org.apache.flink.core.memory.MemorySegment> freeMemTarget,
org.apache.flink.api.common.typeutils.TypeSerializer<E> accessors,
int numBlocks) |
ChannelReaderInputViewIterator(IOManager ioAccess,
FileIOChannel.ID channel,
List<org.apache.flink.core.memory.MemorySegment> segments,
List<org.apache.flink.core.memory.MemorySegment> freeMemTarget,
org.apache.flink.api.common.typeutils.TypeSerializer<E> accessors,
int numBlocks) |
SeekableFileChannelInputView(IOManager ioManager,
FileIOChannel.ID channelId,
MemoryManager memManager,
List<org.apache.flink.core.memory.MemorySegment> memory,
int sizeOfLastBlock) |
SpillingBuffer(IOManager ioManager,
org.apache.flink.core.memory.MemorySegmentSource memSource,
int segmentSize) |
| Modifier and Type | Class and Description |
|---|---|
class |
IOManagerAsync
A version of the
IOManager that uses asynchronous I/O. |
| Constructor and Description |
|---|
ResultPartition(String owningTaskName,
org.apache.flink.api.common.JobID jobId,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
boolean eagerlyDeployConsumers,
int numberOfSubpartitions,
ResultPartitionManager partitionManager,
ResultPartitionConsumableNotifier partitionConsumableNotifier,
IOManager ioManager,
IOManager.IOMode defaultIoMode) |
| Constructor and Description |
|---|
SerializedUpdateBuffer(List<org.apache.flink.core.memory.MemorySegment> memSegments,
int segmentSize,
IOManager ioManager) |
| Modifier and Type | Method and Description |
|---|---|
IOManager |
BatchTask.getIOManager() |
IOManager |
TaskContext.getIOManager() |
| Modifier and Type | Method and Description |
|---|---|
protected JoinTaskIterator<IT1,IT2,OT> |
RightOuterJoinDriver.getNonReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
protected JoinTaskIterator<IT1,IT2,OT> |
FullOuterJoinDriver.getNonReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
protected JoinTaskIterator<IT1,IT2,OT> |
LeftOuterJoinDriver.getNonReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
protected abstract JoinTaskIterator<IT1,IT2,OT> |
AbstractOuterJoinDriver.getNonReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
protected JoinTaskIterator<IT1,IT2,OT> |
RightOuterJoinDriver.getReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
protected JoinTaskIterator<IT1,IT2,OT> |
FullOuterJoinDriver.getReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
protected JoinTaskIterator<IT1,IT2,OT> |
LeftOuterJoinDriver.getReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
protected abstract JoinTaskIterator<IT1,IT2,OT> |
AbstractOuterJoinDriver.getReusingOuterJoinIterator(DriverStrategy driverStrategy,
org.apache.flink.util.MutableObjectIterator<IT1> in1,
org.apache.flink.util.MutableObjectIterator<IT2> in2,
org.apache.flink.api.common.typeutils.TypeSerializer<IT1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<IT1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<IT2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<IT2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparatorFactory<IT1,IT2> pairComparatorFactory,
MemoryManager memoryManager,
IOManager ioManager,
double driverMemFraction) |
| Constructor and Description |
|---|
TempBarrier(AbstractInvokable owner,
org.apache.flink.util.MutableObjectIterator<T> input,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<T> serializerFactory,
MemoryManager memManager,
IOManager ioManager,
int numPages) |
| Modifier and Type | Field and Description |
|---|---|
protected IOManager |
MutableHashTable.ioManager
The I/O manager used to instantiate writers for the spilled partitions.
|
| Modifier and Type | Method and Description |
|---|---|
void |
HashPartition.finalizeBuildPhase(IOManager ioAccess,
FileIOChannel.Enumerator probeChannelEnumerator,
LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment> bufferReturnQueue) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
HashJoinIteratorBase.getHashJoin(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBloomFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildSecondReOpenableHashJoinIterator.getHashJoin(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
ReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
<BT,PT> MutableHashTable<BT,PT> |
NonReusingBuildFirstReOpenableHashJoinIterator.getHashJoin(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean useBitmapFilters) |
void |
HashPartition.prepareProbePhase(IOManager ioAccess,
FileIOChannel.Enumerator probeChannelEnumerator,
LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment> bufferReturnQueue) |
int |
HashPartition.spillPartition(List<org.apache.flink.core.memory.MemorySegment> target,
IOManager ioAccess,
FileIOChannel.ID targetChannel,
LinkedBlockingQueue<org.apache.flink.core.memory.MemorySegment> bufferReturnQueue)
Spills this partition to disk and sets it up such that it continues spilling records that are added to
it.
|
| Constructor and Description |
|---|
MutableHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> comparator,
List<org.apache.flink.core.memory.MemorySegment> memorySegments,
IOManager ioManager) |
MutableHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> comparator,
List<org.apache.flink.core.memory.MemorySegment> memorySegments,
IOManager ioManager,
boolean useBloomFilters) |
MutableHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> comparator,
List<org.apache.flink.core.memory.MemorySegment> memorySegments,
IOManager ioManager,
int avgRecordLen,
boolean useBloomFilters) |
NonReusingBuildFirstHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildFirstReOpenableHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
NonReusingBuildSecondReOpenableHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReOpenableMutableHashTable(org.apache.flink.api.common.typeutils.TypeSerializer<BT> buildSideSerializer,
org.apache.flink.api.common.typeutils.TypeSerializer<PT> probeSideSerializer,
org.apache.flink.api.common.typeutils.TypeComparator<BT> buildSideComparator,
org.apache.flink.api.common.typeutils.TypeComparator<PT> probeSideComparator,
org.apache.flink.api.common.typeutils.TypePairComparator<PT,BT> comparator,
List<org.apache.flink.core.memory.MemorySegment> memorySegments,
IOManager ioManager,
boolean useBitmapFilters) |
ReusingBuildFirstHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildFirstReOpenableHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V2,V1> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
ReusingBuildSecondReOpenableHashJoinIterator(org.apache.flink.util.MutableObjectIterator<V1> firstInput,
org.apache.flink.util.MutableObjectIterator<V2> secondInput,
org.apache.flink.api.common.typeutils.TypeSerializer<V1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<V1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<V2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<V2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<V1,V2> pairComparator,
MemoryManager memManager,
IOManager ioManager,
AbstractInvokable ownerTask,
double memoryFraction,
boolean probeSideOuterJoin,
boolean buildSideOuterJoin,
boolean useBitmapFilters) |
| Constructor and Description |
|---|
SpillingResettableIterator(Iterator<T> input,
org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
int numPages,
AbstractInvokable parentTask) |
SpillingResettableIterator(Iterator<T> input,
org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
List<org.apache.flink.core.memory.MemorySegment> memory) |
SpillingResettableMutableObjectIterator(org.apache.flink.util.MutableObjectIterator<T> input,
org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
int numPages,
AbstractInvokable parentTask) |
SpillingResettableMutableObjectIterator(org.apache.flink.util.MutableObjectIterator<T> input,
org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer,
MemoryManager memoryManager,
IOManager ioManager,
List<org.apache.flink.core.memory.MemorySegment> memory) |
| Modifier and Type | Field and Description |
|---|---|
protected IOManager |
UnilateralSortMerger.SpillingThread.ioManager |
| Modifier and Type | Method and Description |
|---|---|
protected UnilateralSortMerger.ThreadBase<E> |
CombiningUnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
List<org.apache.flink.core.memory.MemorySegment> sortReadMemory,
List<org.apache.flink.core.memory.MemorySegment> writeMemory,
int maxFileHandles) |
protected UnilateralSortMerger.ThreadBase<E> |
UnilateralSortMerger.getSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memoryManager,
IOManager ioManager,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
List<org.apache.flink.core.memory.MemorySegment> sortReadMemory,
List<org.apache.flink.core.memory.MemorySegment> writeMemory,
int maxFileHandles) |
| Constructor and Description |
|---|
AbstractMergeInnerJoinIterator(org.apache.flink.util.MutableObjectIterator<T1> input1,
org.apache.flink.util.MutableObjectIterator<T2> input2,
org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
AbstractMergeIterator(org.apache.flink.util.MutableObjectIterator<T1> input1,
org.apache.flink.util.MutableObjectIterator<T2> input2,
org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
AbstractMergeOuterJoinIterator(org.apache.flink.api.common.operators.base.OuterJoinOperatorBase.OuterJoinType outerJoinType,
org.apache.flink.util.MutableObjectIterator<T1> input1,
org.apache.flink.util.MutableObjectIterator<T2> input2,
org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
CombiningSpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
List<org.apache.flink.core.memory.MemorySegment> sortReadMemory,
List<org.apache.flink.core.memory.MemorySegment> writeMemory,
int maxNumFileHandles) |
CombiningUnilateralSortMerger(org.apache.flink.api.common.functions.GroupCombineFunction<E,E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
org.apache.flink.util.MutableObjectIterator<E> input,
AbstractInvokable parentTask,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
CombiningUnilateralSortMerger(org.apache.flink.api.common.functions.GroupCombineFunction<E,E> combineStub,
MemoryManager memoryManager,
IOManager ioManager,
org.apache.flink.util.MutableObjectIterator<E> input,
AbstractInvokable parentTask,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled)
Creates a new sorter that reads the data from a given reader and provides an iterator returning that
data in a sorted manner.
|
LargeRecordHandler(org.apache.flink.api.common.typeutils.TypeSerializer<T> serializer,
org.apache.flink.api.common.typeutils.TypeComparator<T> comparator,
IOManager ioManager,
MemoryManager memManager,
List<org.apache.flink.core.memory.MemorySegment> memory,
AbstractInvokable memoryOwner,
int maxFilehandles) |
NonReusingMergeInnerJoinIterator(org.apache.flink.util.MutableObjectIterator<T1> input1,
org.apache.flink.util.MutableObjectIterator<T2> input2,
org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
NonReusingMergeOuterJoinIterator(org.apache.flink.api.common.operators.base.OuterJoinOperatorBase.OuterJoinType outerJoinType,
org.apache.flink.util.MutableObjectIterator<T1> input1,
org.apache.flink.util.MutableObjectIterator<T2> input2,
org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
ReusingMergeInnerJoinIterator(org.apache.flink.util.MutableObjectIterator<T1> input1,
org.apache.flink.util.MutableObjectIterator<T2> input2,
org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
ReusingMergeOuterJoinIterator(org.apache.flink.api.common.operators.base.OuterJoinOperatorBase.OuterJoinType outerJoinType,
org.apache.flink.util.MutableObjectIterator<T1> input1,
org.apache.flink.util.MutableObjectIterator<T2> input2,
org.apache.flink.api.common.typeutils.TypeSerializer<T1> serializer1,
org.apache.flink.api.common.typeutils.TypeComparator<T1> comparator1,
org.apache.flink.api.common.typeutils.TypeSerializer<T2> serializer2,
org.apache.flink.api.common.typeutils.TypeComparator<T2> comparator2,
org.apache.flink.api.common.typeutils.TypePairComparator<T1,T2> pairComparator,
MemoryManager memoryManager,
IOManager ioManager,
int numMemoryPages,
AbstractInvokable parentTask) |
SpillingThread(ExceptionHandler<IOException> exceptionHandler,
UnilateralSortMerger.CircularQueues<E> queues,
AbstractInvokable parentTask,
MemoryManager memManager,
IOManager ioManager,
org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
List<org.apache.flink.core.memory.MemorySegment> sortReadMemory,
List<org.apache.flink.core.memory.MemorySegment> writeMemory,
int maxNumFileHandles)
Creates the spilling thread.
|
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
org.apache.flink.util.MutableObjectIterator<E> input,
AbstractInvokable parentTask,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
double memoryFraction,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
org.apache.flink.util.MutableObjectIterator<E> input,
AbstractInvokable parentTask,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
IOManager ioManager,
org.apache.flink.util.MutableObjectIterator<E> input,
AbstractInvokable parentTask,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
double memoryFraction,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
List<org.apache.flink.core.memory.MemorySegment> memory,
IOManager ioManager,
org.apache.flink.util.MutableObjectIterator<E> input,
AbstractInvokable parentTask,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
UnilateralSortMerger(MemoryManager memoryManager,
List<org.apache.flink.core.memory.MemorySegment> memory,
IOManager ioManager,
org.apache.flink.util.MutableObjectIterator<E> input,
AbstractInvokable parentTask,
org.apache.flink.api.common.typeutils.TypeSerializerFactory<E> serializerFactory,
org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
int numSortBuffers,
int maxNumFileHandles,
float startSpillingFraction,
boolean noSpillingMemory,
boolean handleLargeRecords,
boolean objectReuseEnabled) |
| Modifier and Type | Method and Description |
|---|---|
IOManager |
RuntimeEnvironment.getIOManager() |
| Constructor and Description |
|---|
RuntimeEnvironment(org.apache.flink.api.common.JobID jobId,
JobVertexID jobVertexId,
ExecutionAttemptID executionId,
org.apache.flink.api.common.ExecutionConfig executionConfig,
org.apache.flink.api.common.TaskInfo taskInfo,
org.apache.flink.configuration.Configuration jobConfiguration,
org.apache.flink.configuration.Configuration taskConfiguration,
ClassLoader userCodeClassLoader,
MemoryManager memManager,
IOManager ioManager,
BroadcastVariableManager bcVarManager,
AccumulatorRegistry accumulatorRegistry,
InputSplitProvider splitProvider,
Map<String,Future<org.apache.flink.core.fs.Path>> distCacheEntries,
ResultPartitionWriter[] writers,
InputGate[] inputGates,
ActorGateway jobManager,
TaskManagerRuntimeInfo taskManagerInfo,
TaskMetricGroup metrics,
Task containingTask) |
Task(TaskDeploymentDescriptor tdd,
MemoryManager memManager,
IOManager ioManager,
NetworkEnvironment networkEnvironment,
BroadcastVariableManager bcVarManager,
ActorGateway taskManagerActor,
ActorGateway jobManagerActor,
scala.concurrent.duration.FiniteDuration actorAskTimeout,
LibraryCacheManager libraryCache,
FileCache fileCache,
TaskManagerRuntimeInfo taskManagerConfig,
TaskMetricGroup metricGroup)
IMPORTANT: This constructor may not start any work that would need to
be undone in the case of a failing task deployment.
|
Copyright © 2014–2016 The Apache Software Foundation. All rights reserved.