org.apache.flink.runtime.io.disk.iomanager

Class AsynchronousFileIOChannel<T,R extends org.apache.flink.runtime.io.disk.iomanager.IORequest>

java.lang.Object

org.apache.flink.runtime.io.disk.iomanager.AbstractFileIOChannel

org.apache.flink.runtime.io.disk.iomanager.AsynchronousFileIOChannel<T,R>

Type Parameters:

R - The type of request (e.g. ReadRequest or WriteRequest issued by this access to the I/O threads.

All Implemented Interfaces:

FileIOChannel

Direct Known Subclasses:

AsynchronousBlockReader, AsynchronousBlockWriterWithCallback, AsynchronousBufferFileReader, AsynchronousBufferFileSegmentReader, AsynchronousBufferFileWriter, AsynchronousBulkBlockReader

public abstract class AsynchronousFileIOChannel<T,R extends org.apache.flink.runtime.io.disk.iomanager.IORequest>
extends AbstractFileIOChannel

A base class for readers and writers that accept read or write requests for whole blocks. The request is delegated to an asynchronous I/O thread. After completion of the I/O request, the memory segment of the block is added to a collection to be returned.

The asynchrony of the access makes it possible to implement read-ahead or write-behind types of I/O accesses.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.disk.iomanager.FileIOChannel

FileIOChannel.Enumerator, FileIOChannel.ID

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	closed Flag marking this channel as closed
protected Object	closeLock The lock that is used during closing to synchronize the thread that waits for all requests to be handled with the asynchronous I/O thread.
protected IOException	exception An exception that was encountered by the asynchronous request handling thread.
protected RequestQueue<R>	requestQueue A request queue for submitting asynchronous requests to the corresponding IO worker thread.
protected AtomicInteger	requestsNotReturned An atomic integer that counts the number of requests that we still wait for to return.
protected RequestDoneCallback<T>	resultHandler Handler for completed requests

Fields inherited from class org.apache.flink.runtime.io.disk.iomanager.AbstractFileIOChannel

fileChannel, id, LOG

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AsynchronousFileIOChannel(FileIOChannel.ID channelID, RequestQueue<R> requestQueue, RequestDoneCallback<T> callback, boolean writeEnabled) Creates a new channel access to the path indicated by the given ID.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	addRequest(R request)
void	checkErroneous() Checks the exception state of this channel.
void	close() Closes the channel and waits until all pending asynchronous requests are processed.
void	closeAndDelete() This method waits for all pending asynchronous requests to return.
protected void	handleProcessedBuffer(T buffer, IOException ex) Handles a processed Buffer.
boolean	isClosed() Checks whether the channel has been closed.
protected boolean	registerAllRequestsProcessedListener(NotificationListener listener) Registers a listener to be notified when all outstanding requests have been processed.

Methods inherited from class org.apache.flink.runtime.io.disk.iomanager.AbstractFileIOChannel

deleteChannel, getChannelID, getNioFileChannel, getSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

closeLock

protected final Object closeLock

The lock that is used during closing to synchronize the thread that waits for all requests to be handled with the asynchronous I/O thread.

requestQueue

protected final RequestQueue<R extends org.apache.flink.runtime.io.disk.iomanager.IORequest> requestQueue

A request queue for submitting asynchronous requests to the corresponding IO worker thread.

requestsNotReturned

protected final AtomicInteger requestsNotReturned

An atomic integer that counts the number of requests that we still wait for to return.

resultHandler

protected final RequestDoneCallback<T> resultHandler

Handler for completed requests

exception

protected volatile IOException exception

An exception that was encountered by the asynchronous request handling thread.

closed

protected volatile boolean closed

Flag marking this channel as closed

Constructor Detail

AsynchronousFileIOChannel

protected AsynchronousFileIOChannel(FileIOChannel.ID channelID,
                         RequestQueue<R> requestQueue,
                         RequestDoneCallback<T> callback,
                         boolean writeEnabled)
                             throws IOException

Creates a new channel access to the path indicated by the given ID. The channel accepts buffers to be read/written and hands them to the asynchronous I/O thread. After being processed, the buffers are returned by adding the to the given queue.

Parameters:

channelID - The id describing the path of the file that the channel accessed.

requestQueue - The queue that this channel hands its IO requests to.

callback - The callback to be invoked when a request is done.

writeEnabled - Flag describing whether the channel should be opened in read/write mode, rather than in read-only mode.

Throws:

IOException - Thrown, if the channel could no be opened.

Method Detail

isClosed

public boolean isClosed()

Description copied from interface: FileIOChannel

Checks whether the channel has been closed.

Specified by:

isClosed in interface FileIOChannel

Specified by:

isClosed in class AbstractFileIOChannel

Returns:

True if the channel has been closed, false otherwise.

close

public void close()
           throws IOException

Closes the channel and waits until all pending asynchronous requests are processed. The underlying FileChannel is closed even if an exception interrupts the closing.

Important: the isClosed() method returns true immediately after this method has been called even when there are outstanding requests.

Specified by:

close in interface FileIOChannel

Specified by:

close in class AbstractFileIOChannel

Throws:

IOException - Thrown, if an I/O exception occurred while waiting for the buffers, or if the closing was interrupted.

closeAndDelete

public void closeAndDelete()
                    throws IOException

This method waits for all pending asynchronous requests to return. When the last request has returned, the channel is closed and deleted.

Even if an exception interrupts the closing, such that not all request are handled, the underlying FileChannel is closed and deleted.

Specified by:

closeAndDelete in interface FileIOChannel

Overrides:

closeAndDelete in class AbstractFileIOChannel

Throws:

IOException - Thrown, if an I/O exception occurred while waiting for the buffers, or if the closing was interrupted.

checkErroneous

public final void checkErroneous()
                          throws IOException

Checks the exception state of this channel. The channel is erroneous, if one of its requests could not be processed correctly.

Throws:

IOException - Thrown, if the channel is erroneous. The thrown exception contains the original exception that defined the erroneous state as its cause.

handleProcessedBuffer

protected final void handleProcessedBuffer(T buffer,
                         IOException ex)

Handles a processed Buffer. This method is invoked by the asynchronous IO worker threads upon completion of the IO request with the provided buffer and/or an exception that occurred while processing the request for that buffer.

Parameters:

buffer - The buffer to be processed.

ex - The exception that occurred in the I/O threads when processing the buffer's request.

addRequest

protected final void addRequest(R request)
                         throws IOException

Throws:

IOException

registerAllRequestsProcessedListener

protected boolean registerAllRequestsProcessedListener(NotificationListener listener)
                                                throws IOException

Registers a listener to be notified when all outstanding requests have been processed.

New requests can arrive right after the listener got notified. Therefore, it is not safe to assume that the number of outstanding requests is still zero after a notification unless there was a close right before the listener got called.

Returns true, if the registration was successful. A registration can fail, if there are no outstanding requests when trying to register a listener.

Throws:

IOException