org.apache.flink.runtime.jobmaster

Class JobMaster

java.lang.Object

org.apache.flink.runtime.rpc.RpcEndpoint<JobMasterGateway>

org.apache.flink.runtime.jobmaster.JobMaster

public class JobMaster
extends RpcEndpoint<JobMasterGateway>

JobMaster implementation. The job master is responsible for the execution of a single JobGraph.

It offers the following methods as part of its rpc interface to interact with the JobMaster remotely:

• updateTaskExecutionState(java.util.UUID, org.apache.flink.runtime.taskmanager.TaskExecutionState) updates the task execution state for given task

Field Summary

Fields

Modifier and Type	Field and Description
static String	ARCHIVE_NAME
static String	JOB_MANAGER_NAME Default names for Flink's distributed components

Fields inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

log

Constructor Summary

Constructors

Constructor and Description

JobMaster(RpcService rpcService, ResourceID resourceId, JobGraph jobGraph, Configuration configuration, HighAvailabilityServices highAvailabilityService, HeartbeatServices heartbeatServices, ScheduledExecutorService executor, BlobLibraryCacheManager libraryCacheManager, RestartStrategyFactory restartStrategyFactory, Time rpcAskTimeout, JobManagerMetricGroup jobManagerMetricGroup, OnCompletionActions jobCompletionActions, FatalErrorHandler errorHandler, ClassLoader userCodeLoader)

Method Summary

Methods

Modifier and Type	Method and Description
void	acknowledgeCheckpoint(JobID jobID, ExecutionAttemptID executionAttemptID, long checkpointId, CheckpointMetrics checkpointMetrics, SubtaskState checkpointState)
void	declineCheckpoint(JobID jobID, ExecutionAttemptID executionAttemptID, long checkpointID, Throwable reason)
void	disconnectResourceManager(UUID jobManagerLeaderId, UUID resourceManagerLeaderId, Exception cause)
void	disconnectTaskManager(ResourceID resourceID, Exception cause)
void	failSlot(ResourceID taskManagerId, AllocationID allocationId, UUID leaderId, Exception cause)
void	heartbeatFromResourceManager(ResourceID resourceID)
void	heartbeatFromTaskManager(ResourceID resourceID)
KvStateLocation	lookupKvStateLocation(String registrationName)
void	notifyKvStateRegistered(JobVertexID jobVertexId, KeyGroupRange keyGroupRange, String registrationName, KvStateID kvStateId, KvStateServerAddress kvStateServerAddress)
void	notifyKvStateUnregistered(JobVertexID jobVertexId, KeyGroupRange keyGroupRange, String registrationName)
Future<Iterable<SlotOffer>>	offerSlots(ResourceID taskManagerId, Iterable<SlotOffer> slots, UUID leaderId)
Future<RegistrationResponse>	registerTaskManager(String taskManagerRpcAddress, TaskManagerLocation taskManagerLocation, UUID leaderId)
ClassloadingProps	requestClassloadingProps()
SerializedInputSplit	requestNextInputSplit(UUID leaderSessionID, JobVertexID vertexID, ExecutionAttemptID executionAttempt)
ExecutionState	requestPartitionState(UUID leaderSessionID, IntermediateDataSetID intermediateResultId, ResultPartitionID resultPartitionId)
Acknowledge	scheduleOrUpdateConsumers(UUID leaderSessionID, ResultPartitionID partitionID)
void	shutDown() Suspend the job and shutdown all other services including rpc.
void	start() Starts the rpc endpoint.
void	start(UUID leaderSessionID) Start the rpc service and begin to run the job.
void	startJobExecution()
void	suspendExecution(Throwable cause) Suspending job, all the running tasks will be cancelled, and communication with other components will be disposed.
Acknowledge	updateTaskExecutionState(UUID leaderSessionID, TaskExecutionState taskExecutionState) Updates the task execution state for a given task.

Methods inherited from class org.apache.flink.runtime.rpc.RpcEndpoint

callAsync, getAddress, getEndpointId, getMainThreadExecutor, getRpcService, getSelf, getSelfGatewayType, getTerminationFuture, runAsync, scheduleRunAsync, scheduleRunAsync, validateRunsInMainThread

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

JOB_MANAGER_NAME

public static final String JOB_MANAGER_NAME

Default names for Flink's distributed components

See Also:

Constant Field Values

ARCHIVE_NAME

public static final String ARCHIVE_NAME

See Also:

Constant Field Values

Constructor Detail

JobMaster

public JobMaster(RpcService rpcService,
         ResourceID resourceId,
         JobGraph jobGraph,
         Configuration configuration,
         HighAvailabilityServices highAvailabilityService,
         HeartbeatServices heartbeatServices,
         ScheduledExecutorService executor,
         BlobLibraryCacheManager libraryCacheManager,
         RestartStrategyFactory restartStrategyFactory,
         Time rpcAskTimeout,
         @Nullable
         JobManagerMetricGroup jobManagerMetricGroup,
         OnCompletionActions jobCompletionActions,
         FatalErrorHandler errorHandler,
         ClassLoader userCodeLoader)
          throws Exception

Throws:

Exception

Method Detail

start

public void start()

Description copied from class: RpcEndpoint

Starts the rpc endpoint. This tells the underlying rpc server that the rpc endpoint is ready to process remote procedure calls. IMPORTANT: Whenever you override this method, call the parent implementation to enable rpc processing. It is advised to make the parent call last.

Overrides:

start in class RpcEndpoint<JobMasterGateway>

start

public void start(UUID leaderSessionID)
           throws Exception

Start the rpc service and begin to run the job.

Parameters:

leaderSessionID - The necessary leader id for running the job.

Throws:

Exception

shutDown

public void shutDown()
              throws Exception

Suspend the job and shutdown all other services including rpc.

Overrides:

shutDown in class RpcEndpoint<JobMasterGateway>

Throws:

Exception - indicating that the something went wrong while shutting the RPC endpoint down

startJobExecution

public void startJobExecution()

suspendExecution

public void suspendExecution(Throwable cause)

Suspending job, all the running tasks will be cancelled, and communication with other components will be disposed.

Mostly job is suspended because of the leadership has been revoked, one can be restart this job by calling the start(UUID) method once we take the leadership back again.

Parameters:

cause - The reason of why this job been suspended.

updateTaskExecutionState

public Acknowledge updateTaskExecutionState(UUID leaderSessionID,
                                   TaskExecutionState taskExecutionState)
                                     throws Exception

Updates the task execution state for a given task.

Parameters:

taskExecutionState - New task execution state for a given task

Returns:

Acknowledge the task execution state update

Throws:

Exception

requestNextInputSplit

public SerializedInputSplit requestNextInputSplit(UUID leaderSessionID,
                                         JobVertexID vertexID,
                                         ExecutionAttemptID executionAttempt)
                                           throws Exception

Throws:

Exception

requestPartitionState

public ExecutionState requestPartitionState(UUID leaderSessionID,
                                   IntermediateDataSetID intermediateResultId,
                                   ResultPartitionID resultPartitionId)
                                     throws Exception

Throws:

Exception

scheduleOrUpdateConsumers

public Acknowledge scheduleOrUpdateConsumers(UUID leaderSessionID,
                                    ResultPartitionID partitionID)
                                      throws Exception

Throws:

Exception

disconnectTaskManager

public void disconnectTaskManager(ResourceID resourceID,
                         Exception cause)

acknowledgeCheckpoint

public void acknowledgeCheckpoint(JobID jobID,
                         ExecutionAttemptID executionAttemptID,
                         long checkpointId,
                         CheckpointMetrics checkpointMetrics,
                         SubtaskState checkpointState)
                           throws CheckpointException

Throws:

CheckpointException

declineCheckpoint

public void declineCheckpoint(JobID jobID,
                     ExecutionAttemptID executionAttemptID,
                     long checkpointID,
                     Throwable reason)

lookupKvStateLocation

public KvStateLocation lookupKvStateLocation(String registrationName)
                                      throws Exception

Throws:

Exception

notifyKvStateRegistered

public void notifyKvStateRegistered(JobVertexID jobVertexId,
                           KeyGroupRange keyGroupRange,
                           String registrationName,
                           KvStateID kvStateId,
                           KvStateServerAddress kvStateServerAddress)

notifyKvStateUnregistered

public void notifyKvStateUnregistered(JobVertexID jobVertexId,
                             KeyGroupRange keyGroupRange,
                             String registrationName)

requestClassloadingProps

public ClassloadingProps requestClassloadingProps()
                                           throws Exception

Throws:

Exception

offerSlots

public Future<Iterable<SlotOffer>> offerSlots(ResourceID taskManagerId,
                                     Iterable<SlotOffer> slots,
                                     UUID leaderId)
                                       throws Exception

Throws:

Exception

failSlot

public void failSlot(ResourceID taskManagerId,
            AllocationID allocationId,
            UUID leaderId,
            Exception cause)
              throws Exception

Throws:

Exception

registerTaskManager

public Future<RegistrationResponse> registerTaskManager(String taskManagerRpcAddress,
                                               TaskManagerLocation taskManagerLocation,
                                               UUID leaderId)
                                                 throws Exception

Throws:

Exception

disconnectResourceManager

public void disconnectResourceManager(UUID jobManagerLeaderId,
                             UUID resourceManagerLeaderId,
                             Exception cause)
                               throws Exception

Throws:

Exception

heartbeatFromTaskManager

public void heartbeatFromTaskManager(ResourceID resourceID)

heartbeatFromResourceManager

public void heartbeatFromResourceManager(ResourceID resourceID)