org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair

Class FairScheduler

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>, Recoverable, ResourceScheduler, YarnScheduler

@InterfaceAudience.LimitedPrivate(value="yarn")
 @InterfaceStability.Unstable
public class FairScheduler
extends AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

A scheduler that schedules resources between a set of queues. The scheduler keeps track of the resources used by each queue, and attempts to maintain fairness by scheduling tasks at queues whose allocations are farthest below an ideal fair distribution. The fair scheduler supports hierarchical queues. All queues descend from a queue named "root". Available resources are distributed among the children of the root queue in the typical fair scheduling fashion. Then, the children distribute the resources assigned to them to their children in the same fashion. Applications may only be scheduled on leaf queues. Queues can be specified as children of other queues by placing them as sub-elements of their parents in the fair scheduler configuration file. A queue's name starts with the names of its parents, with periods as separators. So a queue named "queue1" under the root named, would be referred to as "root.queue1", and a queue named "queue2" under a queue named "parent1" would be referred to as "root.parent1.queue2".

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	assignMultiple
static org.apache.hadoop.yarn.api.records.Resource	CONTAINER_RESERVED
protected boolean	continuousSchedulingEnabled
protected int	continuousSchedulingSleepMs
protected long	lastPreemptionUpdateTime
protected int	maxAssign
protected long	nodeLocalityDelayMs
protected double	nodeLocalityThreshold
protected boolean	preemptionEnabled
protected long	preemptionInterval
protected float	preemptionUtilizationThreshold
protected long	rackLocalityDelayMs
protected double	rackLocalityThreshold
protected boolean	sizeBasedWeight
protected long	THREAD_JOIN_TIMEOUT_MS
protected long	updateInterval
protected long	waitTimeBeforeKill
protected WeightAdjuster	weightAdjuster

Fields inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler

applications, clusterResource, EMPTY_ALLOCATION, EMPTY_CONTAINER_LIST, minimumAllocation, nmExpireInterval, nodes, rmContext

Constructor Summary

Constructors

Constructor and Description
FairScheduler()

Method Summary

Modifier and Type	Method and Description
protected void	addApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId, String queueName, String user, boolean isAppRecovering) Add a new application to the scheduler, with a given id, queue name, and user.
protected void	addApplicationAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId, boolean transferStateFromPreviousAttempt, boolean isAttemptRecovering) Add a new application attempt to the scheduler.
Allocation	allocate(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId, List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask, List<org.apache.hadoop.yarn.api.records.ContainerId> release, List<String> blacklistAdditions, List<String> blacklistRemovals) The main api between the ApplicationMaster and the Scheduler.
boolean	checkAccess(org.apache.hadoop.security.UserGroupInformation callerUGI, org.apache.hadoop.yarn.api.records.QueueACL acl, String queueName) Check if the user has permission to perform the operation.
protected void	completedContainer(RMContainer rmContainer, org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus, RMContainerEventType event) Clean up a completed container.
AllocationConfiguration	getAllocationConfiguration()
List<org.apache.hadoop.yarn.api.records.ApplicationAttemptId>	getAppsInQueue(String queueName) Gets the apps under a given queue
ResourceWeights	getAppWeight(FSAppAttempt app)
org.apache.hadoop.yarn.util.Clock	getClock()
FairSchedulerConfiguration	getConf()
RMContainerTokenSecretManager	getContainerTokenSecretManager()
int	getContinuousSchedulingSleepMs()
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairSchedulerEventLog	getEventLog()
org.apache.hadoop.yarn.api.records.Resource	getIncrementResourceCapability()
long	getNodeLocalityDelayMs()
double	getNodeLocalityThreshold()
int	getNumClusterNodes() Get the number of nodes available in the cluster.
Set<String>	getPlanQueues() Gets the list of names for queues managed by the Reservation System
org.apache.hadoop.yarn.api.records.QueueInfo	getQueueInfo(String queueName, boolean includeChildQueues, boolean recursive) Get queue information
QueueManager	getQueueManager()
List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo>	getQueueUserAclInfo() Get acls for queues for current user.
long	getRackLocalityDelayMs()
double	getRackLocalityThreshold()
org.apache.hadoop.yarn.util.resource.ResourceCalculator	getResourceCalculator()
QueueMetrics	getRootQueueMetrics() Get the root queue for the scheduler.
FSAppAttempt	getSchedulerApp(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)
EnumSet<org.apache.hadoop.yarn.proto.YarnServiceProtos.SchedulerResourceTypes>	getSchedulingResourceTypes() Return a collection of the resource types that are considered when scheduling
void	handle(SchedulerEvent event)
boolean	isContinuousSchedulingEnabled()
String	moveApplication(org.apache.hadoop.yarn.api.records.ApplicationId appId, String queueName) Moves the given application to the given queue
protected void	preemptResources(org.apache.hadoop.yarn.api.records.Resource toPreempt) Preempt a quantity of resources.
protected void	preemptTasksIfNecessary() Check for queues that need tasks preempted, either because they have been below their guaranteed share for minSharePreemptionTimeout or they have been below their fair share threshold for the fairSharePreemptionTimeout.
void	recover(RMStateStore.RMState state)
void	reinitialize(org.apache.hadoop.conf.Configuration conf, RMContext rmContext) Re-initialize the ResourceScheduler.
void	removeQueue(String queueName) Only supports removing empty leaf queues
protected org.apache.hadoop.yarn.api.records.Resource	resToPreempt(FSLeafQueue sched, long curTime) Return the resource amount that this queue is allowed to preempt, if any.
void	serviceInit(org.apache.hadoop.conf.Configuration conf)
void	serviceStart()
void	serviceStop()
void	setEntitlement(String queueName, QueueEntitlement entitlement) This method increase the entitlement for current queue (must respect invariants, e.g., no overcommit of parents, non negative, etc.).
void	setRMContext(RMContext rmContext) Set RMContext for ResourceScheduler.
protected void	update() Recompute the internal variables used by the scheduler - per-job weights, fair shares, deficits, minimum slot allocations, and amount of used and required resources per job.
void	updateNodeResource(RMNode nm, org.apache.hadoop.yarn.api.records.ResourceOption resourceOption) Process resource update on a node and update Queue.
protected void	warnOrKillContainer(RMContainer container)

Methods inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler

addQueue, containerLaunchedOnNode, createReleaseCache, getApplicationAttempt, getAppResourceUsageReport, getClusterResource, getCurrentAttemptForContainer, getMaximumResourceCapability, getMaximumResourceCapability, getMinimumResourceCapability, getNodeReport, getPendingResourceRequestsForAttempt, getRMContainer, getSchedulerAppInfo, getSchedulerApplications, getSchedulerNode, getTransferredContainers, initMaximumResourceCapability, killAllAppsInQueue, moveAllApps, recoverContainersOnNode, recoverResourceRequestForContainer, refreshMaximumAllocation, releaseContainers, updateMaximumAllocation

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CONTAINER_RESERVED

public static final org.apache.hadoop.yarn.api.records.Resource CONTAINER_RESERVED

updateInterval

protected long updateInterval

THREAD_JOIN_TIMEOUT_MS

protected final long THREAD_JOIN_TIMEOUT_MS

See Also:

Constant Field Values

lastPreemptionUpdateTime

protected long lastPreemptionUpdateTime

preemptionEnabled

protected boolean preemptionEnabled

preemptionUtilizationThreshold

protected float preemptionUtilizationThreshold

preemptionInterval

protected long preemptionInterval

waitTimeBeforeKill

protected long waitTimeBeforeKill

sizeBasedWeight

protected boolean sizeBasedWeight

weightAdjuster

protected WeightAdjuster weightAdjuster

continuousSchedulingEnabled

protected boolean continuousSchedulingEnabled

continuousSchedulingSleepMs

protected int continuousSchedulingSleepMs

nodeLocalityThreshold

protected double nodeLocalityThreshold

rackLocalityThreshold

protected double rackLocalityThreshold

nodeLocalityDelayMs

protected long nodeLocalityDelayMs

rackLocalityDelayMs

protected long rackLocalityDelayMs

assignMultiple

protected boolean assignMultiple

maxAssign

protected int maxAssign

Constructor Detail

FairScheduler

public FairScheduler()

Method Detail

getConf

public FairSchedulerConfiguration getConf()

getQueueManager

public QueueManager getQueueManager()

update

protected void update()

Recompute the internal variables used by the scheduler - per-job weights, fair shares, deficits, minimum slot allocations, and amount of used and required resources per job.

preemptTasksIfNecessary

protected void preemptTasksIfNecessary()

Check for queues that need tasks preempted, either because they have been below their guaranteed share for minSharePreemptionTimeout or they have been below their fair share threshold for the fairSharePreemptionTimeout. If such queues exist, compute how many tasks of each type need to be preempted and then select the right ones using preemptTasks.

preemptResources

protected void preemptResources(org.apache.hadoop.yarn.api.records.Resource toPreempt)

Preempt a quantity of resources. Each round, we start from the root queue, level-by-level, until choosing a candidate application. The policy for prioritizing preemption for each queue depends on its SchedulingPolicy: (1) fairshare/DRF, choose the ChildSchedulable that is most over its fair share; (2) FIFO, choose the childSchedulable that is latest launched. Inside each application, we further prioritize preemption by choosing containers with lowest priority to preempt. We make sure that no queue is placed below its fair share in the process.

warnOrKillContainer

protected void warnOrKillContainer(RMContainer container)

resToPreempt

protected org.apache.hadoop.yarn.api.records.Resource resToPreempt(FSLeafQueue sched,
                                                                   long curTime)

Return the resource amount that this queue is allowed to preempt, if any. If the queue has been below its min share for at least its preemption timeout, it should preempt the difference between its current share and this min share. If it has been below its fair share preemption threshold for at least the fairSharePreemptionTimeout, it should preempt enough tasks to get up to its full fair share. If both conditions hold, we preempt the max of the two amounts (this shouldn't happen unless someone sets the timeouts to be identical for some reason).

getContainerTokenSecretManager

public RMContainerTokenSecretManager getContainerTokenSecretManager()

getAppWeight

public ResourceWeights getAppWeight(FSAppAttempt app)

getIncrementResourceCapability

public org.apache.hadoop.yarn.api.records.Resource getIncrementResourceCapability()

getNodeLocalityThreshold

public double getNodeLocalityThreshold()

getRackLocalityThreshold

public double getRackLocalityThreshold()

getNodeLocalityDelayMs

public long getNodeLocalityDelayMs()

getRackLocalityDelayMs

public long getRackLocalityDelayMs()

isContinuousSchedulingEnabled

public boolean isContinuousSchedulingEnabled()

getContinuousSchedulingSleepMs

public int getContinuousSchedulingSleepMs()

getClock

public org.apache.hadoop.yarn.util.Clock getClock()

getEventLog

public org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairSchedulerEventLog getEventLog()

addApplication

protected void addApplication(org.apache.hadoop.yarn.api.records.ApplicationId applicationId,
                              String queueName,
                              String user,
                              boolean isAppRecovering)

Add a new application to the scheduler, with a given id, queue name, and user. This will accept a new app even if the user or queue is above configured limits, but the app will not be marked as runnable.

addApplicationAttempt

protected void addApplicationAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId,
                                     boolean transferStateFromPreviousAttempt,
                                     boolean isAttemptRecovering)

Add a new application attempt to the scheduler.

completedContainer

protected void completedContainer(RMContainer rmContainer,
                                  org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus,
                                  RMContainerEventType event)

Clean up a completed container.

Specified by:

completedContainer in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

allocate

public Allocation allocate(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId,
                           List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask,
                           List<org.apache.hadoop.yarn.api.records.ContainerId> release,
                           List<String> blacklistAdditions,
                           List<String> blacklistRemovals)

Description copied from interface: YarnScheduler

The main api between the ApplicationMaster and the Scheduler. The ApplicationMaster is updating his future resource requirements and may release containers he doens't need.

Returns:

the Allocation for the application

getSchedulerApp

public FSAppAttempt getSchedulerApp(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

getResourceCalculator

public org.apache.hadoop.yarn.util.resource.ResourceCalculator getResourceCalculator()

getRootQueueMetrics

public QueueMetrics getRootQueueMetrics()

Description copied from interface: YarnScheduler

Get the root queue for the scheduler.

Returns:

the root queue for the scheduler.

handle

public void handle(SchedulerEvent event)

recover

public void recover(RMStateStore.RMState state)
             throws Exception

Throws:

Exception

setRMContext

public void setRMContext(RMContext rmContext)

Description copied from interface: ResourceScheduler

Set RMContext for ResourceScheduler. This method should be called immediately after instantiating a scheduler once.

Parameters:

rmContext - created by ResourceManager

serviceInit

public void serviceInit(org.apache.hadoop.conf.Configuration conf)
                 throws Exception

Overrides:

serviceInit in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

Throws:

Exception

serviceStart

public void serviceStart()
                  throws Exception

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

Exception

serviceStop

public void serviceStop()
                 throws Exception

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

reinitialize

public void reinitialize(org.apache.hadoop.conf.Configuration conf,
                         RMContext rmContext)
                  throws IOException

Description copied from interface: ResourceScheduler

Re-initialize the ResourceScheduler.

Parameters:

conf - configuration

Throws:

IOException

getQueueInfo

public org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(String queueName,
                                                                 boolean includeChildQueues,
                                                                 boolean recursive)
                                                          throws IOException

Description copied from interface: YarnScheduler

Get queue information

Parameters:

queueName - queue name

includeChildQueues - include child queues?

recursive - get children queues?

Returns:

queue information

Throws:

IOException

getQueueUserAclInfo

public List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueUserAclInfo()

Description copied from interface: YarnScheduler

Get acls for queues for current user.

Returns:

acls for queues for current user

getNumClusterNodes

public int getNumClusterNodes()

Description copied from interface: YarnScheduler

Get the number of nodes available in the cluster.

Returns:

the number of available nodes.

checkAccess

public boolean checkAccess(org.apache.hadoop.security.UserGroupInformation callerUGI,
                           org.apache.hadoop.yarn.api.records.QueueACL acl,
                           String queueName)

Description copied from interface: YarnScheduler

Check if the user has permission to perform the operation. If the user has QueueACL.ADMINISTER_QUEUE permission, this user can view/modify the applications in this queue

Returns:

true if the user has the permission, false otherwise

getAllocationConfiguration

public AllocationConfiguration getAllocationConfiguration()

getAppsInQueue

public List<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> getAppsInQueue(String queueName)

Description copied from interface: YarnScheduler

Gets the apps under a given queue

Parameters:

queueName - the name of the queue.

Returns:

a collection of app attempt ids in the given queue.

moveApplication

public String moveApplication(org.apache.hadoop.yarn.api.records.ApplicationId appId,
                              String queueName)
                       throws org.apache.hadoop.yarn.exceptions.YarnException

Description copied from interface: YarnScheduler

Moves the given application to the given queue

Specified by:

moveApplication in interface YarnScheduler

Overrides:

moveApplication in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

Returns:

the name of the queue the application was placed into

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if the move cannot be carried out

updateNodeResource

public void updateNodeResource(RMNode nm,
                               org.apache.hadoop.yarn.api.records.ResourceOption resourceOption)

Process resource update on a node and update Queue.

Overrides:

updateNodeResource in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

getSchedulingResourceTypes

public EnumSet<org.apache.hadoop.yarn.proto.YarnServiceProtos.SchedulerResourceTypes> getSchedulingResourceTypes()

Return a collection of the resource types that are considered when scheduling

Specified by:

getSchedulingResourceTypes in interface YarnScheduler

Overrides:

getSchedulingResourceTypes in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

Returns:

an EnumSet containing the resource types

getPlanQueues

public Set<String> getPlanQueues()
                          throws org.apache.hadoop.yarn.exceptions.YarnException

Description copied from interface: YarnScheduler

Gets the list of names for queues managed by the Reservation System

Specified by:

getPlanQueues in interface YarnScheduler

Overrides:

getPlanQueues in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

Returns:

the list of queues which support reservations

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

setEntitlement

public void setEntitlement(String queueName,
                           QueueEntitlement entitlement)
                    throws org.apache.hadoop.yarn.exceptions.YarnException

Description copied from interface: YarnScheduler

This method increase the entitlement for current queue (must respect invariants, e.g., no overcommit of parents, non negative, etc.). Entitlement is a general term for weights in FairScheduler, capacity for the CapacityScheduler, etc.

Specified by:

setEntitlement in interface YarnScheduler

Overrides:

setEntitlement in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

Parameters:

queueName - the queue for which we change entitlement

entitlement - the new entitlement for the queue (capacity, maxCapacity, etc..)

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

removeQueue

public void removeQueue(String queueName)
                 throws org.apache.hadoop.yarn.exceptions.YarnException

Only supports removing empty leaf queues

Specified by:

removeQueue in interface YarnScheduler

Overrides:

removeQueue in class AbstractYarnScheduler<FSAppAttempt,FSSchedulerNode>

Parameters:

queueName - name of queue to remove

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if queue to remove is either not a leaf or if its not empty