org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair

Class FSLeafQueue

java.lang.Object

org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue

org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue

All Implemented Interfaces:

Schedulable, Queue

@InterfaceAudience.Private
 @InterfaceStability.Unstable
public class FSLeafQueue
extends FSQueue

Field Summary

Fields inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue

parent, policy, recordFactory, scheduler

Constructor Summary

Constructors

Constructor and Description
FSLeafQueue(String name, FairScheduler scheduler, FSParentQueue parent)

Method Summary

Modifier and Type	Method and Description
void	addAMResourceUsage(org.apache.hadoop.yarn.api.records.Resource amResource)
void	addApp(FSAppAttempt app, boolean runnable)
org.apache.hadoop.yarn.api.records.Resource	assignContainer(FSSchedulerNode node) Assign a container on this node if possible, and return the amount of resources assigned.
boolean	canRunAppAM(org.apache.hadoop.yarn.api.records.Resource amResource) Check whether this queue can run this application master under the maxAMShare limit
void	clearPreemptedResources()
void	collectSchedulerApplications(Collection<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> apps) Adds all applications in the queue and its subqueues to the given collection.
ActiveUsersManager	getActiveUsersManager()
org.apache.hadoop.yarn.api.records.Resource	getAmResourceUsage()
List<FSQueue>	getChildQueues() Gets the children of this queue, if any.
List<FSAppAttempt>	getCopyOfNonRunnableAppSchedulables()
org.apache.hadoop.yarn.api.records.Resource	getDemand() Maximum number of resources required by this Schedulable.
long	getLastTimeAtFairShareThreshold()
long	getLastTimeAtMinShare()
int	getNumActiveApps() TODO: Based on how frequently this is called, we might want to club counting pending and active apps in the same method.
int	getNumNonRunnableApps()
int	getNumPendingApps()
int	getNumRunnableApps() Return the number of apps for which containers can be allocated.
List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo>	getQueueUserAclInfo(org.apache.hadoop.security.UserGroupInformation user) Get queue ACLs for given user.
org.apache.hadoop.yarn.api.records.Resource	getResourceUsage() Get the aggregate amount of resources consumed by the schedulable.
boolean	isNonRunnableApp(FSAppAttempt attempt)
boolean	isRunnableApp(FSAppAttempt attempt)
RMContainer	preemptContainer() Preempt a container from this Schedulable if possible.
void	recomputeShares() Recomputes the shares for all child queues and applications based on this queue's current share
void	recoverContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource, SchedulerApplicationAttempt schedulerAttempt, RMContainer rmContainer) Recover the state of the queue for a given container.
boolean	removeApp(FSAppAttempt app) Removes the given app from this queue.
boolean	removeNonRunnableApp(FSAppAttempt app) Removes the given app if it is non-runnable and belongs to this queue
void	resetPreemptedResources()
void	setPolicy(SchedulingPolicy policy)
void	setWeights(float weight) Allows setting weight for a dynamically created queue Currently only used for reservation based queues
void	updateDemand() Refresh the Schedulable's demand and those of its children if any.
void	updateStarvationStats() Update the preemption fields for the queue, i.e.

Methods inherited from class org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue

assignContainerPreCheck, getAccessibleNodeLabels, getDefaultNodeLabelExpression, getFairShare, getFairSharePreemptionThreshold, getFairSharePreemptionTimeout, getMaxShare, getMetrics, getMinShare, getMinSharePreemptionTimeout, getName, getParent, getPolicy, getPriority, getQueueInfo, getQueueName, getStartTime, getSteadyFairShare, getWeights, hasAccess, isActive, setFairShare, setFairSharePreemptionThreshold, setFairSharePreemptionTimeout, setMinSharePreemptionTimeout, setSteadyFairShare, throwPolicyDoesnotApplyException, toString, updatePreemptionVariables

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FSLeafQueue

public FSLeafQueue(String name,
                   FairScheduler scheduler,
                   FSParentQueue parent)

Method Detail

addApp

public void addApp(FSAppAttempt app,
                   boolean runnable)

removeApp

public boolean removeApp(FSAppAttempt app)

Removes the given app from this queue.

Returns:

whether or not the app was runnable

removeNonRunnableApp

public boolean removeNonRunnableApp(FSAppAttempt app)

Removes the given app if it is non-runnable and belongs to this queue

Returns:

true if the app is removed, false otherwise

isRunnableApp

public boolean isRunnableApp(FSAppAttempt attempt)

isNonRunnableApp

public boolean isNonRunnableApp(FSAppAttempt attempt)

resetPreemptedResources

public void resetPreemptedResources()

clearPreemptedResources

public void clearPreemptedResources()

getCopyOfNonRunnableAppSchedulables

public List<FSAppAttempt> getCopyOfNonRunnableAppSchedulables()

collectSchedulerApplications

public void collectSchedulerApplications(Collection<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> apps)

Description copied from class: FSQueue

Adds all applications in the queue and its subqueues to the given collection.

Specified by:

collectSchedulerApplications in class FSQueue

Parameters:

apps - the collection to add the applications to

setPolicy

public void setPolicy(SchedulingPolicy policy)
               throws AllocationConfigurationException

Specified by:

setPolicy in class FSQueue

Throws:

AllocationConfigurationException

recomputeShares

public void recomputeShares()

Description copied from class: FSQueue

Recomputes the shares for all child queues and applications based on this queue's current share

Specified by:

recomputeShares in class FSQueue

getDemand

public org.apache.hadoop.yarn.api.records.Resource getDemand()

Description copied from interface: Schedulable

Maximum number of resources required by this Schedulable. This is defined as number of currently utilized resources + number of unlaunched resources (that are either not yet launched or need to be speculated).

getResourceUsage

public org.apache.hadoop.yarn.api.records.Resource getResourceUsage()

Description copied from interface: Schedulable

Get the aggregate amount of resources consumed by the schedulable.

getAmResourceUsage

public org.apache.hadoop.yarn.api.records.Resource getAmResourceUsage()

updateDemand

public void updateDemand()

Description copied from interface: Schedulable

Refresh the Schedulable's demand and those of its children if any.

assignContainer

public org.apache.hadoop.yarn.api.records.Resource assignContainer(FSSchedulerNode node)

Description copied from interface: Schedulable

Assign a container on this node if possible, and return the amount of resources assigned.

preemptContainer

public RMContainer preemptContainer()

Description copied from interface: Schedulable

Preempt a container from this Schedulable if possible.

getChildQueues

public List<FSQueue> getChildQueues()

Description copied from class: FSQueue

Gets the children of this queue, if any.

Specified by:

getChildQueues in class FSQueue

getQueueUserAclInfo

public List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueUserAclInfo(org.apache.hadoop.security.UserGroupInformation user)

Description copied from interface: Queue

Get queue ACLs for given user.

Parameters:

user - username

Returns:

queue ACLs for user

getLastTimeAtMinShare

public long getLastTimeAtMinShare()

getLastTimeAtFairShareThreshold

public long getLastTimeAtFairShareThreshold()

getNumRunnableApps

public int getNumRunnableApps()

Description copied from class: FSQueue

Return the number of apps for which containers can be allocated. Includes apps in subqueues.

Specified by:

getNumRunnableApps in class FSQueue

getNumNonRunnableApps

public int getNumNonRunnableApps()

getNumPendingApps

public int getNumPendingApps()

getNumActiveApps

public int getNumActiveApps()

TODO: Based on how frequently this is called, we might want to club counting pending and active apps in the same method.

getActiveUsersManager

public ActiveUsersManager getActiveUsersManager()

canRunAppAM

public boolean canRunAppAM(org.apache.hadoop.yarn.api.records.Resource amResource)

Check whether this queue can run this application master under the maxAMShare limit

Parameters:

amResource -

Returns:

true if this queue can run

addAMResourceUsage

public void addAMResourceUsage(org.apache.hadoop.yarn.api.records.Resource amResource)

recoverContainer

public void recoverContainer(org.apache.hadoop.yarn.api.records.Resource clusterResource,
                             SchedulerApplicationAttempt schedulerAttempt,
                             RMContainer rmContainer)

Description copied from interface: Queue

Recover the state of the queue for a given container.

Parameters:

clusterResource - the resource of the cluster

schedulerAttempt - the application for which the container was allocated

rmContainer - the container that was recovered.

updateStarvationStats

public void updateStarvationStats()

Update the preemption fields for the queue, i.e. the times since last was at its guaranteed share and over its fair share threshold.

setWeights

public void setWeights(float weight)

Allows setting weight for a dynamically created queue Currently only used for reservation based queues

Parameters:

weight - queue weight