org.apache.hadoop.yarn.server.nodemanager

Class WindowsSecureContainerExecutor

java.lang.Object

org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

org.apache.hadoop.yarn.server.nodemanager.WindowsSecureContainerExecutor

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class WindowsSecureContainerExecutor
extends DefaultContainerExecutor

Windows secure container executor (WSCE). This class offers a secure container executor on Windows, similar to the LinuxContainerExecutor. As the NM does not run on a high privileged context, this class delegates elevated operations to the helper hadoopwintuilsvc, implemented by the winutils.exe running as a service. JNI and LRPC is used to communicate with the privileged service.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

DefaultContainerExecutor.LocalWrapperScriptBuilder

Nested classes/interfaces inherited from class org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

ContainerExecutor.DelayedProcessKiller, ContainerExecutor.ExitCode, ContainerExecutor.Signal

Field Summary

Fields

Modifier and Type	Field and Description
static String	LOCALIZER_PID_FORMAT

Fields inherited from class org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

lfs

Fields inherited from class org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

TASK_LAUNCH_SCRIPT_PERMISSION

Constructor Summary

Constructors

Constructor and Description
WindowsSecureContainerExecutor()

Method Summary

Modifier and Type	Method and Description
protected org.apache.hadoop.util.Shell.CommandExecutor	buildCommandExecutor(String wrapperScriptPath, String containerIdStr, String userName, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.yarn.api.records.Resource resource, File wordDir, Map<String,String> environment)
protected void	copyFile(org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst, String owner)
protected void	createDir(org.apache.hadoop.fs.Path dirPath, org.apache.hadoop.fs.permission.FsPermission perms, boolean createParent, String owner)
protected DefaultContainerExecutor.LocalWrapperScriptBuilder	getLocalWrapperScriptBuilder(String containerIdStr, org.apache.hadoop.fs.Path containerWorkDir)
protected String[]	getRunCommand(String command, String groupId, String userName, org.apache.hadoop.fs.Path pidFile, org.apache.hadoop.conf.Configuration conf)
protected void	killContainer(String pid, ContainerExecutor.Signal signal) Send a specified signal to the specified pid
org.apache.hadoop.fs.Path	localizeClasspathJar(org.apache.hadoop.fs.Path classPathJar, org.apache.hadoop.fs.Path pwd, String owner) On Windows the ContainerLaunch creates a temporary special jar manifest of other jars to workaround the CLASSPATH length.
void	setConf(org.apache.hadoop.conf.Configuration conf)
protected void	setScriptExecutable(org.apache.hadoop.fs.Path script, String owner)
void	startLocalizer(org.apache.hadoop.fs.Path nmPrivateContainerTokens, InetSocketAddress nmAddr, String user, String appId, String locId, LocalDirsHandlerService dirsHandler) Prepare the environment for containers in this application to execute.

Methods inherited from class org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor

containerIsAlive, deleteAsUser, getWorkingDir, init, isContainerProcessAlive, launchContainer, signalContainer

Methods inherited from class org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

activateContainer, deactivateContainer, getConf, getPidFilePath, getProcessId, getRunCommand, isContainerActive, logOutput, reacquireContainer, writeLaunchEnv

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOCALIZER_PID_FORMAT

public static final String LOCALIZER_PID_FORMAT

See Also:

Constant Field Values

Constructor Detail

WindowsSecureContainerExecutor

public WindowsSecureContainerExecutor()
                               throws IOException,
                                      URISyntaxException

Throws:

IOException

URISyntaxException

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

Overrides:

setConf in class ContainerExecutor

getRunCommand

protected String[] getRunCommand(String command,
                                 String groupId,
                                 String userName,
                                 org.apache.hadoop.fs.Path pidFile,
                                 org.apache.hadoop.conf.Configuration conf)

Overrides:

getRunCommand in class ContainerExecutor

getLocalWrapperScriptBuilder

protected DefaultContainerExecutor.LocalWrapperScriptBuilder getLocalWrapperScriptBuilder(String containerIdStr,
                                                                                          org.apache.hadoop.fs.Path containerWorkDir)

Overrides:

getLocalWrapperScriptBuilder in class DefaultContainerExecutor

copyFile

protected void copyFile(org.apache.hadoop.fs.Path src,
                        org.apache.hadoop.fs.Path dst,
                        String owner)
                 throws IOException

Overrides:

copyFile in class DefaultContainerExecutor

Throws:

IOException

createDir

protected void createDir(org.apache.hadoop.fs.Path dirPath,
                         org.apache.hadoop.fs.permission.FsPermission perms,
                         boolean createParent,
                         String owner)
                  throws IOException

Overrides:

createDir in class DefaultContainerExecutor

Throws:

IOException

setScriptExecutable

protected void setScriptExecutable(org.apache.hadoop.fs.Path script,
                                   String owner)
                            throws IOException

Overrides:

setScriptExecutable in class DefaultContainerExecutor

Throws:

IOException

localizeClasspathJar

public org.apache.hadoop.fs.Path localizeClasspathJar(org.apache.hadoop.fs.Path classPathJar,
                                                      org.apache.hadoop.fs.Path pwd,
                                                      String owner)
                                               throws IOException

Description copied from class: ContainerExecutor

On Windows the ContainerLaunch creates a temporary special jar manifest of other jars to workaround the CLASSPATH length. In a secure cluster this jar must be localized so that the container has access to it. This function localizes on-demand the jar.

Overrides:

localizeClasspathJar in class ContainerExecutor

Throws:

IOException

startLocalizer

public void startLocalizer(org.apache.hadoop.fs.Path nmPrivateContainerTokens,
                           InetSocketAddress nmAddr,
                           String user,
                           String appId,
                           String locId,
                           LocalDirsHandlerService dirsHandler)
                    throws IOException,
                           InterruptedException

Description copied from class: ContainerExecutor

Prepare the environment for containers in this application to execute.

 For $x in local.dirs
   create $x/$user/$appId
 Copy $nmLocal/appTokens  -> $N/$user/$appId
 For $rsrc in private resources
   Copy $rsrc  -> $N/$user/filecache/[idef]
 For $rsrc in job resources
   Copy $rsrc  -> $N/$user/$appId/filecache/idef
 

Overrides:

startLocalizer in class DefaultContainerExecutor

Parameters:

nmPrivateContainerTokens - path to localized credentials, rsrc by NM

nmAddr - RPC address to contact NM

user - user name of application owner

appId - id of the application

dirsHandler - NM local dirs service, for nm-local-dirs and nm-log-dirs

Throws:

IOException - For most application init failures

InterruptedException - If application init thread is halted by NM

buildCommandExecutor

protected org.apache.hadoop.util.Shell.CommandExecutor buildCommandExecutor(String wrapperScriptPath,
                                                                            String containerIdStr,
                                                                            String userName,
                                                                            org.apache.hadoop.fs.Path pidFile,
                                                                            org.apache.hadoop.yarn.api.records.Resource resource,
                                                                            File wordDir,
                                                                            Map<String,String> environment)
                                                                     throws IOException

Overrides:

buildCommandExecutor in class DefaultContainerExecutor

Throws:

IOException

killContainer

protected void killContainer(String pid,
                             ContainerExecutor.Signal signal)
                      throws IOException

Description copied from class: DefaultContainerExecutor

Send a specified signal to the specified pid

Overrides:

killContainer in class DefaultContainerExecutor

Parameters:

pid - the pid of the process [group] to signal.

signal - signal to send (for logging).

Throws:

IOException