org.deeplearning4j.parallelism

Class ParallelWrapper

java.lang.Object

org.deeplearning4j.parallelism.ParallelWrapper

All Implemented Interfaces:

AutoCloseable

public class ParallelWrapper
extends Object
implements AutoCloseable

This is simple data-parallel wrapper suitable for multi-cpu/multi-gpu environments. PLEASE NOTE: This implementation is NOT NUMA-aware.

Author:

raver119@gmail.com

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParallelWrapper.Builder<T extends org.deeplearning4j.nn.api.Model>
static class	ParallelWrapper.TrainingMode

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	averageUpdaters
protected int	averagingFrequency
protected boolean	debug
protected ThreadPoolExecutor	executorService
protected org.deeplearning4j.optimize.solvers.accumulation.GradientsAccumulator	gradientsAccumulator
protected boolean	isMQ
protected AtomicLong	iterationsCounter
protected boolean	legacyAveraging
protected List<org.deeplearning4j.optimize.api.IterationListener>	listeners
protected org.deeplearning4j.nn.api.Model	model
protected int	prefetchSize
protected boolean	reportScore
protected AtomicBoolean	stopFit
protected StatsStorageRouter	storageRouter
protected TrainerContext	trainerContext
protected Object[]	trainerContextArgs
protected boolean	wasAveraged
protected AtomicInteger	workerCounter
protected int	workers
protected org.deeplearning4j.nn.conf.WorkspaceMode	workspaceMode
protected Trainer[]	zoo

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ParallelWrapper(org.deeplearning4j.nn.api.Model model, int workers, int prefetchSize)

Method Summary

Modifier and Type	Method and Description
void	broadcastGradients(org.deeplearning4j.optimize.listeners.SharedGradient gradients) This method will propagate gradients across all workers
void	close()
void	fit(org.nd4j.linalg.dataset.api.iterator.DataSetIterator source) This method takes DataSetIterator, and starts training over it by scheduling DataSets to different executors
void	fit(org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator source)
protected void	init()
void	setListeners(Collection<org.deeplearning4j.optimize.api.IterationListener> listeners) This method allows you to specify IterationListeners for this model.
void	setListeners(org.deeplearning4j.optimize.api.IterationListener... listeners) This method allows you to specify IterationListeners for this model.
void	setListeners(StatsStorageRouter statsStorage, Collection<? extends org.deeplearning4j.optimize.api.IterationListener> listeners) Set the listeners, along with a StatsStorageRouter that the results will be shuffled to (in the case of any listeners that implement the RoutingIterationListener interface)
void	setListeners(StatsStorageRouter statsStorage, org.deeplearning4j.optimize.api.IterationListener... listeners) Set the listeners, along with a StatsStorageRouter that the results will be shuffled to (in the case of any listeners that implement the RoutingIterationListener interface)
void	shutdown() This method causes all threads used for parallel training to stop
void	stopFit() Will stop a fit operation from continuing to iterate.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

model

protected org.deeplearning4j.nn.api.Model model

workers

protected int workers

prefetchSize

protected int prefetchSize

averagingFrequency

protected int averagingFrequency

zoo

protected Trainer[] zoo

trainerContext

protected TrainerContext trainerContext

iterationsCounter

protected AtomicLong iterationsCounter

reportScore

protected boolean reportScore

averageUpdaters

protected boolean averageUpdaters

legacyAveraging

protected boolean legacyAveraging

wasAveraged

protected boolean wasAveraged

stopFit

protected AtomicBoolean stopFit

listeners

protected List<org.deeplearning4j.optimize.api.IterationListener> listeners

storageRouter

protected StatsStorageRouter storageRouter

isMQ

protected boolean isMQ

workspaceMode

protected org.deeplearning4j.nn.conf.WorkspaceMode workspaceMode

trainerContextArgs

protected Object[] trainerContextArgs

debug

protected boolean debug

executorService

protected ThreadPoolExecutor executorService

workerCounter

protected final AtomicInteger workerCounter

gradientsAccumulator

protected org.deeplearning4j.optimize.solvers.accumulation.GradientsAccumulator gradientsAccumulator

Constructor Detail

ParallelWrapper

protected ParallelWrapper(org.deeplearning4j.nn.api.Model model,
                          int workers,
                          int prefetchSize)

Method Detail

init

protected void init()

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception

shutdown

public void shutdown()

This method causes all threads used for parallel training to stop

stopFit

public void stopFit()

Will stop a fit operation from continuing to iterate.

fit

public void fit(@NonNull
                org.nd4j.linalg.dataset.api.iterator.MultiDataSetIterator source)

Parameters:

source -

setListeners

public void setListeners(@NonNull
                         Collection<org.deeplearning4j.optimize.api.IterationListener> listeners)

This method allows you to specify IterationListeners for this model. Note that for listeners like StatsListener (that have state that will be sent somewhere), consider instead using setListeners(StatsStorageRouter, Collection)

Parameters:

listeners - Listeners to set

setListeners

public void setListeners(@NonNull
                         org.deeplearning4j.optimize.api.IterationListener... listeners)

This method allows you to specify IterationListeners for this model. Note that for listeners like StatsListener (that have state that will be sent somewhere), consider instead using setListeners(StatsStorageRouter, Collection)

Parameters:

listeners - Listeners to set

setListeners

public void setListeners(StatsStorageRouter statsStorage,
                         org.deeplearning4j.optimize.api.IterationListener... listeners)

Set the listeners, along with a StatsStorageRouter that the results will be shuffled to (in the case of any listeners that implement the RoutingIterationListener interface)

Parameters:

statsStorage - Stats storage router to place the results into

listeners - Listeners to set

setListeners

public void setListeners(StatsStorageRouter statsStorage,
                         Collection<? extends org.deeplearning4j.optimize.api.IterationListener> listeners)

Set the listeners, along with a StatsStorageRouter that the results will be shuffled to (in the case of any listeners that implement the RoutingIterationListener interface)

Parameters:

statsStorage - Stats storage router to place the results into

listeners - Listeners to set

broadcastGradients

public void broadcastGradients(org.deeplearning4j.optimize.listeners.SharedGradient gradients)

This method will propagate gradients across all workers

Parameters:

gradients -

fit

public void fit(@NonNull
                org.nd4j.linalg.dataset.api.iterator.DataSetIterator source)

This method takes DataSetIterator, and starts training over it by scheduling DataSets to different executors

Parameters:

source -