org.deeplearning4j.parallelism

Class ParallelInference.Builder

java.lang.Object

org.deeplearning4j.parallelism.ParallelInference.Builder

Enclosing class:

ParallelInference

public static class ParallelInference.Builder
extends Object

Constructor Summary

Constructors

Constructor and Description
Builder(org.deeplearning4j.nn.api.Model model)

Method Summary

Modifier and Type	Method and Description
ParallelInference.Builder	batchLimit(int limit) This method defines, how many input samples can be batched within given time frame.
ParallelInference	build() This method builds new ParallelInference instance
ParallelInference.Builder	inferenceMode(InferenceMode inferenceMode) This method allows you to define mode that'll be used during inference.
ParallelInference.Builder	queueLimit(int limit) This method defines buffer queue size.
ParallelInference.Builder	workers(int workers) This method defines, how many model copies will be used for inference.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder(@NonNull
               org.deeplearning4j.nn.api.Model model)

Method Detail

inferenceMode

public ParallelInference.Builder inferenceMode(@NonNull
                                               InferenceMode inferenceMode)

This method allows you to define mode that'll be used during inference. Options are: SEQUENTIAL: Input will be sent to last-used worker unmodified. BATCHED: Multiple inputs will be packed into single batch, and sent to last-used device.

Parameters:

inferenceMode -

Returns:

workers

public ParallelInference.Builder workers(int workers)

This method defines, how many model copies will be used for inference. PLEASE NOTE: This method primarily suited for multi-GPU systems

Parameters:

workers -

Returns:

batchLimit

public ParallelInference.Builder batchLimit(int limit)

This method defines, how many input samples can be batched within given time frame. PLEASE NOTE: This value has no effect in SEQUENTIAL inference mode

Parameters:

limit -

Returns:

queueLimit

public ParallelInference.Builder queueLimit(int limit)

This method defines buffer queue size. Default value: 64

Parameters:

limit -

Returns:

build

public ParallelInference build()

This method builds new ParallelInference instance

Returns: