org.nd4j.parameterserver.distributed.transport

Class BaseTransport

java.lang.Object

org.nd4j.parameterserver.distributed.transport.BaseTransport

All Implemented Interfaces:

Transport

Direct Known Subclasses:

MulticastTransport, RoutedTransport

public abstract class BaseTransport
extends Object
implements Transport

Author:

raver119@gmail.com

Nested Class Summary

Nested classes/interfaces inherited from interface org.nd4j.parameterserver.distributed.transport.Transport

Transport.ThreadingModel

Field Summary

Fields

Modifier and Type	Field and Description
protected io.aeron.Aeron	aeron
protected Clipboard	clipboard
protected Map<Long,MeaningfulMessage>	completed
protected io.aeron.Aeron.Context	context
protected io.aeron.driver.MediaDriver	driver
protected org.agrona.concurrent.IdleStrategy	feedbackIdler
protected AtomicLong	frameCount
protected org.agrona.concurrent.IdleStrategy	idler
protected String	ip
protected io.aeron.FragmentAssembler	messageHandlerForClients
protected io.aeron.FragmentAssembler	messageHandlerForShards
protected LinkedBlockingQueue<VoidMessage>	messages
protected NodeRole	nodeRole
protected long	originatorId
protected int	port
protected io.aeron.Publication	publicationForClients
protected io.aeron.Publication	publicationForShards
protected AtomicBoolean	runner
protected short	shardIndex
protected io.aeron.Subscription	subscriptionForClients
protected io.aeron.Subscription	subscriptionForShards
protected short	targetIndex
protected Thread	threadA
protected Thread	threadB
protected Transport.ThreadingModel	threadingModel
protected String	unicastChannelUri
protected VoidConfiguration	voidConfiguration

Constructor Summary

Constructors

Constructor and Description
BaseTransport()

Method Summary

Modifier and Type	Method and Description
void	addClient(String ip, int port)
void	addShard(String ip, int port)
protected void	clientMessageHandler(org.agrona.DirectBuffer buffer, int offset, int length, io.aeron.logbuffer.Header header) This message handler is responsible for receiving messages on Client side
String	getIp()
long	getOwnOriginatorId() This method returns ID of this Transport instance
int	getPort()
void	init(VoidConfiguration voidConfiguration, Clipboard clipboard, NodeRole role, String localIp, int localPort, short shardIndex) This method does initialization of Transport instance
protected void	internalMessageHandler(org.agrona.DirectBuffer buffer, int offset, int length, io.aeron.logbuffer.Header header) This message handler is responsible for receiving coordination messages on Shard side
void	launch(Transport.ThreadingModel threading) This method starts transport mechanisms.
int	numberOfKnownClients() This method returns number of known Clients
int	numberOfKnownShards() This method returns number of known Shards
VoidMessage	peekMessage() This method peeks 1 message from "incoming messages" queue, returning null if queue is empty PLEASE NOTE: This method is suitable for debug purposes only
void	putMessage(VoidMessage message) This method puts message into processing queue
void	receiveMessage(VoidMessage message) This method saves incoming message to the Queue, for later dispatch from higher-level code, like actual TrainingFunction or VoidParameterServer itself
protected void	sendCommandToShard(VoidMessage message) This command is possible to issue only from Client
protected abstract void	sendCoordinationCommand(VoidMessage message) This command is possible to issue only from Shard
protected abstract void	sendFeedbackToClient(VoidMessage message) This command is possible to issue only from Shard
void	sendMessage(VoidMessage message) This method accepts message for delivery, routing is applied according on message type
MeaningfulMessage	sendMessageAndGetResponse(VoidMessage message) This method accepts message for delivery, and blocks until response delivered
void	sendMessageToAllClients(VoidMessage message, Long... exclusions)
void	sendMessageToAllShards(VoidMessage message)
void	setIpAndPort(String ip, int port)
protected void	shardMessageHandler(org.agrona.DirectBuffer buffer, int offset, int length, io.aeron.logbuffer.Header header) This message handler is responsible for receiving messages on Shard side
void	shutdown() This method stops transport system.
protected void	shutdownSilent()
VoidMessage	takeMessage() This method takes 1 message from "incoming messages" queue, blocking if queue is empty

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.nd4j.parameterserver.distributed.transport.Transport

getShardIndex, getTargetIndex

Field Detail

voidConfiguration

protected VoidConfiguration voidConfiguration

nodeRole

protected NodeRole nodeRole

aeron

protected io.aeron.Aeron aeron

context

protected io.aeron.Aeron.Context context

unicastChannelUri

protected String unicastChannelUri

ip

protected String ip

port

protected int port

driver

protected io.aeron.driver.MediaDriver driver

publicationForShards

protected io.aeron.Publication publicationForShards

publicationForClients

protected io.aeron.Publication publicationForClients

subscriptionForShards

protected io.aeron.Subscription subscriptionForShards

subscriptionForClients

protected io.aeron.Subscription subscriptionForClients

messageHandlerForShards

protected io.aeron.FragmentAssembler messageHandlerForShards

messageHandlerForClients

protected io.aeron.FragmentAssembler messageHandlerForClients

messages

protected LinkedBlockingQueue<VoidMessage> messages

completed

protected Map<Long,MeaningfulMessage> completed

runner

protected AtomicBoolean runner

threadA

protected Thread threadA

threadB

protected Thread threadB

clipboard

protected Clipboard clipboard

frameCount

protected AtomicLong frameCount

idler

protected org.agrona.concurrent.IdleStrategy idler

feedbackIdler

protected org.agrona.concurrent.IdleStrategy feedbackIdler

threadingModel

protected Transport.ThreadingModel threadingModel

originatorId

protected long originatorId

targetIndex

protected short targetIndex

shardIndex

protected short shardIndex

Constructor Detail

BaseTransport

public BaseTransport()

Method Detail

getOwnOriginatorId

public long getOwnOriginatorId()

Description copied from interface: Transport

This method returns ID of this Transport instance

Specified by:

getOwnOriginatorId in interface Transport

Returns:

sendMessageAndGetResponse

public MeaningfulMessage sendMessageAndGetResponse(@NonNull
                                                   VoidMessage message)

Description copied from interface: Transport

This method accepts message for delivery, and blocks until response delivered

Specified by:

sendMessageAndGetResponse in interface Transport

Returns:

setIpAndPort

public void setIpAndPort(@NonNull
                         String ip,
                         int port)

Specified by:

setIpAndPort in interface Transport

sendMessage

public void sendMessage(@NonNull
                        VoidMessage message)

Description copied from interface: Transport

This method accepts message for delivery, routing is applied according on message type

Specified by:

sendMessage in interface Transport

shardMessageHandler

protected void shardMessageHandler(org.agrona.DirectBuffer buffer,
                                   int offset,
                                   int length,
                                   io.aeron.logbuffer.Header header)

This message handler is responsible for receiving messages on Shard side

Parameters:

buffer -

offset -

length -

header -

internalMessageHandler

protected void internalMessageHandler(org.agrona.DirectBuffer buffer,
                                      int offset,
                                      int length,
                                      io.aeron.logbuffer.Header header)

This message handler is responsible for receiving coordination messages on Shard side

Parameters:

buffer -

offset -

length -

header -

clientMessageHandler

protected void clientMessageHandler(org.agrona.DirectBuffer buffer,
                                    int offset,
                                    int length,
                                    io.aeron.logbuffer.Header header)

This message handler is responsible for receiving messages on Client side

Parameters:

buffer -

offset -

length -

header -

sendMessageToAllShards

public void sendMessageToAllShards(VoidMessage message)

Specified by:

sendMessageToAllShards in interface Transport

Parameters:

message -

init

public void init(VoidConfiguration voidConfiguration,
                 Clipboard clipboard,
                 NodeRole role,
                 String localIp,
                 int localPort,
                 short shardIndex)

This method does initialization of Transport instance

Specified by:

init in interface Transport

Parameters:

voidConfiguration -

clipboard -

role -

localIp -

localPort -

shardIndex -

launch

public void launch(@NonNull
                   Transport.ThreadingModel threading)

This method starts transport mechanisms. PLEASE NOTE: init() method should be called prior to launch() call

Specified by:

launch in interface Transport

shutdownSilent

protected void shutdownSilent()

shutdown

public void shutdown()

This method stops transport system.

Specified by:

shutdown in interface Transport

receiveMessage

public void receiveMessage(VoidMessage message)

This method saves incoming message to the Queue, for later dispatch from higher-level code, like actual TrainingFunction or VoidParameterServer itself

Specified by:

receiveMessage in interface Transport

Parameters:

message -

takeMessage

public VoidMessage takeMessage()

This method takes 1 message from "incoming messages" queue, blocking if queue is empty

Specified by:

takeMessage in interface Transport

Returns:

putMessage

public void putMessage(@NonNull
                       VoidMessage message)

This method puts message into processing queue

Specified by:

putMessage in interface Transport

Parameters:

message -

peekMessage

public VoidMessage peekMessage()

This method peeks 1 message from "incoming messages" queue, returning null if queue is empty PLEASE NOTE: This method is suitable for debug purposes only

Specified by:

peekMessage in interface Transport

Returns:

sendCommandToShard

protected void sendCommandToShard(VoidMessage message)

This command is possible to issue only from Client

Parameters:

message -

sendCoordinationCommand

protected abstract void sendCoordinationCommand(VoidMessage message)

This command is possible to issue only from Shard

Parameters:

message -

sendFeedbackToClient

protected abstract void sendFeedbackToClient(VoidMessage message)

This command is possible to issue only from Shard

Parameters:

message -

addClient

public void addClient(String ip,
                      int port)

Specified by:

addClient in interface Transport

getIp

public String getIp()

Specified by:

getIp in interface Transport

getPort

public int getPort()

Specified by:

getPort in interface Transport

numberOfKnownClients

public int numberOfKnownClients()

Description copied from interface: Transport

This method returns number of known Clients

Specified by:

numberOfKnownClients in interface Transport

Returns:

numberOfKnownShards

public int numberOfKnownShards()

Description copied from interface: Transport

This method returns number of known Shards

Specified by:

numberOfKnownShards in interface Transport

Returns:

addShard

public void addShard(String ip,
                     int port)

Specified by:

addShard in interface Transport

sendMessageToAllClients

public void sendMessageToAllClients(VoidMessage message,
                                    Long... exclusions)

Specified by:

sendMessageToAllClients in interface Transport