org.datavec.api.split

Class InputStreamInputSplit

java.lang.Object

org.datavec.api.split.InputStreamInputSplit

All Implemented Interfaces:

Serializable, InputSplit, Writable

public class InputStreamInputSplit
extends Object
implements InputSplit

Input stream input split. The normal pattern is reading the whole input stream and turning that in to a record. This is meant for streaming raw data rather than normal mini batch pre processing.

Author:

Adam Gibson

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
InputStreamInputSplit(InputStream is)
InputStreamInputSplit(InputStream is, File path) Instantiate with the given file as a uri
InputStreamInputSplit(InputStream is, String path) Instantiate with the given file as a uri
InputStreamInputSplit(InputStream is, URI path) Instantiate with the given file as a uri

Method Summary

Modifier and Type	Method and Description
InputStream	getIs()
WritableType	getType()
long	length() Length of the split
URI[]	locations() Locations of the splits
Iterator<URI>	locationsIterator()
Iterator<String>	locationsPathIterator()
void	readFields(DataInput in) Deserialize the fields of this object from in.
void	reset() Reset the InputSplit without reinitializing it from scratch.
void	setIs(InputStream is)
double	toDouble() Convert Writable to double.
float	toFloat() Convert Writable to float.
int	toInt() Convert Writable to int.
long	toLong() Convert Writable to long.
void	write(DataOutput out) Serialize the fields of this object to out.
void	writeType(DataOutput out) Write the type (a single short value) to the DataOutput.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

InputStreamInputSplit

public InputStreamInputSplit(InputStream is,
                             String path)

Instantiate with the given file as a uri

Parameters:

is - the input stream to use

path - the path to use

InputStreamInputSplit

public InputStreamInputSplit(InputStream is,
                             File path)

Instantiate with the given file as a uri

Parameters:

is - the input stream to use

path - the path to use

InputStreamInputSplit

public InputStreamInputSplit(InputStream is,
                             URI path)

Instantiate with the given file as a uri

Parameters:

is - the input stream to use

path - the path to use

InputStreamInputSplit

public InputStreamInputSplit(InputStream is)

Method Detail

length

public long length()

Description copied from interface: InputSplit

Length of the split

Specified by:

length in interface InputSplit

Returns:

locations

public URI[] locations()

Description copied from interface: InputSplit

Locations of the splits

Specified by:

locations in interface InputSplit

Returns:

locationsIterator

public Iterator<URI> locationsIterator()

Specified by:

locationsIterator in interface InputSplit

locationsPathIterator

public Iterator<String> locationsPathIterator()

Specified by:

locationsPathIterator in interface InputSplit

reset

public void reset()

Description copied from interface: InputSplit

Reset the InputSplit without reinitializing it from scratch. In many cases, this is a no-op. For InputSplits that have randomization: reset should shuffle the order.

Specified by:

reset in interface InputSplit

write

public void write(DataOutput out)
           throws IOException

Description copied from interface: Writable

Serialize the fields of this object to out.

Specified by:

write in interface Writable

Parameters:

out - DataOuput to serialize this object into.

Throws:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

Description copied from interface: Writable

Deserialize the fields of this object from in.

For efficiency, implementations should attempt to re-use storage in the existing object where possible.

Specified by:

readFields in interface Writable

Parameters:

in - DataInput to deseriablize this object from.

Throws:

IOException

getIs

public InputStream getIs()

setIs

public void setIs(InputStream is)

toDouble

public double toDouble()

Description copied from interface: Writable

Convert Writable to double. Whether this is supported depends on the specific writable.

Specified by:

toDouble in interface Writable

toFloat

public float toFloat()

Description copied from interface: Writable

Convert Writable to float. Whether this is supported depends on the specific writable.

Specified by:

toFloat in interface Writable

toInt

public int toInt()

Description copied from interface: Writable

Convert Writable to int. Whether this is supported depends on the specific writable.

Specified by:

toInt in interface Writable

toLong

public long toLong()

Description copied from interface: Writable

Convert Writable to long. Whether this is supported depends on the specific writable.

Specified by:

toLong in interface Writable

getType

public WritableType getType()

Specified by:

getType in interface Writable

writeType

public void writeType(DataOutput out)
               throws IOException

Description copied from interface: Writable

Write the type (a single short value) to the DataOutput. See WritableFactory for details.

Specified by:

writeType in interface Writable

Parameters:

out - DataOutput to write to

Throws:

IOException - For errors during writing