org.datavec.api.split

Class TransformSplit

java.lang.Object

org.datavec.api.split.BaseInputSplit

org.datavec.api.split.TransformSplit

All Implemented Interfaces:

Serializable, InputSplit, Writable

public class TransformSplit
extends BaseInputSplit

InputSplit implementation that maps the URIs of a given BaseInputSplit to new URIs. Useful when features and labels are in different files sharing a common naming scheme, and the name of the output file can be determined given the name of the input file.

Author:

Ede Meijer

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	TransformSplit.URITransform

Field Summary

Fields inherited from class org.datavec.api.split.BaseInputSplit

iterationOrder, length, uriStrings

Constructor Summary

Constructors

Constructor and Description
TransformSplit(BaseInputSplit sourceSplit, TransformSplit.URITransform transform) Apply a given transformation to the raw URI objects

Method Summary

Modifier and Type	Method and Description
static TransformSplit	ofSearchReplace(BaseInputSplit sourceSplit, String search, String replace) Static factory method, replace the string version of the URI with a simple search-replace pair
void	readFields(DataInput in) Deserialize the fields of this object from in.
void	reset() Reset the InputSplit without reinitializing it from scratch.
void	write(DataOutput out) Serialize the fields of this object to out.

Methods inherited from class org.datavec.api.split.BaseInputSplit

getType, length, locations, locationsIterator, locationsPathIterator, sample, toDouble, toFloat, toInt, toLong, writeType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TransformSplit

public TransformSplit(@NonNull
                      BaseInputSplit sourceSplit,
                      @NonNull
                      TransformSplit.URITransform transform)
               throws URISyntaxException

Apply a given transformation to the raw URI objects

Parameters:

sourceSplit - the split with URIs to transform

transform - transform operation that returns a new URI based on an input URI

Throws:

URISyntaxException - thrown if the transformed URI is malformed

Method Detail

ofSearchReplace

public static TransformSplit ofSearchReplace(@NonNull
                                             BaseInputSplit sourceSplit,
                                             @NonNull
                                             String search,
                                             @NonNull
                                             String replace)
                                      throws URISyntaxException

Static factory method, replace the string version of the URI with a simple search-replace pair

Parameters:

sourceSplit - the split with URIs to transform

search - the string to search

replace - the string to replace with

Throws:

URISyntaxException - thrown if the transformed URI is malformed

write

public void write(DataOutput out)
           throws IOException

Description copied from interface: Writable

Serialize the fields of this object to out.

Parameters:

out - DataOuput to serialize this object into.

Throws:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

Description copied from interface: Writable

Deserialize the fields of this object from in.

For efficiency, implementations should attempt to re-use storage in the existing object where possible.

Parameters:

in - DataInput to deseriablize this object from.

Throws:

IOException

reset

public void reset()

Description copied from interface: InputSplit

Reset the InputSplit without reinitializing it from scratch. In many cases, this is a no-op. For InputSplits that have randomization: reset should shuffle the order.