org.datavec.hadoop.records.reader.mapfile.index

Class LongIndexToKey

java.lang.Object

org.datavec.hadoop.records.reader.mapfile.index.LongIndexToKey

All Implemented Interfaces:

IndexToKey

public class LongIndexToKey
extends Object
implements IndexToKey

A default implementation of IndexToKey that assumes (strictly requires) keys that are LongWritable values, where all values are both unique and contiguous (0 to numRecords()-1)
This allows for easy inference of the number of records, and identify mapping between indexes and keys.

Author:

Alex Black

Constructor Summary

Constructors

Constructor and Description
LongIndexToKey()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.io.LongWritable	getKeyForIndex(long index) Get the key for the given index
long	getNumRecords() Getter infer the number of records in the given map file(s)
List<org.nd4j.linalg.primitives.Pair<Long,Long>>	initialize(org.apache.hadoop.io.MapFile.Reader[] readers, Class<? extends org.apache.hadoop.io.Writable> valueClass) Initialise the instance, and return the first and last record indexes (inclusive) for each reader

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LongIndexToKey

public LongIndexToKey()

Method Detail

initialize

public List<org.nd4j.linalg.primitives.Pair<Long,Long>> initialize(org.apache.hadoop.io.MapFile.Reader[] readers,
                                                                   Class<? extends org.apache.hadoop.io.Writable> valueClass)
                                                            throws IOException

Description copied from interface: IndexToKey

Initialise the instance, and return the first and last record indexes (inclusive) for each reader

Specified by:

initialize in interface IndexToKey

Parameters:

readers - The underlying map file readers

Throws:

IOException

getKeyForIndex

public org.apache.hadoop.io.LongWritable getKeyForIndex(long index)

Description copied from interface: IndexToKey

Get the key for the given index

Specified by:

getKeyForIndex in interface IndexToKey

Parameters:

index - 0 to getNumRecords(reader)

Returns:

The key for the given index

getNumRecords

public long getNumRecords()
                   throws IOException

Description copied from interface: IndexToKey

Getter infer the number of records in the given map file(s)

Specified by:

getNumRecords in interface IndexToKey

Returns:

Number of records in the map file(s)

Throws:

IOException