org.deeplearning4j.models.word2vec.wordstore.inmemory

Class InMemoryLookupCache

java.lang.Object

org.deeplearning4j.models.word2vec.wordstore.inmemory.InMemoryLookupCache

All Implemented Interfaces:

Serializable, VocabCache<VocabWord>

Deprecated.

@Deprecated
public class InMemoryLookupCache
extends Object
implements VocabCache<VocabWord>, Serializable

In memory lookup cache for smaller datasets PLEASE NOTE: Consider using AbstractCache instead.

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
org.nd4j.linalg.primitives.Counter<String>	docFrequencies Deprecated.
Map<String,VocabWord>	tokens Deprecated.
Map<String,VocabWord>	vocabs Deprecated.
org.nd4j.linalg.primitives.Counter<String>	wordFrequencies Deprecated.

Constructor Summary

Constructors

Constructor and Description
InMemoryLookupCache() Deprecated.
InMemoryLookupCache(boolean addUnk) Deprecated.

Method Summary

Modifier and Type	Method and Description
void	addToken(VocabWord word) Deprecated. Adds a token to the cache
void	addWordToIndex(int index, long elementId) Deprecated.
void	addWordToIndex(int index, String word) Deprecated.
boolean	containsWord(String word) Deprecated. Returns true if the cache contains the given word
int	docAppearedIn(String word) Deprecated. Count of documents a word appeared in
VocabWord	elementAtIndex(int index) Deprecated. Returns SequenceElement at the given index or null
boolean	equals(Object o) Deprecated.
Map<String,VocabWord>	getTokens() Deprecated.
Map<String,VocabWord>	getVocabs() Deprecated.
org.nd4j.linalg.primitives.Counter<String>	getWordFrequencies() Deprecated.
int	hashCode() Deprecated.
boolean	hasToken(String token) Deprecated. Returns whether the cache contains this token or not
void	importVocabulary(VocabCache<VocabWord> vocabCache) Deprecated. imports vocabulary
void	incrementDocCount(String word, long howMuch) Deprecated. Increment the document count
void	incrementTotalDocCount() Deprecated. Increment the doc count
void	incrementTotalDocCount(long by) Deprecated. Increment the doc count
void	incrementWordCount(String word) Deprecated. Increment the count for the given word
void	incrementWordCount(String word, int increment) Deprecated. Increment the count for the given word by the amount increment
int	indexOf(String word) Deprecated. Returns the index of a given word
static InMemoryLookupCache	load(InputStream from) Deprecated. Load a look up cache from an input stream delimited by \n
void	loadVocab() Deprecated. Load vocab
int	numWords() Deprecated. Returns the number of words in the cache
void	putVocabWord(String word) Deprecated.
void	removeElement(String label) Deprecated. Removes element with specified label from vocabulary Please note: Huffman index should be updated after element removal
void	removeElement(VocabWord element) Deprecated. Removes specified element from vocabulary Please note: Huffman index should be updated after element removal
void	saveVocab() Deprecated. Saves the vocab: this allow for reuse of word frequencies
void	setCountForDoc(String word, long count) Deprecated. Set the count for the number of documents the word appears in
void	setTokens(Map<String,VocabWord> tokens) Deprecated.
void	setVocabs(Map<String,VocabWord> vocabs) Deprecated.
void	setWordFrequencies(org.nd4j.linalg.primitives.Counter<String> cnt) Deprecated.
VocabWord	tokenFor(long id) Deprecated.
VocabWord	tokenFor(String word) Deprecated. Returns the token (again not necessarily in the vocab) for this word
Collection<VocabWord>	tokens() Deprecated. All of the tokens in the cache, (not necessarily apart of the vocab)
String	toString() Deprecated.
long	totalNumberOfDocs() Deprecated. Returns the total of number of documents encountered in the corpus
long	totalWordOccurrences() Deprecated. The total number of word occurrences
void	updateWordsOccurencies() Deprecated. Updates counters
boolean	vocabExists() Deprecated. Vocab exists already
Collection<VocabWord>	vocabWords() Deprecated. Returns all of the vocab word nodes
String	wordAtIndex(int index) Deprecated. Returns the word contained at the given index or null
VocabWord	wordFor(long id) Deprecated.
VocabWord	wordFor(String word) Deprecated.
int	wordFrequency(String word) Deprecated. Returns the number of times the word has occurred
Collection<String>	words() Deprecated. Returns all of the words in the vocab

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

wordFrequencies

public org.nd4j.linalg.primitives.Counter<String> wordFrequencies

Deprecated.

docFrequencies

public org.nd4j.linalg.primitives.Counter<String> docFrequencies

Deprecated.

vocabs

public Map<String,VocabWord> vocabs

Deprecated.

tokens

public Map<String,VocabWord> tokens

Deprecated.

Constructor Detail

InMemoryLookupCache

public InMemoryLookupCache()

Deprecated.

InMemoryLookupCache

@Deprecated
public InMemoryLookupCache(boolean addUnk)

Deprecated.

Method Detail

setWordFrequencies

public void setWordFrequencies(org.nd4j.linalg.primitives.Counter<String> cnt)

Deprecated.

getVocabs

public Map<String,VocabWord> getVocabs()

Deprecated.

setVocabs

public void setVocabs(Map<String,VocabWord> vocabs)

Deprecated.

getWordFrequencies

public org.nd4j.linalg.primitives.Counter<String> getWordFrequencies()

Deprecated.

setTokens

public void setTokens(Map<String,VocabWord> tokens)

Deprecated.

getTokens

public Map<String,VocabWord> getTokens()

Deprecated.

words

public Collection<String> words()

Deprecated.

Returns all of the words in the vocab

Specified by:

words in interface VocabCache<VocabWord>

incrementWordCount

public void incrementWordCount(String word)

Deprecated.

Increment the count for the given word

Specified by:

incrementWordCount in interface VocabCache<VocabWord>

Parameters:

word - the word to increment the count for

incrementWordCount

public void incrementWordCount(String word,
                               int increment)

Deprecated.

Increment the count for the given word by the amount increment

Specified by:

incrementWordCount in interface VocabCache<VocabWord>

Parameters:

word - the word to increment the count for

increment - the amount to increment by

wordFrequency

public int wordFrequency(String word)

Deprecated.

Returns the number of times the word has occurred

Specified by:

wordFrequency in interface VocabCache<VocabWord>

Parameters:

word - the word to retrieve the occurrence frequency for

Returns:

0 if hasn't occurred or the number of times the word occurs

containsWord

public boolean containsWord(String word)

Deprecated.

Returns true if the cache contains the given word

Specified by:

containsWord in interface VocabCache<VocabWord>

Parameters:

word - the word to check for

Returns:

wordAtIndex

public String wordAtIndex(int index)

Deprecated.

Returns the word contained at the given index or null

Specified by:

wordAtIndex in interface VocabCache<VocabWord>

Parameters:

index - the index of the word to get

Returns:

the word at the given index

elementAtIndex

public VocabWord elementAtIndex(int index)

Deprecated.

Description copied from interface: VocabCache

Returns SequenceElement at the given index or null

Specified by:

elementAtIndex in interface VocabCache<VocabWord>

Returns:

indexOf

public int indexOf(String word)

Deprecated.

Returns the index of a given word

Specified by:

indexOf in interface VocabCache<VocabWord>

Parameters:

word - the index of a given word

Returns:

the index of a given word or -1 if not found

vocabWords

public Collection<VocabWord> vocabWords()

Deprecated.

Returns all of the vocab word nodes

Specified by:

vocabWords in interface VocabCache<VocabWord>

Returns:

totalWordOccurrences

public long totalWordOccurrences()

Deprecated.

The total number of word occurrences

Specified by:

totalWordOccurrences in interface VocabCache<VocabWord>

Returns:

the total number of word occurrences

wordFor

public VocabWord wordFor(String word)

Deprecated.

Specified by:

wordFor in interface VocabCache<VocabWord>

Parameters:

word -

Returns:

wordFor

public VocabWord wordFor(long id)

Deprecated.

Specified by:

wordFor in interface VocabCache<VocabWord>

addWordToIndex

public void addWordToIndex(int index,
                           String word)

Deprecated.

Specified by:

addWordToIndex in interface VocabCache<VocabWord>

Parameters:

index -

word -

addWordToIndex

public void addWordToIndex(int index,
                           long elementId)

Deprecated.

Specified by:

addWordToIndex in interface VocabCache<VocabWord>

putVocabWord

@Deprecated
public void putVocabWord(String word)

Deprecated.

Description copied from interface: VocabCache

Inserts the word as a vocab word (it gets the vocab word from the internal token store). Note that the index must be set on the token.

Specified by:

putVocabWord in interface VocabCache<VocabWord>

Parameters:

word -

numWords

public int numWords()

Deprecated.

Returns the number of words in the cache

Specified by:

numWords in interface VocabCache<VocabWord>

Returns:

the number of words in the cache

docAppearedIn

public int docAppearedIn(String word)

Deprecated.

Description copied from interface: VocabCache

Count of documents a word appeared in

Specified by:

docAppearedIn in interface VocabCache<VocabWord>

Parameters:

word - the number of documents the word appeared in

Returns:

incrementDocCount

public void incrementDocCount(String word,
                              long howMuch)

Deprecated.

Description copied from interface: VocabCache

Increment the document count

Specified by:

incrementDocCount in interface VocabCache<VocabWord>

Parameters:

word - the word to increment by

setCountForDoc

public void setCountForDoc(String word,
                           long count)

Deprecated.

Description copied from interface: VocabCache

Set the count for the number of documents the word appears in

Specified by:

setCountForDoc in interface VocabCache<VocabWord>

Parameters:

word - the word to set the count for

count - the count of the word

totalNumberOfDocs

public long totalNumberOfDocs()

Deprecated.

Description copied from interface: VocabCache

Returns the total of number of documents encountered in the corpus

Specified by:

totalNumberOfDocs in interface VocabCache<VocabWord>

Returns:

the total number of docs in the corpus

incrementTotalDocCount

public void incrementTotalDocCount()

Deprecated.

Description copied from interface: VocabCache

Increment the doc count

Specified by:

incrementTotalDocCount in interface VocabCache<VocabWord>

incrementTotalDocCount

public void incrementTotalDocCount(long by)

Deprecated.

Description copied from interface: VocabCache

Increment the doc count

Specified by:

incrementTotalDocCount in interface VocabCache<VocabWord>

Parameters:

by - the number to increment by

tokens

public Collection<VocabWord> tokens()

Deprecated.

Description copied from interface: VocabCache

All of the tokens in the cache, (not necessarily apart of the vocab)

Specified by:

tokens in interface VocabCache<VocabWord>

Returns:

the tokens for this cache

addToken

public void addToken(VocabWord word)

Deprecated.

Description copied from interface: VocabCache

Adds a token to the cache

Specified by:

addToken in interface VocabCache<VocabWord>

Parameters:

word - the word to add

tokenFor

public VocabWord tokenFor(String word)

Deprecated.

Description copied from interface: VocabCache

Returns the token (again not necessarily in the vocab) for this word

Specified by:

tokenFor in interface VocabCache<VocabWord>

Parameters:

word - the word to get the token for

Returns:

the vocab word for this token

tokenFor

public VocabWord tokenFor(long id)

Deprecated.

Specified by:

tokenFor in interface VocabCache<VocabWord>

hasToken

public boolean hasToken(String token)

Deprecated.

Description copied from interface: VocabCache

Returns whether the cache contains this token or not

Specified by:

hasToken in interface VocabCache<VocabWord>

Parameters:

token - the token to tes

Returns:

whether the token exists in the cache or not

importVocabulary

public void importVocabulary(VocabCache<VocabWord> vocabCache)

Deprecated.

Description copied from interface: VocabCache

imports vocabulary

Specified by:

importVocabulary in interface VocabCache<VocabWord>

updateWordsOccurencies

public void updateWordsOccurencies()

Deprecated.

Description copied from interface: VocabCache

Updates counters

Specified by:

updateWordsOccurencies in interface VocabCache<VocabWord>

removeElement

public void removeElement(String label)

Deprecated.

Description copied from interface: VocabCache

Removes element with specified label from vocabulary Please note: Huffman index should be updated after element removal

Specified by:

removeElement in interface VocabCache<VocabWord>

Parameters:

label - label of the element to be removed

removeElement

public void removeElement(VocabWord element)

Deprecated.

Description copied from interface: VocabCache

Removes specified element from vocabulary Please note: Huffman index should be updated after element removal

Specified by:

removeElement in interface VocabCache<VocabWord>

Parameters:

element - SequenceElement to be removed

saveVocab

public void saveVocab()

Deprecated.

Description copied from interface: VocabCache

Saves the vocab: this allow for reuse of word frequencies

Specified by:

saveVocab in interface VocabCache<VocabWord>

vocabExists

public boolean vocabExists()

Deprecated.

Description copied from interface: VocabCache

Vocab exists already

Specified by:

vocabExists in interface VocabCache<VocabWord>

Returns:

load

public static InMemoryLookupCache load(InputStream from)

Deprecated.

Load a look up cache from an input stream delimited by \n

Parameters:

from - the input stream to read from

Returns:

the in memory lookup cache

loadVocab

public void loadVocab()

Deprecated.

Description copied from interface: VocabCache

Load vocab

Specified by:

loadVocab in interface VocabCache<VocabWord>

equals

public boolean equals(Object o)

Deprecated.

Overrides:

equals in class Object

hashCode

public int hashCode()

Deprecated.

Overrides:

hashCode in class Object

toString

public String toString()

Deprecated.

Overrides:

toString in class Object