org.apache.arrow.memory

Class BaseAllocator

java.lang.Object

org.apache.arrow.memory.BaseAllocator

All Implemented Interfaces:

AutoCloseable, BufferAllocator

Direct Known Subclasses:

RootAllocator

public abstract class BaseAllocator
extends Object
implements BufferAllocator

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Accountant.AllocationOutcome Describes the type of outcome that occurred when trying to account for allocation of memory.
class	BaseAllocator.Reservation
static class	BaseAllocator.Verbosity

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DEBUG
static String	DEBUG_ALLOCATOR
static int	DEBUG_LOG_LENGTH
protected org.apache.arrow.memory.Accountant	parent The parent allocator
protected long	reservation The amount of memory reserved for this allocator.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseAllocator(AllocationListener listener, String name, long initReservation, long maxAllocation)
protected	BaseAllocator(BaseAllocator parentAllocator, String name, long initReservation, long maxAllocation)

Method Summary

Methods

Modifier and Type	Method and Description
void	assertOpen() Asserts (using java assertions) that the provided allocator is currently open.
ArrowBuf	buffer(int initialRequestSize) Allocate a new or reused buffer of the provided size.
ArrowBuf	buffer(int initialRequestSize, BufferManager manager) Allocate a new or reused buffer of the provided size.
void	close() Close this Accountant.
long	getAllocatedMemory() Return the current amount of allocated memory that this Accountant is managing accounting for.
ArrowByteBufAllocator	getAsByteBufAllocator() Returns the allocator this allocator falls back to when it needs more memory.
ArrowBuf	getEmpty() Get a reference to the empty buffer associated with this allocator.
long	getHeadroom()
long	getLimit() Return the current limit of this Accountant.
String	getName() Return the name of this allocator.
long	getPeakMemoryAllocation() The peak memory allocated by this Accountant.
static StringBuilder	indent(StringBuilder sb, int indent)
static boolean	isDebug()
boolean	isOverLimit()
BufferAllocator	newChildAllocator(String name, long initReservation, long maxAllocation) Create a new child allocator.
AllocationReservation	newReservation() Create an allocation reservation.
static int	nextPowerOfTwo(int val) Rounds up the provided value to the nearest power of two.
static long	nextPowerOfTwo(long val) Rounds up the provided value to the nearest power of two.
void	releaseBytes(long size)
void	setLimit(long newLimit) Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.
String	toString()
String	toVerboseString() Provide a verbose string of the current allocator state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.arrow.memory.BufferAllocator

getAllocatedMemory, getHeadroom, getLimit, getPeakMemoryAllocation, isOverLimit, setLimit

Field Detail

DEBUG_ALLOCATOR

public static final String DEBUG_ALLOCATOR

See Also:

Constant Field Values

DEBUG_LOG_LENGTH

public static final int DEBUG_LOG_LENGTH

See Also:

Constant Field Values

DEBUG

public static final boolean DEBUG

parent

protected final org.apache.arrow.memory.Accountant parent

The parent allocator

reservation

protected final long reservation

The amount of memory reserved for this allocator. Releases below this amount of memory will not be returned to the parent Accountant until this Accountant is closed.

Constructor Detail

BaseAllocator

protected BaseAllocator(AllocationListener listener,
             String name,
             long initReservation,
             long maxAllocation)
                 throws OutOfMemoryException

Throws:

OutOfMemoryException

BaseAllocator

protected BaseAllocator(BaseAllocator parentAllocator,
             String name,
             long initReservation,
             long maxAllocation)
                 throws OutOfMemoryException

Throws:

OutOfMemoryException

Method Detail

nextPowerOfTwo

public static int nextPowerOfTwo(int val)

Rounds up the provided value to the nearest power of two.

Parameters:

val - An integer value.

Returns:

The closest power of two of that value.

nextPowerOfTwo

public static long nextPowerOfTwo(long val)

Rounds up the provided value to the nearest power of two.

Parameters:

val - A long value.

Returns:

The closest power of two of that value.

indent

public static StringBuilder indent(StringBuilder sb,
                   int indent)

isDebug

public static boolean isDebug()

assertOpen

public void assertOpen()

Description copied from interface: BufferAllocator

Asserts (using java assertions) that the provided allocator is currently open. If assertions are disabled, this is a no-op.

Specified by:

assertOpen in interface BufferAllocator

getName

public String getName()

Description copied from interface: BufferAllocator

Return the name of this allocator. This is a human readable name that can help debugging. Typically provides coordinates about where this allocator was created

Specified by:

getName in interface BufferAllocator

Returns:

the name of the allocator

getEmpty

public ArrowBuf getEmpty()

Description copied from interface: BufferAllocator

Get a reference to the empty buffer associated with this allocator. Empty buffers are special because we don't worry about them leaking or managing reference counts on them since they don't actually point to any memory.

Specified by:

getEmpty in interface BufferAllocator

Returns:

the empty buffer

buffer

public ArrowBuf buffer(int initialRequestSize)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

Returns:

a new ArrowBuf, or null if the request can't be satisfied

buffer

public ArrowBuf buffer(int initialRequestSize,
              BufferManager manager)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

manager - A buffer manager to manage reallocation.

Returns:

a new ArrowBuf, or null if the request can't be satisfied

getAsByteBufAllocator

public ArrowByteBufAllocator getAsByteBufAllocator()

Description copied from interface: BufferAllocator

Returns the allocator this allocator falls back to when it needs more memory.

Specified by:

getAsByteBufAllocator in interface BufferAllocator

Returns:

the underlying allocator used by this allocator

newChildAllocator

public BufferAllocator newChildAllocator(String name,
                                long initReservation,
                                long maxAllocation)

Description copied from interface: BufferAllocator

Create a new child allocator.

Specified by:

newChildAllocator in interface BufferAllocator

Parameters:

name - the name of the allocator.

initReservation - the initial space reservation (obtained from this allocator)

maxAllocation - maximum amount of space the new allocator can allocate

Returns:

the new allocator, or null if it can't be created

newReservation

public AllocationReservation newReservation()

Description copied from interface: BufferAllocator

Create an allocation reservation. A reservation is a way of building up a request for a buffer whose size is not known in advance. See

Specified by:

newReservation in interface BufferAllocator

Returns:

the newly created reservation

See Also:

AllocationReservation

close

public void close()

Close this Accountant. This will release any reservation bytes back to a parent Accountant.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface BufferAllocator

toString

public String toString()

Overrides:

toString in class Object

toVerboseString

public String toVerboseString()

Provide a verbose string of the current allocator state. Includes the state of all child allocators, along with historical logs of each object and including stacktraces.

Specified by:

toVerboseString in interface BufferAllocator

Returns:

A Verbose string of current allocator state.

releaseBytes

public void releaseBytes(long size)

isOverLimit

public boolean isOverLimit()

getLimit

public long getLimit()

Return the current limit of this Accountant.

Returns:

Limit in bytes.

setLimit

public void setLimit(long newLimit)

Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.

Parameters:

newLimit - The limit in bytes.

getAllocatedMemory

public long getAllocatedMemory()

Return the current amount of allocated memory that this Accountant is managing accounting for. Note this does not include reservation memory that hasn't been allocated.

Returns:

Currently allocate memory in bytes.

getPeakMemoryAllocation

public long getPeakMemoryAllocation()

The peak memory allocated by this Accountant.

Returns:

The peak allocated memory in bytes.

getHeadroom

public long getHeadroom()