org.apache.flink.runtime.query.netty.message

Class KvStateRequestSerializer

java.lang.Object

org.apache.flink.runtime.query.netty.message.KvStateRequestSerializer

public final class KvStateRequestSerializer
extends Object

Serialization and deserialization of messages exchanged between KvStateClient and KvStateServer.

The binary messages have the following format:

                     <------ Frame ------------------------->
                    +----------------------------------------+
                    |        HEADER (8)      | PAYLOAD (VAR) |
 +------------------+----------------------------------------+
 | FRAME LENGTH (4) | VERSION (4) | TYPE (4) | CONTENT (VAR) |
 +------------------+----------------------------------------+
 

The concrete content of a message depends on the KvStateRequestType.

Constructor Summary

Constructors

Constructor and Description
KvStateRequestSerializer()

Method Summary

Methods

Modifier and Type	Method and Description
static KvStateRequestType	deserializeHeader(io.netty.buffer.ByteBuf buf) Deserializes the header and returns the request type.
static <K,N> Tuple2<K,N>	deserializeKeyAndNamespace(byte[] serializedKeyAndNamespace, TypeSerializer<K> keySerializer, TypeSerializer<N> namespaceSerializer) Deserializes the key and namespace into a Tuple2.
static KvStateRequest	deserializeKvStateRequest(io.netty.buffer.ByteBuf buf) Deserializes the KvState request message.
static KvStateRequestFailure	deserializeKvStateRequestFailure(io.netty.buffer.ByteBuf buf) Deserializes the KvState request failure.
static KvStateRequestResult	deserializeKvStateRequestResult(io.netty.buffer.ByteBuf buf) Deserializes the KvState request result.
static <T> List<T>	deserializeList(byte[] serializedValue, TypeSerializer<T> serializer) Deserializes all values with the given serializer.
static <UK,UV> Map<UK,UV>	deserializeMap(byte[] serializedValue, TypeSerializer<UK> keySerializer, TypeSerializer<UV> valueSerializer) Deserializes all kv pairs with the given serializer.
static Throwable	deserializeServerFailure(io.netty.buffer.ByteBuf buf) Deserializes the KvState request failure.
static <T> T	deserializeValue(byte[] serializedValue, TypeSerializer<T> serializer) Deserializes the value with the given serializer.
static <K,N> byte[]	serializeKeyAndNamespace(K key, TypeSerializer<K> keySerializer, N namespace, TypeSerializer<N> namespaceSerializer) Serializes the key and namespace into a ByteBuffer.
static io.netty.buffer.ByteBuf	serializeKvStateRequest(io.netty.buffer.ByteBufAllocator alloc, long requestId, KvStateID kvStateId, byte[] serializedKeyAndNamespace) Allocates a buffer and serializes the KvState request into it.
static io.netty.buffer.ByteBuf	serializeKvStateRequestFailure(io.netty.buffer.ByteBufAllocator alloc, long requestId, Throwable cause) Allocates a buffer and serializes the KvState request failure into it.
static io.netty.buffer.ByteBuf	serializeKvStateRequestResult(io.netty.buffer.ByteBufAllocator alloc, long requestId, byte[] serializedResult) Allocates a buffer and serializes the KvState request result into it.
static <UK,UV> byte[]	serializeMap(Iterable<Map.Entry<UK,UV>> entries, TypeSerializer<UK> keySerializer, TypeSerializer<UV> valueSerializer) Serializes all values of the Iterable with the given serializer.
static io.netty.buffer.ByteBuf	serializeServerFailure(io.netty.buffer.ByteBufAllocator alloc, Throwable cause) Allocates a buffer and serializes the server failure into it.
static <T> byte[]	serializeValue(T value, TypeSerializer<T> serializer) Serializes the value with the given serializer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KvStateRequestSerializer

public KvStateRequestSerializer()

Method Detail

serializeKvStateRequest

public static io.netty.buffer.ByteBuf serializeKvStateRequest(io.netty.buffer.ByteBufAllocator alloc,
                                              long requestId,
                                              KvStateID kvStateId,
                                              byte[] serializedKeyAndNamespace)

Allocates a buffer and serializes the KvState request into it.

Parameters:

alloc - ByteBuf allocator for the buffer to serialize message into

requestId - ID for this request

kvStateId - ID of the requested KvState instance

serializedKeyAndNamespace - Serialized key and namespace to request from the KvState instance.

Returns:

Serialized KvState request message

serializeKvStateRequestResult

public static io.netty.buffer.ByteBuf serializeKvStateRequestResult(io.netty.buffer.ByteBufAllocator alloc,
                                                    long requestId,
                                                    byte[] serializedResult)

Allocates a buffer and serializes the KvState request result into it.

Parameters:

alloc - ByteBuf allocator for the buffer to serialize message into

requestId - ID for this request

serializedResult - Serialized Result

Returns:

Serialized KvState request result message

serializeKvStateRequestFailure

public static io.netty.buffer.ByteBuf serializeKvStateRequestFailure(io.netty.buffer.ByteBufAllocator alloc,
                                                     long requestId,
                                                     Throwable cause)
                                                              throws IOException

Allocates a buffer and serializes the KvState request failure into it.

Parameters:

alloc - ByteBuf allocator for the buffer to serialize message into

requestId - ID of the request responding to

cause - Failure cause

Returns:

Serialized KvState request failure message

Throws:

IOException - Serialization failures are forwarded

serializeServerFailure

public static io.netty.buffer.ByteBuf serializeServerFailure(io.netty.buffer.ByteBufAllocator alloc,
                                             Throwable cause)
                                                      throws IOException

Allocates a buffer and serializes the server failure into it.

The cause must not be or contain any user types as causes.

Parameters:

alloc - ByteBuf allocator for the buffer to serialize message into

cause - Failure cause

Returns:

Serialized server failure message

Throws:

IOException - Serialization failures are forwarded

deserializeHeader

public static KvStateRequestType deserializeHeader(io.netty.buffer.ByteBuf buf)

Deserializes the header and returns the request type.

Parameters:

buf - Buffer to deserialize (expected to be at header position)

Returns:

Deserialzied request type

Throws:

IllegalArgumentException - If unexpected message version or message type

deserializeKvStateRequest

public static KvStateRequest deserializeKvStateRequest(io.netty.buffer.ByteBuf buf)

Deserializes the KvState request message.

Important: the returned buffer is sliced from the incoming ByteBuf stream and retained. Therefore, it needs to be recycled by the consumer.

Parameters:

buf - Buffer to deserialize (expected to be positioned after header)

Returns:

Deserialized KvStateRequest

deserializeKvStateRequestResult

public static KvStateRequestResult deserializeKvStateRequestResult(io.netty.buffer.ByteBuf buf)

Deserializes the KvState request result.

Parameters:

buf - Buffer to deserialize (expected to be positioned after header)

Returns:

Deserialized KvStateRequestResult

deserializeKvStateRequestFailure

public static KvStateRequestFailure deserializeKvStateRequestFailure(io.netty.buffer.ByteBuf buf)
                                                              throws IOException,
                                                                     ClassNotFoundException

Deserializes the KvState request failure.

Parameters:

buf - Buffer to deserialize (expected to be positioned after header)

Returns:

Deserialized KvStateRequestFailure

Throws:

IOException

ClassNotFoundException

deserializeServerFailure

public static Throwable deserializeServerFailure(io.netty.buffer.ByteBuf buf)
                                          throws IOException,
                                                 ClassNotFoundException

Deserializes the KvState request failure.

Parameters:

buf - Buffer to deserialize (expected to be positioned after header)

Returns:

Deserialized KvStateRequestFailure

Throws:

IOException - Serialization failure are forwarded

ClassNotFoundException - If Exception type can not be loaded

serializeKeyAndNamespace

public static <K,N> byte[] serializeKeyAndNamespace(K key,
                                    TypeSerializer<K> keySerializer,
                                    N namespace,
                                    TypeSerializer<N> namespaceSerializer)
                                       throws IOException

Serializes the key and namespace into a ByteBuffer.

The serialized format matches the RocksDB state backend key format, i.e. the key and namespace don't have to be deserialized for RocksDB lookups.

Type Parameters:

K - Key type

N - Namespace type

Parameters:

key - Key to serialize

keySerializer - Serializer for the key

namespace - Namespace to serialize

namespaceSerializer - Serializer for the namespace

Returns:

Buffer holding the serialized key and namespace

Throws:

IOException - Serialization errors are forwarded

deserializeKeyAndNamespace

public static <K,N> Tuple2<K,N> deserializeKeyAndNamespace(byte[] serializedKeyAndNamespace,
                                           TypeSerializer<K> keySerializer,
                                           TypeSerializer<N> namespaceSerializer)
                                              throws IOException

Deserializes the key and namespace into a Tuple2.

Type Parameters:

K - Key type

N - Namespace

Parameters:

serializedKeyAndNamespace - Serialized key and namespace

keySerializer - Serializer for the key

namespaceSerializer - Serializer for the namespace

Returns:

Tuple2 holding deserialized key and namespace

Throws:

IOException - if the deserialization fails for any reason

serializeValue

public static <T> byte[] serializeValue(T value,
                        TypeSerializer<T> serializer)
                             throws IOException

Serializes the value with the given serializer.

Type Parameters:

T - Type of the value

Parameters:

value - Value of type T to serialize

serializer - Serializer for T

Returns:

Serialized value or null if value null

Throws:

IOException - On failure during serialization

deserializeValue

public static <T> T deserializeValue(byte[] serializedValue,
                     TypeSerializer<T> serializer)
                          throws IOException

Deserializes the value with the given serializer.

Type Parameters:

T - Type of the value

Parameters:

serializedValue - Serialized value of type T

serializer - Serializer for T

Returns:

Deserialized value or null if the serialized value is null

Throws:

IOException - On failure during deserialization

deserializeList

public static <T> List<T> deserializeList(byte[] serializedValue,
                          TypeSerializer<T> serializer)
                               throws IOException

Deserializes all values with the given serializer.

Type Parameters:

T - Type of the value

Parameters:

serializedValue - Serialized value of type List

serializer - Serializer for T

Returns:

Deserialized list or null if the serialized value is null

Throws:

IOException - On failure during deserialization

serializeMap

public static <UK,UV> byte[] serializeMap(Iterable<Map.Entry<UK,UV>> entries,
                          TypeSerializer<UK> keySerializer,
                          TypeSerializer<UV> valueSerializer)
                           throws IOException

Serializes all values of the Iterable with the given serializer.

Type Parameters:

UK - Type of the keys

UV - Type of the values

Parameters:

entries - Key-value pairs to serialize

keySerializer - Serializer for UK

valueSerializer - Serializer for UV

Returns:

Serialized values or null if values null or empty

Throws:

IOException - On failure during serialization

deserializeMap

public static <UK,UV> Map<UK,UV> deserializeMap(byte[] serializedValue,
                                TypeSerializer<UK> keySerializer,
                                TypeSerializer<UV> valueSerializer)
                                 throws IOException

Deserializes all kv pairs with the given serializer.

Type Parameters:

UK - Type of the key

UV - Type of the value.

Parameters:

serializedValue - Serialized value of type Map

keySerializer - Serializer for UK

valueSerializer - Serializer for UV

Returns:

Deserialized map or null if the serialized value is null

Throws:

IOException - On failure during deserialization