See: Description
| Interface | Description |
|---|---|
| Dataset |
Holder for Spark RDD/DStream.
|
| SparkPipelineTranslator |
Translator to support translation between Beam transformations and Spark transformations.
|
| TransformEvaluator<TransformT extends PTransform<?,?>> |
Describe a
PTransform evaluator. |
| Class | Description |
|---|---|
| BoundedDataset<T> |
Holds an RDD or values for deferred conversion to an RDD if needed.
|
| EvaluationContext |
The EvaluationContext allows us to define pipeline instructions and translate between
PObject<T>s or PCollection<T>s and Ts or DStreams/RDDs of Ts. |
| GroupCombineFunctions |
A set of group/combine functions to apply to Spark
RDDs. |
| MultiDoFnFunction<InputT,OutputT> |
DoFunctions ignore outputs that are not the main output.
|
| ReifyTimestampsAndWindowsFunction<K,V> |
Simple
Function to bring the windowing information into the value from the implicit
background representation of the PCollection. |
| SparkAbstractCombineFn |
An abstract for the SparkRunner implementation of
Combine.CombineFn. |
| SparkAssignWindowFn<T,W extends BoundedWindow> |
An implementation of
Window.Assign for the Spark
runner. |
| SparkContextFactory |
The Spark context factory.
|
| SparkGlobalCombineFn<InputT,AccumT,OutputT> |
A
CombineFnBase.GlobalCombineFn
with a CombineWithContext.Context for the SparkRunner. |
| SparkGroupAlsoByWindowViaOutputBufferFn<K,InputT,W extends BoundedWindow> |
An implementation of
GroupByKeyViaGroupByKeyOnly.GroupAlsoByWindow
for the Spark runner. |
| SparkKeyedCombineFn<K,InputT,AccumT,OutputT> |
A
CombineFnBase.GlobalCombineFn
with a CombineWithContext.Context for the SparkRunner. |
| SparkPCollectionView |
SparkPCollectionView is used to pass serialized views to lambdas.
|
| SparkRuntimeContext |
The SparkRuntimeContext allows us to define useful features on the client side before our
data flow program is launched.
|
| StorageLevelPTransform |
Get RDD storage level for the input PCollection (mostly used for testing purpose).
|
| TransformTranslator |
Supports translation between a Beam transform, and Spark's operations on RDDs.
|
| TransformTranslator.Translator |
Translator matches Beam transformation with the appropriate evaluator.
|
| TranslationUtils |
A set of utilities to help translating Beam transformations into Spark transformations.
|
| TranslationUtils.CombineGroupedValues<K,InputT,OutputT> |
A SparkKeyedCombineFn function applied to grouped KVs.
|
| TranslationUtils.TupleTagFilter<V> |
A utility class to filter
TupleTags. |
| WindowingHelpers |
Helper functions for working with windows.
|
Copyright © 2016–2017 The Apache Software Foundation. All rights reserved.