org.apache.beam.runners.spark

Class SparkNativePipelineVisitor

java.lang.Object

org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

org.apache.beam.runners.spark.SparkRunner.Evaluator

org.apache.beam.runners.spark.SparkNativePipelineVisitor

All Implemented Interfaces:

Pipeline.PipelineVisitor

public class SparkNativePipelineVisitor
extends SparkRunner.Evaluator

Pipeline visitor for translating a Beam pipeline into equivalent Spark operations. Used for debugging purposes using SparkRunnerDebugger.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor

Pipeline.PipelineVisitor.CompositeBehavior, Pipeline.PipelineVisitor.Defaults

Field Summary

Fields inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator

ctxt, translator

Method Summary

Modifier and Type	Method and Description
Pipeline.PipelineVisitor.CompositeBehavior	enterCompositeTransform(TransformHierarchy.Node node)

Methods inherited from class org.apache.beam.runners.spark.SparkRunner.Evaluator

isBoundedCollection, shouldDefer, translate, visitPrimitiveTransform

Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

enterPipeline, getPipeline, leaveCompositeTransform, leavePipeline, visitValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

enterCompositeTransform

public Pipeline.PipelineVisitor.CompositeBehavior enterCompositeTransform(TransformHierarchy.Node node)

Specified by:

enterCompositeTransform in interface Pipeline.PipelineVisitor

Overrides:

enterCompositeTransform in class SparkRunner.Evaluator