org.apache.beam.runners.spark.translation.streaming

Class Checkpoint.CheckpointDir

java.lang.Object

org.apache.beam.runners.spark.translation.streaming.Checkpoint.CheckpointDir

Enclosing class:

Checkpoint

public static class Checkpoint.CheckpointDir
extends Object

Checkpoint dir tree. SparkPipelineOptions checkpointDir is used as a root directory under which one directory is created for Spark's checkpoint and another for Beam's Spark runner's fault checkpointing needs. Spark's checkpoint relies on Hadoop's FileSystem and is used for Beam as well rather than FileSystem to be consistent with Spark.

Constructor Summary

Constructors

Constructor and Description
CheckpointDir(String rootCheckpointDir)

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.fs.Path	getBeamCheckpointDir()
org.apache.hadoop.fs.Path	getRootCheckpointDir()
org.apache.hadoop.fs.Path	getSparkCheckpointDir()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CheckpointDir

public CheckpointDir(String rootCheckpointDir)

Method Detail

getRootCheckpointDir

public org.apache.hadoop.fs.Path getRootCheckpointDir()

getSparkCheckpointDir

public org.apache.hadoop.fs.Path getSparkCheckpointDir()

getBeamCheckpointDir

public org.apache.hadoop.fs.Path getBeamCheckpointDir()