Beam代码：PCollection windowedWords = input.apply(Window.into(FixedWindows.of(Duration.standardMinutes(60))));
FlinkBatchPipelineTranslator
Flink API public static void main(String[] args) throws Exception {\tStreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();\tDataStreamWindowedValue source =\tenv.addSource( sourceFunction ).name(transform_name).returns(TypeInformation); \t\t
translation
ReshuffleTranslatorBatch
CombinePerKeyTranslator
PTransformTranslation.PAR_DO_TRANSFORM_URN
URN
input
BatchTransformTranslator
Beam API
urn1 —— ReadSourceTranslator urn2 —— ParDoStreamingTranslator urn3 —— WindowAssignTranslator urn4 —— CombinePerKeyTranslator urn5 —— GroupByKeyTranslator
CreateStreamingFlinkView.CREATE_STREAMING_FLINK_VIEW_URN
PTransformTranslation.GROUP_BY_KEY_TRANSFORM_URN
dataStreams
ParDoStreamingTranslator
WindowAssignTranslatorBatch
currentTransform
ReshuffleTranslatorStreaming
translator
FlattenPCollectionTranslator
GBKIntoKeyedWorkItemsTranslator
WindowAssignTranslator
boundedTranslator
PTransformTranslation.RESHUFFLE_URN
翻译
FlinkStreamingPipelineTranslator.StreamTransformTranslator
Transfrom 类
Beam 的API Pipeline p = Pipeline.create(options); PCollection lines = p.apply(\"ReadLines\
Translator
PipelineOptions defaultOptionsTransformHierarchy transforms
PTransformTranslation.READ_TRANSFORM_URN
CreateViewStreamingTranslator
八种
CombinePerKeyTranslatorBatch
TransformHierarchy
StreamExecutionEnvironment
GroupByKeyTranslatorBatch
GroupByKeyTranslator
Flink API StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();DataStreamWindowedValuesource=env.addSource(sourceFunction).name(transform_name).returns(TypeInformation); env.execute(\"jobName\"); \t\t
FlinkStreamingPipelineTranslator
PTransformTranslation.ASSIGN_WINDOWS_TRANSFORM_URN
ExecutionEnvironment
ReadSourceTranslator
FlinkStreamingTransformTranslators
ParDoTranslatorBatch
运行
SplittableParDo.SPLITTABLE_GBKIKWI_URN
PTransformTranslation.FLATTEN_TRANSFORM_URN
PTransformTranslation.CREATE_VIEW_TRANSFORM_URN
PipeLine
FlinkPipelineExecutionEnvironment
getTranslator(transform)
ParDoTranslationHelper
FlattenPCollectionTranslatorBatch
transforms算子树
fullName
十种
Primitive Node 叶子节点
Node
FlinkRunner
executePipeline方法
1、Pipeline p = Pipeline.create(options);
SPLITTABLE_PROCESS_URN
transfrom 对象
transform: new TransformHierarchy 树
PTransformTranslation.COMBINE_TRANSFORM_URN
translate方法
context
CreatePCollectionViewTranslatorBatch
SplittableProcessElementsStreamingTranslator
注册器接口
this.enclosingNode = null;this.transform = null;this.fullName = \"\";this.inputs =Map();this.outputs = Map();parts = new ArrayList()
ReadSourceTranslatorBatch
boolean isParallel
Transform
output
FlinkStreamingTranslationContext
FlinkBatchTransformTranslators
FlinkRunner的Translator
