Flink uid和uidHash
2021-05-25 23:49:09 0 举报
AI智能生成
flink 源码
作者其他创作
大纲/内容
Flink uid和uidHashMatching Operator State
官方文档
uid
设置org.apache.flink.api.dag.Transformation#setUid
uidHash
设置org.apache.flink.api.dag.Transformation#setUidHash
设置org.apache.flink.streaming.api.graph.StreamGraphGenerator#transform
defaultStreamGraphHasher生成hashes
StreamGraphHasherV2
generateNodeHash判断用户是否设置uidgenerateDeterministicHash orgenerateUserSpecifiedHash
* Generates a deterministic hash from node-local properties and input and output edges.有可能重复
* Generates a hash from a user-specified ID.
legacyStreamGraphHashers生成legacyHashes
StreamGraphUserHashHasher使用用户提供的uid hash
traverseStreamGraphAndGenerateHashes
org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator#createJobVertex
new JobVertexID(hash)
legacyJobVertexIds
JobVertex
JobVertexID
generateDeterministicHash ps 有可能重复,不可靠,所以建议设置uid
idAlternatives
org.apache.flink.runtime.checkpoint.CheckpointCoordinator#restoreSavepoint
org.apache.flink.runtime.checkpoint.Checkpoints#loadAndValidateCheckpoint
org.apache.flink.runtime.checkpoint.savepoint.SavepointV2#convertToOperatorStateSavepointV2
如果使用job vertex id 找不到就考虑使用 legacy job vertex
org.apache.flink.runtime.executiongraph.ExecutionJobVertex#includeLegacyJobVertexIDs
0 条评论
回复 删除
下一页