flink state

2016-10-31 11:24:36   0  举报





仅支持查看

Flink State是Apache Flink流处理框架中用于管理状态的功能。在实时数据处理中，经常需要保存和更新一些数据，以便后续的计算或操作。Flink State提供了一种可靠的方式来管理这些状态信息，包括键值对、列表、映射等数据结构。通过使用Flink State，开发人员可以轻松地将状态信息持久化到外部存储系统（如RocksDB、HDFS等），以确保在故障恢复时能够正确地恢复状态。此外，Flink State还支持容错和故障转移，确保在分布式环境中的状态一致性。总之，Flink State为实时数据处理提供了强大的状态管理功能，有助于提高数据处理的准确性和可靠性。

作者其他创作

大纲/内容

- InternalKeyContext<K> keyContext- metaInfo- keySerializer

StateSnapshotRestore

+ StateSnapshot stateSnapshot()+ StateSnapshotKeyGroupReader keyGroupReader(int readVersionHint)

HeapKeyedStateBackend

HeapSnapshotStrategy[KeyedStateHandle]

+ attribute1:type = defaultValue+ attribute2:type- attribute3:type

+ operation1(params):returnType- operation2(params)- operation3()

SnapshotStrategy<S extends StateObject>

StreamOperatorStateContext

+ isRestored() : Boolean+ keyedStateBackend(): AbstractKeyedStateBackend+ operatorStateBackend(): OperatorStateBackend+ rawOperatorStateInputs(): CloseableIterable<StatePartitionStreamProvider>+ rawKeyedStateInputs(): CloseableIterable<KeyGroupStatePartitionStreamProvider>

StateObject

+ discardState() + getStateSize(): Long

State

+ clear()

KeyGroupRange

+ startKeyGroup: Int+ endKeyGroup: Int

+ iterator: Iterator<Integer>

RocksDBKeyedStateBackend

SnapshotResult<T extends StateObject>

+ T jobManagerOwnedSnapshot+ T taskLocalSnapshot

InternalKeyContext[K]

+ K getCurrentKey()+ int getCurrentKeyGroupIndex()+ int getNumberOfKeyGroups()+ KeyGroupRange getKeyGroupRange()+ void setCurrentKey(@Nonnull K currentKey)+ void setCurrentKeyGroupIndex(int currentKeyGroupIndex)

RocksDBSnapshotStrategyBase

KeyedStateHandle

+ KeyGroupRange getKeyGroupRange()+ KeyedStateHandle getIntersection(KeyGroupRange keyGroupRange)

- keyContext: InternalKeyContext[K]

StreamTaskStateInitializer

+ streamOperatorStateContext(): StreamOperatorStateContext

OperatorStateStore

CheckpointListener

+ void notifyCheckpointComplete(long checkpointId)

HeapSnapshotStrategy

StreamStateHandle

+ FSDataInputStream openInputStream()