1.流处理概念(流处理的常见概念)。
2020-10-16 10:09:29 0 举报AI智能生成
Apache Flink
作者其他创作
大纲/内容
1.流处理概念(流处理的常见概念)
有界和无界数据和处理器
延迟和吞吐量
时间语义
处理时间
事件时间
水印
窗口
触发
2.体系
分层API
SQL/Table API
DtaStream
ProcessFunction
组件
JobManagers
Task Managers
Clients
任务执行
Operator
Task
Setting Parallelism
Operator Level
Execution Environment Level
Client Level
System Level
是
Restart Strategies
Filover Strategy
Slots and Resources
5.Libraries
CEP
State Processor API
Gelly
7.运行和部署
部署模式
Local Cluster
Standalone Cluster
YARN
Mesos
Docker
Kubernates
High Availability
Standalone Cluster High Availability
YARN Cluster High Availability
Command-Line Interface
Job Submission
Job Management
Savepoints
Python REPL
Scala REPL
Security
Kerberos
SSL
File Systems
8.调试和监控
指标
日志
Flink Web UI
Monitoring Checkpointing
Monitoring Back Pressure
Monitoring Watemark
Flink Rest API
Application Profiling
3.State Management
状态后端
MemotySateBackend
FsStateBackend
RocketDBStateBackend
Flink中的各种状态
Operator State
List state
Union List state
Broadcast state
Keyed State
Value state
List state
Map state
容错
Checkpointing
Barriers
Exactly Once&At Least Once
异步状态快照
检查点算法
增量检查点
Savepoints
State Rescalable
Queryable Satte
State Schema Evolution
State Processor API
4. DataStream
Setup Environment
Source
Built-in Source(readTextFile)
Custom Source(addSource)
Sink
Built-in Sink(writeAsText)
Custom Sink(addSink)
Transformations
Basic Transformations
Map
Filter
FlatMap
KeyedStream Transformations
KeyBy
Aggregations
Reduce
MultiStream Transformations
Union
Connect,coMap,coFlatMap
Split&select
Distribution Transformations
Random
Round-Robin
Random
Rescale
Broadcast
Grobal
Custom
Date Types
Types
TypeInformation
Function
Iterations
Time-Based and Window Operators
Time characteristics
Processing Time
Event Time
IngestionTime
Assign Timestamps and generating watermarks
ProcessFunction
TimeService and Timer
Emitting to side Outputs
Windows
Window Assigners
Window Funcitons
Triggers
Evictors
Allowed Latedness
Side Output
Joining
Window Join
Tumbling Window Join
Sliding Window Join
Session Window Join
Interval Join
Async I/O
6. Tabe API&SQL
Streaming Concepts
Dynamic Tbales
Time Attributes
Processing Time
Event Time
Temporal Tables
Query Configuration
SQL
DDL
Create Table/Databases/Function
Drop Create Table/Databases/Function
Alter Create Table/Databases/Function
Query
Operators(Select/Filter/Aggregations/joins/etc.)
SQL Client
Table API
Java/ScalaTable API
Python Table API
Functions
Built-in Functions
Scalar Functions
Aggregate Functions
Column Functions
User Defined Functions
Scalar Functions
Table Functions
Aggregation Functions
Table Aggragation Functions
Connect to External Systems
Table Connectors
Tbale Formats
Tbale Schema
Update Modes
User-defined Table Sources & Sinks
Planners
Flink Planner
Blink Planner
Modules
Catalog
Register an Enternal Catalog
Register Tables in Catalog
Hive Integration
9.生态
Connectors
FileSystem
Hive
HDFS
Queue
Apache Kafka
Amazon Kinesis Streams
RabbitMQ
Key-Value
Cassandra
Redis
Hbase
Search Engine
Elasticsearch
Service
Twitter
第三方组件
10.案例
应用类型
事件驱动
欺诈检测
异常检测
基于规则的提醒
业务流程监控
Web application
数据分析
电信网络的质量监控
在消费者技术中实时数据分析
产品更新分析与实验评估分析
大规模图分析
数据管道
电子商务中实时搜索索引的建立
电子商务ETL
域类型
Datawarehouse
Olap
AI
公司
收藏
0 条评论
下一页
