sparkt 任务提交流程
2019-07-10 09:05:30 51 举报
登录查看完整内容
spark 任务提交流程
作者其他创作
大纲/内容
container(executor)
9
5
submit job1
1
Master
. . .
2
zookeeper
10
Excutor
DAGscheduler
Container
Spark Task
7
ResourceManager
Client
Worker
4
time
Scheduler
StandbyResource Manager
Resouece Manager(RM)
Node Maneger(NM)
. . .
standalone
job2
Client
Yarn HA
8
Capacity Scheduler
ApplicationsManager
job1
submit job2
Spark-submit
Job2
Spark:yarn-cluster
YARN Cluster Scheduler
Node Manager
Applications Manager
6
Yarn 的运作流程
Driver
Application Master(AM)
Zookeeper Cluster
Spark:yarn-client
FIFO Scheduler
Spark Context
AppMaster
queue B
3
client
Job1
queue A
Applications Manager(ASM)
Spark Application Master
ActiveResource Manager
queue default
0 条评论
回复 删除
下一页
