namenode

2015-12-07 15:04:39   10  举报





NameNode是Hadoop分布式文件系统的核心组件之一，它是一个中心化的服务器，负责管理整个文件系统的元数据。在Hadoop集群中，所有存储的数据都被分割成多个块(block)，每个块都会被复制到多个节点上以实现冗余备份和容错。NameNode维护着这些块的位置信息以及它们所属的文件和目录结构等信息。客户端可以通过与NameNode通信来获取文件或目录的元数据，从而定位到需要的数据块并进行读写操作。同时，NameNode还承担着监控和管理整个集群的任务，例如检测故障节点、平衡数据负载等。总之，NameNode是Hadoop文件系统的核心，它的稳定运行对于整个系统的可靠性至关重要。

作者其他创作

大纲/内容

ServicePlugin

FileJournalManager

NNStorage

- blockpoolID- removedStorageDirs- attribute3:type

Configuration

HAState

FSEditLogLoader

+ loadFSEdits

LogsPurgeable

+ purgeLogsOlderThan+ selectInputStreams

LeaseManager

- leases- sortedLeases- sortedLeasesByPath

+ countPath+ addLease+ removeLease+ reassignLease+ renewLease- checkLeases

TransferFsImage

Monitor

FSImageStorageInspector

+ inspectDirectory+ isUpgradeFinalized+ getLatestImages+ getMaxSeenTxId+ needToSave

FSImageTransactionalStorageInspector

NameNodeHttpServer

NameNode

- namenodeRegistration- namenodeRole- haState- haEnabled- emptier

+ main- initialize- loadNamesystem- startCommonServices- stopCommonServices- startTrashEmptier- stopTrashEmptier- startHttpServer- stopHttpServer

HAServiceProtocol

+ getServiceStatus+ transitionToStandby+ transitionToActive+ monitorHealth

BookKeeperJournalManager

CheckpointerThread

NamenodeProtocol

+ rollEditLog+ getBlocks+ startCheckpoint+ endCheckpoint

Storage

- storageType- storageDirs

BlockManager

- blocksMap- corruptReplicasMap- InvalidateBlocks- postponedMisreplicatedBlocks- underReplicatedBlocks- pendingReplicationBlocks

+ activate+ close

Closeable

NamenodeProtocols

FSImageFormat.Saver

Namesystem

StandbyState

QuorumJournalManager

DatanodeStatistics

+ getCapacityTotal+ getCapacityUsed

SnapshotManager

- fsDirectory- snapshottables

NameNodeRpcServer

-serviceRpcServer-clientRpcServer

+ start+ join

FSImage

- lastAppliedTxId

+ format+ recoverTransitionRead+ saveNamespace+ loadFSImage+ loadEdits+ doImportCheckpoint+ openEditLogForWrite+ startCheckpoint+ endCheckpoint- doUpgrade- doRollback- doFinalize

BackupJournalManager

SequentialBlockIdGenerator

+ nextValue

GetUserMappingsProtocol

JournalManager

+ format+ startLogSegment+ finalizeLogSegment+ setOutputBufferCapacity+ recoverUnfinalizedSegments+ close

FSImageFormat.Loader

EditLogTailerThread

SafeModeMonitor

RefreshUserMappingsProtocol

NameNodeResourceMonitor

Runnable

QuorumCall

- successes- exceptions

FSNamesystem

- fsDirectory- fsLock

+ loadFromDisk+ startCommonServices+ stopCommonServices+ startActiveServices+ stopActiveServices+ startStandbyServices+ prepareToStopStandbyServices+ stopStandbyServices- startSecretManager- stopSecretManager

ClientProtocol

+ getBlockLocations+ create+ append+ setReplication+ abandonBlock+ addBlock+ complete+ reportBadBlocks+ renewLease+ recoverLease+ getDatanodeReport+ setSafeMode+ refreshNodes

StorageErrorReporter

HostFileManager

- includes- excludes

+ refresh+ isIncluded+ isExcluded

NameNodeResourceChecker

- volumes

+ hasAvailableDiskSpace

DecommissionManager

FSEditLog

- journalSet- editLogStream- storage- editsDirs- sharedEditsDirs- myTransactionId

+ initJournalsForWrite+ initSharedJournalsForRead+ selectInputStreams+ logEdit- startLogSegment- endCurrentLogSegment- createJournal- rollEditLog

FSImagePreTransactionalStorageInspector