MinerU 系统架构详细设计
2025-02-04 18:35:57 4 举报AI智能生成
MinerU 系统架构详细设计
作者其他创作
大纲/内容
Core Layer
QueryProcessor class
prompt_templates()
general_template: str
summary_template: str
qa_template: str
analysis_template: str
process_query()
qwen2_transform()
bge_embedding_transform()
metrics()
TextCleaner class
clean_text()
normalize_text()
DocumentReader class
supported_formats: Dict[str, List[str]]()
format_validators: Dict[str, Callable]()
read_document()
chunk_document()
metrics()
VectorProcessor class
model_configs: Dict[str, Dict]()
generate_embeddings()
process_bge()
process_bm25()
metrics()
MilvusManager class
collection_configs: Dict[str, Dict]
database_operations()
insert_vectors()
search_vectors()
delete_vectors()
update_vectors()
optimize_collection()
metrics()
connection_pool_manage()
metrics()
Support Layer
Logger class
log_config
standard_logging_methods()
rotate_logs()
metrics()
Cache class
cache_config
cache_operations()
get_cache()
clear_cache()
metrics()
ConfigManager class
config_operations
update_config()
validate_config()
metrics()
QueueManager class
queue_config
queue_operations()
pop_task()
monitor_queue_size()
metrics()
Monitor Layer
MetricsCollector class
collector_config
collect_metrics()
aggregate_metrics()
metrics()
SystemMonitor class
monitor_config
monitor_operations()
monitor_resources()
alert_system()
metrics
LoadBalancer class
balancer_config
balance_operations
check_server_health()
redirect_traffic()
metrics
Access Layer
NginxSets
nginx_cluster_config
basic_settings
http_settings
ssl_settings
gzip_settings
upstream_config
backend_servers: Dict[str, Dict]
load_balance_strategy: str
health_check_config: Dict
security_config
cache_config
metrics
connection_metrics
request_metrics
upstream_metrics
error_metrics
收藏
0 条评论
下一页
