MinerU 系统架构详细设计
2025-02-04 18:35:57 4 举报AI智能生成
MinerU 系统架构详细设计
作者其他创作
大纲/内容
Core Layer
QueryProcessor class
prompt_templates()
general_template: str
summary_template: str
qa_template: str
analysis_template: str
process_query()
qwen2_transform()
bge_embedding_transform()
metrics()<br>
TextCleaner class
clean_text()
normalize_text()
DocumentReader class
supported_formats: Dict[str, List[str]]()
format_validators: Dict[str, Callable]()
read_document()
chunk_document()
metrics()<br>
VectorProcessor class
model_configs: Dict[str, Dict]()<br>
generate_embeddings()<br>
process_bge()
process_bm25()
metrics()
MilvusManager class<br>
collection_configs: Dict[str, Dict]<br>
database_operations()
insert_vectors()<br>
search_vectors()<br>
delete_vectors()
update_vectors()
optimize_collection()<br>
metrics()<br>
connection_pool_manage()
metrics()
Support Layer
Logger class
log_config<br>
standard_logging_methods()
rotate_logs()<br>
metrics()<br>
Cache class
cache_config<br>
cache_operations()<br>
get_cache()
clear_cache()<br>
metrics()<br>
ConfigManager class
config_operations
update_config()
validate_config()<br>
metrics()<br>
QueueManager class
queue_config
queue_operations()<br>
pop_task()
monitor_queue_size()<br>
metrics()<br>
Monitor Layer
MetricsCollector class<br>
collector_config
collect_metrics()<br>
aggregate_metrics()<br>
metrics()<br>
SystemMonitor class
monitor_config<br>
monitor_operations()<br>
monitor_resources()
alert_system()<br>
metrics<br>
LoadBalancer class<br>
balancer_config<br>
balance_operations<br>
check_server_health()<br>
redirect_traffic()<br>
metrics
Access Layer
NginxSets
nginx_cluster_config
basic_settings<br>
http_settings<br>
ssl_settings
gzip_settings<br>
upstream_config
backend_servers: Dict[str, Dict]<br>
load_balance_strategy: str<br>
health_check_config: Dict<br>
security_config<br>
cache_config<br>
metrics<br>
connection_metrics<br>
request_metrics<br>
upstream_metrics<br>
error_metrics<br>
收藏
0 条评论
下一页
