启用CBO与自动统计收集时,MSCK REPAIR是否生成表统计信息?
Great question—let’s break this down clearly so you know exactly what to expect.
Core Behavior Explanation:
MSCK REPAIR TABLE <table>exists solely to sync Hive’s metastore metadata with the actual partition directories on HDFS. It adds new partitions or removes stale ones from the metastore, but it will NOT generate or update the table/partition statistics needed for CBO—even whenhive.cbo.enable=trueandhive.stats.autogather=trueare enabled.The
hive.stats.autogather=trueparameter only triggers automatic statistics collection during data-writing operations likeINSERT,LOAD DATA, orCREATE TABLE AS SELECT. Since MSCK is just a metadata synchronization tool (not a data ingestion/write action), it doesn’t trigger this auto-gather logic.To get the accurate statistics the Cost-Based Optimizer (CBO) relies on (like row counts, data sizes, and column distribution), you must explicitly run the
ANALYZEcommand:- For full table statistics:
ANALYZE TABLE <tablename> COMPUTE STATISTICS - For new partitions synced via MSCK:
ANALYZE TABLE <tablename> PARTITION (<partition_col>=<value>) COMPUTE STATISTICS
- For full table statistics:
This ensures the CBO has the data it needs to build optimized query execution plans.
内容的提问来源于stack exchange,提问作者Yaniv Irony

