PostgreSQL分区表批处理CPU占满:执行计划异常求助
分区表批处理查询性能异常问题
我有一个每日处理约5万行数据的批处理任务,执行的查询如下(涉及分区表):
select * from billing abstractbi0_ inner join sale sale1_ on abstractbi0_.sale_billing_date = sale1_.billing_date and abstractbi0_.sale_id = sale1_.id where abstractbi0_.dtype in ('INVOICE_CORRECTED_BEFORE_BILLING', 'INVOICE', 'GOODWILL', 'REFUND', 'ZERO_SUM') and sale1_.proposal_id = '47037059-d231-40d9-a0f5-242577596b5c' and abstractbi0_.billing_date = '2022-09-27' and sale1_.billing_date = '2022-09-27';
每次批处理启动时CPU占用率达到100%,性能极差;在高CPU阶段执行analyze后,性能提升约20倍。
执行analyze前的执行计划:
Nested Loop (cost=0.85..14.67 rows=1 width=574) Join Filter: (abstractbi0_.sale_id = sale1_.id) -> Index Scan using billing_p2022_09_sale_billing_date_sale_id_idx on billing_p2022_09 abstractbi0_ (cost=0.43..8.45 rows=1 width=467) Index Cond: (sale_billing_date = '2022-09-27'::date) Filter: ((billing_date = '2022-09-27'::date) AND ((dtype)::text = ANY ('{INVOICE_CORRECTED_BEFORE_BILLING,INVOICE,GOODWILL,REFUND,ZERO_SUM}'::text[]))) -> Index Scan using sale_p2022_09_pkey on sale_p2022_09 sale1_ (cost=0.43..6.20 rows=1 width=107) Index Cond: (billing_date = '2022-09-27'::date) Filter: (proposal_id = '47037059-d231-40d9-a0f5-242577596b5c'::uuid)
执行analyze后的执行计划:
Nested Loop (cost=0.85..16.91 rows=1 width=1084) -> Index Scan using sale_p2022_09_billing_date_proposal_id_idx on sale_p2022_09 sale1_ (cost=0.43..8.45 rows=1 width=107) Index Cond: ((billing_date = '2022-09-27'::date) AND (proposal_id = '47037059-d231-40d9-a0f5-242577596b5c'::uuid)) -> Index Scan using billing_p2022_09_sale_billing_date_sale_id_billing_date_dty_idx on billing_p2022_09 abstractbi0_ (cost=0.43..8.46 rows=1 width=977) Index Cond: ((sale_billing_date = '2022-09-27'::date) AND (sale_id = sale1_.id) AND (billing_date = '2022-09-27'::date)) Filter: ((dtype)::text = ANY ('{INVOICE_CORRECTED_BEFORE_BILLING,INVOICE,GOODWILL,REFUND,ZERO_SUM}'::text[]))
第二个执行计划不会导致CPU占满,且速度比第一个快20倍,但次日批处理时又会回到糟糕的执行计划,CPU再次占满。
由于涉及分区表,根据文档autovacuum不支持分区表,我尝试了以下方案均无效:
- 仅依赖autovacuum
- 结合autovacuum定期执行vacuum analyze
- 禁用autovacuum后定期执行vacuum analyze
目前我仅能考虑调整某个cpu_*_cost参数来强制优化器选择第二个执行计划,但担心会影响全局执行计划。请问还有其他解决方案吗?注:查询由Spring/JPA自动生成。
内容的提问来源于stack exchange,提问作者user505877
相关产品推荐
相关产品推荐

