如何用pg_hint_plan定义双连接方向?多条件查询优化求助
我负责数据库运维,但无法修改应用传入的查询语句。当前涉及多张大表的查询中,PostgreSQL误判表的选择性,生成了非最优执行计划,此前我用pg_hint_plan解决过其他查询问题,但本次遇到了困难。
该查询用于搜索产品标题、产品标题的德语翻译,或无德语翻译时的英语翻译,SQL语句如下:
select * from product left outer join translation as t on (product.textnr = t.textnr and ( (t.language = 'DE' and t.text <> '') or (t.language = 'EN' and t.text <> '' and not exists ( select text from translation where translation.textnr = product.textnr and (translation.language = 'DE') and translation.text <> '')) ) ) where (product.title = 'query' or translation.text like 'query%')
该查询执行时长约40秒,我遇到的具体问题:
- 仅存在
product.title = 'query'条件时,通过/*+ Leading((product t)) NestLoop(product t) */先过滤product表再关联translation,查询可在100ms内完成; - 仅存在
translation.text like 'query%'条件时,通过/*+ Leading((t product )) NestLoop(t product) */反向过滤关联,查询也可在约100ms完成; - 但当两个条件同时存在时,由于textnr上的左外连接,我无法确定查询起始点。
我想知道是否可以让PostgreSQL/pg_hint_plan对同一基础表的两个不同连接结果做Hash Join?比如尝试使用如下hint:
/*+ Leading((t product ) (product t)) HashJoin ( NestLoop(t product) NestLoop(product t) )*/
但此写法会报错:pg_hint_plan: hint syntax error at or near "(product t))"
附执行计划:
QUERY PLAN | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Hash Join (cost=1693149.03..35181808.38 rows=25422 width=187) (actual time=19191.927..45245.126 rows=28 loops=1) | Output: product.textnr, katalog[...] Hash Cond: ((katalog.[some additional table ...] Buffers: shared hit=25454832 read=160622, temp read=26535 written=26535 | I/O Timings: shared/local read=501.711, temp read=45.572 write=184.356 | -> Nested Loop (cost=1689535.09..35177844.89 rows=25422 width=187) (actual time=13131.144..45210.910 rows=64 loops=1) | Output: product.textnr, katalog[...] Buffers: shared hit=25454799 read=159756, temp read=26535 written=26535 | I/O Timings: shared/local read=497.810, temp read=45.572 write=184.356 | -> Hash Left Join (cost=1689534.65..35132069.03 rows=18000 width=69) (actual time=13130.004..45153.182 rows=89 loops=1) | Output: product.textnr, product.... Hash Cond: ((product.textnr)::text = (t.textnr)::text) | Join Filter: (((t.language)::text = 'DE'::text) OR (((t.language)::text = 'EN'::text) AND (NOT (SubPlan 1)))) | Rows Removed by Join Filter: 3998465 | Filter: (((product.title)::text = 'query'::text) OR (t.text)::text) ~~ 'query%'::text)) | Rows Removed by Filter: 8088550 | Buffers: shared hit=25454564 read=159664, temp read=26535 written=26535 | I/O Timings: shared/local read=440.991, temp read=45.572 write=184.356 | -> Seq Scan on public.product (cost=0.00..375433.59 rows=8119159 width=69) (actual time=0.006..3219.008 rows=8088639 loops=1) | Output: product.title, ... Buffers: shared hit=179827 read=114415 | I/O Timings: shared/local read=322.814 | -> Hash (cost=1595405.93..1595405.93 rows=4049898 width=63) (actual time=8272.352..8272.353 rows=1628548 loops=1) | Output: t.textnr, t.language, t.text | Buckets: 1048576 Batches: 8 Memory Usage: 24151kB | Buffers: shared hit=1278468 read=44483, temp written=11703 | I/O Timings: shared/local read=102.369, temp write=71.090 | -> Index Scan using translation2 on public.translation t (cost=0.69..1595405.93 rows=4049898 width=63) (actual time=326.774..7775.822 rows=1628548 loops=1) | Output: t.textnr, t.language, t.text | Index Cond: [additional join condition] | Filter: ((t.text::text) <> ''::text) AND (((t.language)::text = 'DE'::text) OR ((t.language)::text = 'EN'::text))) | Rows Removed by Filter: 5377105 | Buffers: shared hit=1278468 read=44483 | I/O Timings: shared/local read=102.369 | SubPlan 1 | -> Index Scan using translation_pkey on public.translation (cost=0.69..3.42 rows=2 width=0) (actual time=0.007..0.007 rows=1 loops=3998465) | Index Cond: ((([additional join condition]) AND ((translation.textnr)::text = (product.textnr)::text) AND ((translation.language)::text = 'DE'::text)) | Filter: (upper((translation.text)::text) <> ''::text) | Buffers: shared hit=23993142 read=766 | I/O Timings: shared/local read=15.808 | -> Index Scan using katalog3 on public.katalog (cost=0.43..2.14 rows=40 width=132) (actual time=0.565..0.646 rows=1 loops=89) | Output: katalog.[...]| Index Cond: [...] | Buffers: shared hit=235 read=92 | I/O Timings: shared/local read=56.818 | -> Hash (cost=2105.64..2105.64 rows=120664 width=17) (actual time=34.023..34.024 rows=120169 loops=1) | Output: [additional data table colums...] | Buckets: 131072 Batches: 1 Memory Usage: 6820kB | Buffers: shared hit=33 read=866 | I/O Timings: shared/local read=3.901 | -> Seq Scan on public.[additional data table] (cost=0.00..2105.64 rows=120664 width=17) (actual time=0.861..13.874 rows=120169 loops=1) | Output: [additional data table colums...] | Buffers: shared hit=33 read=866 | I/O Timings: shared/local read=3.901 | Settings: effective_cache_size = '16094504kB', jit = 'off', random_page_cost = '1.1', temp_buffers = '64MB', work_mem = '32MB' | Query Identifier: 703631274929839679 | Planning: | Buffers: shared hit=63 | Planning Time: 1.220 ms | Execution Time: 45245.280 ms |
首先你要明确:pg_hint_plan的语法不支持直接对“两个独立连接结果集”指定Hash Join,它的hint是基于表的连接顺序和方式,而非子结果集,所以你之前的写法会报错。针对这个场景,有三个可行的解决方向:
1. 拆分查询逻辑,用Union All合并高效结果
虽然不能修改应用的原SQL,但可以创建物化视图或存储函数,把原查询拆成两个你已经验证过的高效分支,再合并结果:
-- 分支1:匹配product.title的结果,用你已验证的高效计划 select * from product left outer join translation as t on (product.textnr = t.textnr and ( (t.language = 'DE' and t.text <> '') or (t.language = 'EN' and t.text <> '' and not exists ( select text from translation where translation.textnr = product.textnr and (translation.language = 'DE') and translation.text <> '')) ) ) where product.title = 'query' union all -- 分支2:匹配translation.text的结果,排除已在分支1出现的记录 select * from product left outer join translation as t on (product.textnr = t.textnr and ( (t.language = 'DE' and t.text <> '') or (t.language = 'EN' and t.text <> '' and not exists ( select text from translation where translation.textnr = product.textnr and (translation.language = 'DE') and translation.text <> '')) ) ) where translation.text like 'query%' and product.title != 'query'
之后让应用调用这个物化视图或函数,两个分支都会走你确认过的100ms级执行计划,最后合并结果即可。
2. 用pg_hint_plan强制单一路径
如果必须保留原SQL结构,直接强制走其中一个高效路径即可——比如如果product.title='query'的结果集远小于translation.text like 'query%'的结果,就强制先走product表:
/*+ Leading((product t)) NestLoop(product t) IndexScan(t translation2) */ select * from product left outer join translation as t on (product.textnr = t.textnr and ( (t.language = 'DE' and t.text <> '') or (t.language = 'EN' and t.text <> '' and not exists ( select text from translation where translation.textnr = product.textnr and (translation.language = 'DE') and translation.text <> '')) ) ) where (product.title = 'query' or translation.text like 'query%')
反之,如果translation分支的结果更小,就强制先走translation表:
/*+ Leading((t product)) NestLoop(t product) IndexScan(t translation2) IndexScan(product product_title_idx) -- 这里替换成product表上title字段的索引名 */
这种方式不需要处理结果集合并,直接让PostgreSQL按你指定的高效路径执行。
3. 更新统计信息,让PostgreSQL自动选优
PostgreSQL误判选择性大多是因为统计信息过时或采样不足,先尝试更新表的统计信息:
ANALYZE product; ANALYZE translation;
如果表数据量极大,可以提高采样率,让统计信息更准确:
ANALYZE product (title, textnr) WITH (sample_rate = 0.1); ANALYZE translation (textnr, language, text) WITH (sample_rate = 0.1);
更新后PostgreSQL可能会自动生成最优执行计划,不需要依赖hint。
内容的提问来源于stack exchange,提问作者Harald

