如何优化PostgreSQL中类账本表的查询视图?
优化PostgreSQL双向影片对比视图的查询性能
问题背景
我在PostgreSQL中维护一个类账本表events,用于记录用户添加/移除影片对比的操作事件,所有事件以带时间戳的形式存储,数据行不会被删除:
CREATE TABLE events ( date timestamp with time zone NOT NULL, parent_film_id varchar(8) NOT NULL, comp_film_id varchar(8) NOT NULL, event_type varchar(20) NOT NULL );
我需要创建一个视图,展示指定film_id对应的当前有效影片对比关系,核心要求是:若影片A是B的对比项,则B也必须是A的对比项。
我尝试了以下视图创建语句:
CREATE OR REPLACE VIEW comps AS ( WITH bidirectional_events AS ( (SELECT DISTINCT ON (e.comp_film_id, e.parent_film_id) e.date, e.parent_film_id AS comp_film_id, e.comp_film_id AS parent_film_id, e.event_type FROM events AS e WHERE e.event_type = 'create' OR e.event_type = 'remove' ORDER BY e.comp_film_id, e.parent_film_id, date DESC) UNION (SELECT DISTINCT ON (parent_film_id, comp_film_id) * FROM events WHERE event_type = 'create' OR event_type = 'remove' ORDER BY parent_film_id, comp_film_id, date DESC)) SELECT date, comp_film_id, parent_film_id FROM bidirectional_events WHERE event_type = 'create');
但查询该视图中单个ID的对比关系需要数百毫秒,远慢于直接查询单个影片的所有事件(仅需数毫秒),如何优化?
现有索引情况
我已为events表添加以下索引,但对视图查询速度无明显改善:
Indexes: "comp_idx" btree (comp_film_id) "comp_parent_date_desc_idx" btree (comp_film_id, parent_film_id, date DESC) "event_idx" btree (event_type) "parent_comp_date_desc_idx" btree (parent_film_id, comp_film_id, date DESC) "parent_idx" btree (parent_film_id)
查询单个影片ID(99196)的执行计划
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 查询计划 | |-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | CTE Scan on bidirectional_events (cost=33198.98..34340.43 rows=1 width=168) (actual time=1233.483..1640.063 rows=33 loops=1) | | Output: bidirectional_events.date, bidirectional_events.comp_film_id, bidirectional_events.parent_film_id, bidirectional_events.territory_id, bidirectional_events.company_id, bidirectional_events.user_id, bidirectional_events.source_id | | Filter: (((bidirectional_events.event_type)::text = 'create'::text) AND ((bidirectional_events.comp_film_id)::text = '99196'::text)) | | 被过滤的行数: 117790 | | Buffers: shared hit=2571, temp read=1670 written=2491 | | CTE bidirectional_events | | -> Unique (cost=32171.68..33198.98 rows=45658 width=226) (actual time=1227.012..1526.222 rows=117823 loops=1) | | Output: e.date, e.parent_film_id, e.comp_film_id, e.territory_id, e.company_id, e.user_id, e.event_type, e.source_id | | Buffers: shared hit=2571, temp read=1670 written=1674 | | -> Sort (cost=32171.68..32285.82 rows=45658 width=226) (actual time=1227.009..1328.931 rows=117838 loops=1) | | Output: e.date, e.parent_film_id, e.comp_film_id, e.territory_id, e.company_id, e.user_id, e.event_type, e.source_id | | 排序键: e.date, e.parent_film_id, e.comp_film_id, e.territory_id, e.company_id, e.user_id, e.event_type, e.source_id | | 排序方法: external merge Disk: 6568kB | | Buffers: shared hit=2571, temp read=1670 written=1674 | | -> Append (cost=11140.74..23643.57 rows=45658 width=226) (actual time=298.843..1076.515 rows=117838 loops=1) | | Buffers: shared hit=2562, temp read=849 written=851 | | -> Unique (cost=11140.74..11593.50 rows=22829 width=41) (actual time=298.841..447.298 rows=58919 loops=1) | | Output: e.date, e.parent_film_id, e.comp_film_id, e.territory_id, e.company_id, e.user_id, e.event_type, e.source_id | | Buffers: shared hit=1281, temp read=424 written=425 | | -> Sort (cost=11140.74..11291.66 rows=60367 width=41) (actual time=298.838..354.656 rows=60875 loops=1) | | Output: e.date, e.parent_film_id, e.comp_film_id, e.territory_id, e.company_id, e.user_id, e.event_type, e.source_id | | 排序键: e.comp_film_id, e.parent_film_id, e.date DESC | | 排序方法: external merge Disk: 3392kB | | Buffers: shared hit=1281, temp read=424 written=425 | | -> Bitmap Heap Scan on public.events e (cost=1317.57..4488.66 rows=60367 width=41) (actual time=3.593..55.910 rows=60875 loops=1) | | Output: e.date, e.parent_film_id, e.comp_film_id, e.territory_id, e.company_id, e.user_id, e.event_type, e.source_id | | Recheck Cond: (((e.event_type)::text = 'create'::text) OR ((e.event_type)::text = 'remove'::text)) | | Heap Blocks: exact=1039 | | Buffers: shared hit=1281 | -> BitmapOr (cost=1317.57..1317.57 rows=60606 width=0) (actual time=3.457..3.461 rows=0 loops=1) | Buffers: shared hit=242 | -> Bitmap Index Scan on event_idx (cost=0.00..1263.68 rows=59635 width=0) (actual time=3.346..3.347 rows=59059 loops=1) | Index Cond: ((e.event_type)::text = 'create'::text) | Buffers: shared hit=232 | -> Bitmap Index Scan on event_idx (cost=0.00..23.70 rows=971 width=0) (actual time=0.107..0.108 rows=1816 loops=1) | Index Cond: ((e.event_type)::text = 'remove'::text) | Buffers: shared hit=10 | -> Unique (cost=11140.74..11593.50 rows=22829 width=41) (actual time=320.770..462.587 rows=58919 loops=1) | Output: events.date, events.comp_film_id, events.parent_film_id, events.territory_id, events.company_id, events.user_id, events.event_type, events.source_id | Buffers: shared hit=1281, temp read=425 written=426 | -> Sort (cost=11140.74..11291.66 rows=60367 width=41) (actual time=320.767..372.770 rows=60875 loops=1) | Output: events.date, events.comp_film_id, events.parent_film_id, events.territory_id, events.company_id, events.user_id, events.event_type, events.source_id | 排序键: events.parent_film_id, events.comp_film_id, events.date DESC | 排序方法: external merge Disk: 3400kB | Buffers: shared hit=1281, temp read=425 written=426 | -> Bitmap Heap Scan on public.events (cost=1317.57..4488.66 rows=60367 width=41) (actual time=3.279..50.067 rows=60875 loops=1) | Output: events.date, events.comp_film_id, events.parent_film_id, events.territory_id, events.company_id, events.user_id, events.event_type, events.source_id | Recheck Cond: (((events.event_type)::text = 'create'::text) OR ((events.event_type)::text = 'remove'::text)) | Heap Blocks: exact=1039 | Buffers: shared hit=1281 | -> BitmapOr (cost=1317.57..1317.57 rows=60606 width=0) (actual time=3.156..3.160 rows=0 loops=1) | Buffers: shared hit=242 | -> Bitmap Index Scan on event_idx (cost=0.00..1263.68 rows=59635 width=0) (actual time=3.044..3.045 rows=59059 loops=1) | Index Cond: ((events.event_type)::text = 'create'::text) | Buffers: shared hit=232 | -> Bitmap Index Scan on event_idx (cost=0.00..23.70 rows=971 width=0) (actual time=0.108..0.108 rows=1816 loops=1) | Index Cond: ((events.event_type)::text = 'remove'::text) | Buffers: shared hit=10 | 规划时间: 0.885 ms | 执行时间: 1644.445 ms +-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
优化方案
1. 重构视图逻辑:先过滤再处理
当前视图会先扫描全表所有事件,再做排序、去重,最后才过滤目标ID,完全浪费了索引的作用。改成先筛选出目标影片的相关事件,再处理双向配对和最新状态:
CREATE OR REPLACE VIEW comps(film_id, comp_film_id, last_updated) AS WITH target_events AS ( -- 只获取和目标影片相关的事件(作为parent或comp) SELECT date, parent_film_id, comp_film_id, event_type FROM events WHERE parent_film_id = $1 OR comp_film_id = $1 ), bidirectional_pairs AS ( -- 生成双向配对(A-B和B-A) SELECT date, parent_film_id AS film_a, comp_film_id AS film_b, event_type FROM target_events UNION ALL SELECT date, comp_film_id AS film_a, parent_film_id AS film_b, event_type FROM target_events ), latest_events AS ( -- 取每个配对的最新事件 SELECT DISTINCT ON (film_a, film_b) film_a, film_b, event_type, date FROM bidirectional_pairs ORDER BY film_a, film_b, date DESC ) -- 保留有效对比关系,排除自身对比 SELECT film_a AS film_id, film_b AS comp_film_id, date AS last_updated FROM latest_events WHERE event_type = 'create' AND film_a != film_b;
2. 优化索引设计
替换现有低效索引,创建覆盖查询条件和排序的复合索引,避免回表扫描:
-- 针对parent_film_id的查询,包含所需字段 CREATE INDEX idx_events_parent_comp_date_event ON events (parent_film_id, comp_film_id, date DESC) INCLUDE (event_type); -- 针对comp_film_id的查询,包含所需字段 CREATE INDEX idx_events_comp_parent_date_event ON events (comp_film_id, parent_film_id, date DESC) INCLUDE (event_type);
3. 改用物化视图(数据更新不频繁时)
如果影片对比关系不是实时更新,用物化视图定期刷新可大幅提升查询速度:
CREATE MATERIALIZED VIEW comps_mv AS WITH all_pairs AS ( SELECT parent_film_id AS film_a, comp_film_id AS film_b, event_type, date FROM events UNION ALL SELECT comp_film_id AS film_a, parent_film_id AS film_b, event_type, date FROM events ), latest_events AS ( SELECT DISTINCT ON (film_a, film_b) film_a, film_b, event_type, date FROM all_pairs ORDER BY film_a, film_b, date DESC ) SELECT film_a AS film_id, film_b AS comp_film_id, date AS last_updated FROM latest_events WHERE event_type = 'create' AND film_a != film_b; -- 为物化视图创建查询索引 CREATE INDEX idx_comps_mv_film_id ON comps_mv (film_id);
刷新物化视图:REFRESH MATERIALIZED VIEW comps_mv;
核心优化思路
原视图慢的根本原因是先处理全表数据,再过滤目标ID,导致大量无效计算。改成先筛选目标影片的相关事件,再处理双向配对和最新状态,可将数据量从几万行压缩到几十行,性能自然大幅提升。
内容的提问来源于stack exchange,提问作者arnfred
相关产品推荐
相关产品推荐

