SQL技术需求:生成T1的TITLE二元组并匹配T2的DESCRIPTION
SQL实现:从文本提取Bigram并匹配关联表内容
问题背景
现有两张表:
- T1:仅含
TITLE字段,存储空格分隔的文本 - T2:含
NAME、DESCRIPTION字段,DESCRIPTION包含无空格拼接的连续词对
需要从T1的TITLE中提取所有相邻词对(拼接成无空格的Bigram),找出在T2的DESCRIPTION中出现过的Bigram,返回对应的TITLE、DESCRIPTION及匹配的Bigram。
示例数据
T1表
| TITLE |
|---|
| age 5 alton john live |
| show must go on |
T2表
| NAME | DESCRIPTION |
|---|---|
| John Bo | altonjohn for kids |
| Alton | show age5 mustgo kids |
预期输出
| TITLE | DESCRIPTION | BIGRAM |
|---|---|---|
| age 5 alton john live | altonjohn for kids | altonjohn |
| age 5 alton john live | show age5 mustgo kids | age5 |
| show must go on | show age5 mustgo kids | mustgo |
解决方案(以PostgreSQL为例)
WITH t1_bigrams AS ( SELECT title, CONCAT(word, next_word) AS bigram FROM ( SELECT title, word, lead(word) OVER (PARTITION BY title ORDER BY idx) AS next_word FROM ( SELECT title, unnest(string_to_array(title, ' ')) AS word, row_number() OVER (PARTITION BY title) AS idx FROM t1 ) split_words ) bigram_pairs WHERE next_word IS NOT NULL ) SELECT tb.title, t2.description, tb.bigram FROM t1_bigrams tb JOIN t2 ON t2.description LIKE '%' || tb.bigram || '%' ORDER BY tb.title, t2.description;
代码说明
- 分割文本生成单词列表:用
string_to_array把TITLE拆成单词数组,unnest展开成单行记录,row_number标记每个单词的位置 - 生成Bigram:通过
lead窗口函数获取每个单词的下一个相邻单词,用CONCAT拼接成无空格的词对 - 关联匹配:将生成的Bigram与T2的
DESCRIPTION做模糊匹配,关联后输出目标字段
其他数据库适配提示
- MySQL:可使用递归CTE实现字符串分割,再结合变量或窗口函数生成Bigram
- SQL Server:利用
STRING_SPLIT分割文本,配合LEAD函数生成词对,逻辑与上述一致
内容的提问来源于stack exchange,提问作者userit1985
相关产品推荐
相关产品推荐

