键值结构Cust_Property表同步Customer表的高效实现方案
问题背景
我有一个名为Customer的客户主表,该表所有数据均来源于键值结构表Cust_Property。
Cust_Property表共包含3列:
CustomerID, Property, Value
Property列存储属性名,例如值为First_Name时对应Value为John,属于预透视的表结构。我需要将Cust_Property表中关联的属性值,同步更新到Customer表的对应列中。
同步规则
- 若
Cust_Property表中出现Customer表不存在的新CustomerID,需在Customer表新增对应行,并填充所有匹配的属性值。 Customer表的所有数据在Cust_Property表中均有对应记录,因此无需全量更新,仅同步新增或发生变更的记录即可。Customer表仅执行新增、更新操作,不执行删除操作。Cust_Property表中若存在Customer表无对应列的属性,直接忽略即可。
测试表DDL与初始数据
CREATE TABLE #Customer ( Customerid int, FirstName varchar(50), LastName varchar(50), Address1 varchar(100), Address2 varchar(100), Address3 varchar(100) ) CREATE TABLE #Cust_Property ( CustomerID int, Property varchar(50), Value varchar(50) ) INSERT INTO #Customer (Customerid, FirstName, LastName, Address1, Address2, Address3) VALUES(1, N'John', N'Smith', N'123 happy lane', NULL, NULL); INSERT INTO #Customer (Customerid, FirstName, LastName, Address1, Address2, Address3) VALUES(2, N'Dwight', N'Schrute', N'33 1st Ave', N'Apt 5', NULL); INSERT INTO #Customer (Customerid, FirstName, LastName, Address1, Address2, Address3) VALUES(3, NULL, NULL, NULL, NULL, NULL); INSERT INTO #Cust_Property (CustomerID, Property, Value) VALUES(3, N'First_Name', N'Michael'); INSERT INTO #Cust_Property (CustomerID, Property, Value) VALUES(3, N'Last_Name', N'Scott'); INSERT INTO #Cust_Property (CustomerID, Property, Value) VALUES(8, N'First_Name', N'Jim'); INSERT INTO #Cust_Property (CustomerID, Property, Value) VALUES(8, N'Last_Name', N'Halpert'); INSERT INTO #Cust_Property (CustomerID, Property, Value) VALUES(8, N'Address1', N'644 Scranton Rd'); INSERT INTO #Cust_Property (CustomerID, Property, Value) VALUES(8, N'Nickname', N'Jimmy'); INSERT INTO #Cust_Property (CustomerID, Property, Value) VALUES(1, N'First_Name', N'John');
测试表初始数据
Customer表初始数据:
| CustomerID | FirstName | LastName | Address1 | Address2 | Address3 |
|---|---|---|---|---|---|
| 1 | John | Smith | 123 happy lane | 空 | 空 |
| 2 | Dwight | Schrute | 33 1st Ave | Apt 5 | 空 |
| 3 | 空 | 空 | 空 | 空 | 空 |
Cust_Property表初始数据:
| CustomerID | Property | Value |
|---|---|---|
| 3 | First_Name | Michael |
| 3 | Last_Name | Scott |
| 8 | First_Name | Jim |
| 8 | Last_Name | Halpert |
| 8 | Address1 | 644 Scranton Rd |
| 8 | Nickname | Jimmy |
| 1 | First_Name | John |
预期同步结果
- 更新CustomerID为3的记录的
First_Name、Last_Name列值 - 新增CustomerID为8的客户记录(该ID在原Customer表不存在)
- 填充客户8的所有匹配属性,忽略
Nickname属性(因Customer表无对应列) - 忽略CustomerID为1的
First_Name属性更新,因为该值与Customer表现有值一致,无需更新
当前实现方案
首先查找并插入不存在的新CustomerID,代码如下:
INSERT INTO #Customer (CustomerID) SELECT DISTINCT CustomerID FROM #Cust_Property a WHERE NOT EXISTS (SELECT * FROM #Customer x WHERE a.CustomerID = x.CustomerID)
插入完成后,逐列编写UPDATE语句更新每个属性:
UPDATE #Customer SET #Customer.FirstName = a.Value FROM #Cust_Property a WHERE #Customer.CustomerID = a.CustomerID AND a.Property = 'First_Name' UPDATE #Customer SET #Customer.LastName = a.Value FROM #Cust_Property a WHERE #Customer.CustomerID = a.CustomerID AND a.Property = 'Last_Name' UPDATE #Customer SET #Customer.Address1 = a.Value FROM #Cust_Property a WHERE #Customer.CustomerID = a.CustomerID AND a.Property = 'Address1' UPDATE #Customer SET #Customer.Address2 = a.Value FROM #Cust_Property a WHERE #Customer.CustomerID = a.CustomerID AND a.Property = 'Address2' UPDATE #Customer SET #Customer.Address3 = a.Value FROM #Cust_Property a WHERE #Customer.CustomerID = a.CustomerID AND a.Property = 'Address3'
问题诉求
实际生产场景中,Cust_Property表包含数百种不同属性,Customer表有40个需要同步的列,客户数据量约200万条。除了为每个列单独编写UPDATE语句外,是否存在更高效的实现方案?当前无法使用ETL工具,若效率更高也可采用Python实现。
内容的提问来源于stack exchange,提问作者Crescent
相关产品推荐
相关产品推荐

