MySQL中JSON字段导致需临时表的SELECT查询性能骤降问题咨询
MySQL JSON字段引发临时表性能问题分析
问题现象
我在MySQL的一张表中新增了一个JSON字段,每行存储约7KB的数据,该表共有约13.5万行数据。添加此字段前,所有查询执行时间均远低于1秒;但现在当EXPLAIN显示需要临时表时,查询耗时约7秒。若将该字段改为TEXT类型,相同查询耗时又回到1秒以内。
核心疑问
- 为何JSON字段会引发如此严重的性能问题?
- 随着JSON字段数据量增长,是否应将其拆分至一对一关联表?
- 若后续需基于JSON字段内的值进行查询,此方案是否会存在问题?
关键背景信息
- 我并未查询或返回该JSON字段;
- 用户对该表的查询方式多样,为所有场景创建索引以避免临时表并不现实;
- 多数查询由应用的Hibernate ORM代码动态生成。
示例查询
(JSON字段时耗时7秒,TEXT字段时耗时不足1秒)
SELECT xaction.XactionId, xaction.PropStreet, xaction.PropNum, xaction.PropStreetUnit FROM Xaction xaction INNER JOIN Team team ON xaction.TeamId = team.TeamId LEFT OUTER JOIN AppUser appuser_primaryAgent ON xaction.AppUserIdPrimaryAgent = appuser_primaryAgent.AppUserId LEFT OUTER JOIN AppUser appuser_coAgent ON xaction.AppUserIdCoAgent = appuser_coAgent.AppUserId LEFT OUTER JOIN AppUser appuser_assistant1 ON xaction.AppUserIdAssistant1 = appuser_assistant1.AppUserId LEFT OUTER JOIN AppUser appuser_assistant2 ON xaction.AppUserIdAssistant2 = appuser_assistant2.AppUserId WHERE team.TeamId = 1 AND ( appuser_primaryAgent.AppUserId = 1 or appuser_coAgent.AppUserId = 1 or appuser_assistant1.AppUserId = 1 or appuser_assistant2.AppUserId = 1 ) GROUP BY xaction.XactionId ORDER BY PropStreet , PropNum , PropStreetUnit
执行计划
+----+-------------+----------------------+------------+--------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+-------------------------------------------+-------+----------+----------------------------------------------+ | id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | +----+-------------+----------------------+------------+--------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+-------------------------------------------+-------+----------+----------------------------------------------+ | 1 | SIMPLE | team | NULL | const | PRIMARY | PRIMARY | 8 | const | 1 | 100.00 | Using index; Using temporary; Using filesort | | 1 | SIMPLE | xaction | NULL | index | PRIMARY,IDX_PropNum,IDX_PropStreet,AppUserIdPrimaryAgent,AppUserIdCoAgent,XactionPropTypeId,XactionSourceId,XactionStatusId,IDX_PropNum_PropStreet_PropStreetNum,AppUserIdAssistant1,AppUserIdAssistant2 | PRIMARY | 8 | NULL | 49515 | 10.00 | Using where | | 1 | SIMPLE | appuser_primaryAgent | NULL | eq_ref | PRIMARY,AppUserId_idx | PRIMARY | 8 | afdata_next.xaction.AppUserIdPrimaryAgent | 1 | 100.00 | Using index | | 1 | SIMPLE | appuser_coAgent | NULL | eq_ref | PRIMARY,AppUserId_idx | PRIMARY | 8 | afdata_next.xaction.AppUserIdCoAgent | 1 | 100.00 | Using index | | 1 | SIMPLE | appuser_assistant1 | NULL | eq_ref | PRIMARY,AppUserId_idx | PRIMARY | 8 | afdata_next.xaction.AppUserIdAssistant1 | 1 | 100.00 | Using index | | 1 | SIMPLE | appuser_assistant2 | NULL | eq_ref | PRIMARY,AppUserId_idx | PRIMARY | 8 | afdata_next.xaction.AppUserIdAssistant2 | 1 | 100.00 | Using where; Using index | +----+-------------+----------------------+------------+--------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------+---------+-------------------------------------------+-------+----------+----------------------------------------------+ 6 rows in set, 1 warning (0.00 sec)
参考表结构
CREATE TABLE `xaction` ( `XactionId` bigint NOT NULL AUTO_INCREMENT, `PropNum` varchar(20) NOT NULL, `PropStreetDir` varchar(10) DEFAULT NULL, `PropStreet` varchar(100) NOT NULL, `PropStreetUnit` varchar(20) DEFAULT NULL, `City` varchar(100) DEFAULT NULL, `State` varchar(4) DEFAULT NULL, `Zip` varchar(10) DEFAULT NULL, `County` varchar(100) DEFAULT NULL, `MlsId` varchar(50) DEFAULT NULL, `TaxId` varchar(50) DEFAULT NULL, `GfNum` varchar(50) DEFAULT NULL, `ListPrice` decimal(13,4) DEFAULT NULL, `ListPriceOriginal` decimal(13,4) DEFAULT NULL, `SqFt` int DEFAULT NULL, `SqFtSource` varchar(50) DEFAULT NULL, `Beds` smallint DEFAULT NULL, `Baths` decimal(4,2) DEFAULT NULL, `YearBuilt` smallint DEFAULT NULL, `LotSize` varchar(100) DEFAULT NULL, `Schools` varchar(255) DEFAULT NULL, `Subdivision` varchar(255) DEFAULT NULL, `LockBoxId` varchar(20) DEFAULT NULL, `LockBox` varchar(255) DEFAULT NULL, `SecurityCode` varchar(255) DEFAULT NULL, `HoaFee` varchar(100) DEFAULT NULL, `HoaFrequency` varchar(100) DEFAULT NULL, `Occupancy` varchar(50) DEFAULT NULL, `Remarks` mediumtext, `Instructions` mediumtext, `ListOtherInfo` mediumtext, `ContractPrice` decimal(13,4) DEFAULT NULL, `OtherParty` varchar(500) DEFAULT NULL, `EarnestMoney` varchar(255) DEFAULT NULL, `DueDiligenceFee` varchar(255) DEFAULT NULL, `Concessions` varchar(255) DEFAULT NULL, `Financing` varchar(50) DEFAULT NULL, `SpecialProvisions` mediumtext, `ContractOtherInfo` mediumtext, `Possession` mediumtext, `EffectiveDate` date DEFAULT NULL, `ClosingDate` date DEFAULT NULL, `ListDate` date DEFAULT NULL, `ExpireDate` date DEFAULT NULL, `ClosedDate` date DEFAULT NULL, `XactionPropTypeId` bigint DEFAULT NULL, `XactionSourceId` bigint DEFAULT NULL, `XactionSide` varchar(10) NOT NULL, `XactionStatusId` bigint NOT NULL, `PercentageCommission` decimal(6,5) DEFAULT NULL, `CommissionNote` varchar(255) DEFAULT NULL, `SplitTeamLead` decimal(6,5) DEFAULT NULL, `SplitPrimaryAgent` decimal(6,5) DEFAULT NULL, `SplitCoAgent` decimal(6,5) DEFAULT NULL, `SplitAssistant1` decimal(6,5) DEFAULT NULL, `SplitAssistant2` decimal(6,5) DEFAULT NULL, `PayoutEstimated` decimal(13,4) DEFAULT NULL, `PayoutActual` decimal(13,4) DEFAULT NULL, `PayoutReferral` decimal(13,4) DEFAULT NULL, `PayoutBroker` decimal(13,4) DEFAULT NULL, `PayoutTeamLead` decimal(13,4) DEFAULT NULL, `PayoutPrimaryAgent` decimal(13,4) DEFAULT NULL, `PayoutCoAgent` decimal(13,4) DEFAULT NULL, `PayoutAssistant1` decimal(13,4) DEFAULT NULL, `PayoutAssistant2` decimal(13,4) DEFAULT NULL, `ExpensesBroker` decimal(13,4) DEFAULT NULL, `ExpensesTeamLead` decimal(13,4) DEFAULT NULL, `GeoLatitude` decimal(9,7) DEFAULT NULL, `GeoLongitude` decimal(10,7) DEFAULT NULL, `GeoLocatorStatus` varchar(10) DEFAULT NULL, `TimeZone` varchar(60) NOT NULL, `FieldDataJson` text, `CreateDateTime` datetime NOT NULL, `EditDateTime` datetime NOT NULL, `TeamId` bigint NOT NULL, `AppUserIdPrimaryAgent` bigint NOT NULL, `AppUserIdCoAgent` bigint DEFAULT NULL, `AppUserIdAssistant1` bigint DEFAULT NULL, `AppUserIdAssistant2` bigint DEFAULT NULL, PRIMARY KEY (`XactionId`), KEY `IDX_PropNum` (`PropNum`), KEY `IDX_PropStreet` (`PropStreet`), KEY `AppUserIdPrimaryAgent` (`AppUserIdPrimaryAgent`), KEY `AppUserIdCoAgent` (`AppUserIdCoAgent`), KEY `XactionPropTypeId` (`XactionPropTypeId`), KEY `XactionSourceId` (`XactionSourceId`), KEY `XactionStatusId` (`XactionStatusId`), KEY `IDX_PropNum_PropStreet_PropStreetNum` (`PropNum`,`PropStreet`,`PropStreetUnit`), KEY `AppUserIdAssistant1` (`AppUserIdAssistant1`), KEY `AppUserIdAssistant2` (`AppUserIdAssistant2`), CONSTRAINT `FK_Xaction_AppUser_Assistant1` FOREIGN KEY (`AppUserIdAssistant1`) REFERENCES `appuser` (`AppUserId`) ON DELETE RESTRICT ON UPDATE RESTRICT, CONSTRAINT `FK_Xaction_AppUser_Assistant2` FOREIGN KEY (`AppUserIdAssistant2`) REFERENCES `appuser` (`AppUserId`), CONSTRAINT `FK_Xaction_AppUser_CoAgent` FOREIGN KEY (`AppUserIdCoAgent`) REFERENCES `appuser` (`AppUserId`) ON DELETE RESTRICT ON UPDATE RESTRICT, CONSTRAINT `FK_Xaction_AppUser_PrimaryAgent` FOREIGN KEY (`AppUserIdPrimaryAgent`) REFERENCES `appuser` (`AppUserId`) ON DELETE RESTRICT ON UPDATE RESTRICT, CONSTRAINT `FK_Xaction_XactionPropTypeLookup` FOREIGN KEY (`XactionPropTypeId`) REFERENCES `xactionproptypelookup` (`XactionPropTypeId`) ON DELETE RESTRICT ON UPDATE RESTRICT, CONSTRAINT `FK_Xaction_XactionSourceLookup` FOREIGN KEY (`XactionSourceId`) REFERENCES `xactionsourcelookup` (`XactionSourceId`) ON DELETE RESTRICT ON UPDATE RESTRICT, CONSTRAINT `FK_Xaction_XactionStatusLookup` FOREIGN KEY (`XactionStatusId`) REFERENCES `xactionstatuslookup` (`XactionStatusId`) ON DELETE RESTRICT ON UPDATE RESTRICT ) ENGINE=InnoDB;
问题解答
1. JSON字段为何引发性能问题?
即使没查询JSON字段,当查询需要创建临时表时,InnoDB仍会把整行数据(包括JSON字段)加载到临时表中。JSON字段和TEXT字段的核心差异在于:
- MySQL对JSON字段会做额外的解析和验证,哪怕只是复制数据到临时表,也会消耗CPU资源解析JSON结构;
- JSON字段采用二进制存储格式(BJSON),复制和处理的开销远大于普通TEXT字段的字节复制;
- 13.5万行×7KB的JSON数据,临时表总数据量接近945MB,加上JSON解析的CPU开销,直接导致查询耗时飙升。
2. 是否应拆分到一对一关联表?
非常建议拆分,原因如下:
- 拆分后主表行数据体积大幅减小,临时表仅加载主表必要字段,内存占用和IO开销显著降低;
- 关联表仅在需要访问JSON数据时才会被查询,不会影响主表常规查询性能;
- 随着JSON数据量增长,主表行膨胀问题会愈发严重,拆分能从根源避免主表性能退化。
3. 后续基于JSON内部值查询的影响?
拆分到关联表后,依然可以高效查询JSON内部的值:
- 可在关联表的JSON字段上创建JSON路径索引,比如
ALTER TABLE xaction_json ADD INDEX idx_json_field (CAST(json_column->>'$.key' AS UNSIGNED));,加速特定JSON路径的查询; - 若需频繁查询JSON中的多个字段,可将常用属性提取为关联表的独立列,创建普通B树索引,性能比JSON索引更优;
- Hibernate可通过一对一关联映射轻松处理主表和关联表的查询,不会增加过多代码复杂度。
内容的提问来源于stack exchange,提问作者Burton
相关产品推荐
相关产品推荐

