MongoDB 200条数据查询过慢问题(Spring Boot+EC2部署)
问题背景
我有一个名为product的MongoDB集合,文档结构如下:
{ "_id" : "64a1583ab9ec356358be7853", "productId" : Long("7"), "name" : "The LG 7 kg 5 Star Semi-Automatic Top Loading Washing Machine (P7020NGAZ)", "description" : "The LG 7 kg 5 Star Semi-Automatic Top Loading Washing Machine is designed to handle a 7 kg laundry load, making it suitable for small to medium-sized households. It features a top-loading design, allowing for easy loading and unloading of clothes. \r\n\r\nBrand LG\r\nModel P7020NGAZ\r\nCapacity 7 Kilograms\r\nMaximum Rotational Speed 1300 RPM\r\nInstallation Type Free Standing\r\nPart Number 20NGAZ\r\nForm Factor Stand alone\r\nSpecial Features Wind Jet Dry, Collar Scrubber, Rat Away Technology, Rust Free Plastic Base\r\nControl Console Knob\r\nNumber of Option Cycles 3\r\nAccess Location Top Load\r\nVoltage 230 Volts\r\nWattage 360 Watts\r\nCertification Energy Star\r\nMaterial Plastic\r\nBatteries Included No\r\nBatteries Required No\r\nManufacturer LG Electronics India Pvt. Ltd.\r\nCountry of Origin India", "photos" : [ "https://electrotrade-uploads.s3.ap-south-1.amazonaws.com/photos/product/mlri14107249018289122.jpg", "https://electrotrade-uploads.s3.ap-south-1.amazonaws.com/photos/product/tl5714107248740599521.jpg", "https://electrotrade-uploads.s3.ap-south-1.amazonaws.com/photos/product/kj0z14107248662853011.jpg" ], "price" : 10800.0, "sellingPrice" : 10800.0, "weight" : 0.0, "reward" : 25.0, "gstPercentage" : 4.5, "minQty" : 1, "maxQty" : 100, "totalStockAvailable" : 150, "featuredProduct" : true, "sku" : "", "categoryId" : Long("2"), "sellerId" : Long("7"), "subCategoryId" : Long("16"), "brand" : DBRef("brand", "64a06d723eaf681bbc77e165"), "user" : { "_id" : "64a122f6b9ec356358be7848", "email" : "kushaljain1256@gmail.com", "username" : "Kushalj", "password" : "{bcrypt}$2a$10$lbwmsJ.gP74thXP/HGbp7O9I6kZnzy.2NSfM5Gmx2U5HRS/9SdDRa", "firstName" : "Kushal ", "lastName" : "Jain", "phoneNumber" : "8005812294", "userId" : Long("7"), "pincode" : "304001", "address" : "Sharma Colony, Tonk", "city" : "Tonk", "state" : "Rajasthan", "pancard" : "undefined", "aadhar" : "undefined", "businessName" : "Rajasthan Electronics", "businessType" : "Electronics & Electrical ", "walletBalance" : 50.0, "active" : true, "mobileVerified" : true, "emailVerified" : true, "subscribed" : false, "referralCode" : "o5be9b", "deviceDetail" : { "deviceToken" : "ekj5ioLhRXmcrk3nztNfnH:APA91bGoak8zfrW-x3Di-rr2PjQ-sIVYh_jCSgBeWj53sC0ytmndKH7YrTgoRwkb5RfVDyBf2iYeysOCMhNNtgzkvb-RBHyTsz8xKgOLWdGBD8xXwQEf4h0nJXdGQy6ZwSvle2tYb08g", "deviceType" : "android", "os" : "android", "osVersion" : "11", "appVersion" : "1.8", "lastModifiedDate" : ISODate("2023-08-03T09:39:26.527+0000") }, "isReferred" : false, "isPushEnabled" : true, "roles" : [ DBRef("role", "649fbe99196ccf0b66ba0398"), DBRef("role", "64a123adb9ec356358be784c") ], "categories" : [ DBRef("category", "64a0598e3eaf681bbc77e160"), DBRef("category", "64a057ce3eaf681bbc77e15f"), DBRef("category", "64a05a323eaf681bbc77e161"), DBRef("category", "64a05b8d3eaf681bbc77e162"), DBRef("category", "64a05d563eaf681bbc77e163"), DBRef("category", "64a0ffa73eaf681bbc77e168"), DBRef("category", "64a1022e3eaf681bbc77e169"), DBRef("category", "64c7ad26ef0147741249e205"), DBRef("category", "64c7adabef0147741249e206"), DBRef("category", "64c7a66aef0147741249e204") ], "displayName" : "BO-7", "createDate" : ISODate("2023-07-02T07:10:46.044+0000"), "lastModifiedDate" : ISODate("2023-07-31T12:53:29.658+0000") }, "note" : "null", "shipping" : { "shippingType" : "ORDER_AMOUNT", "amount" : 40.0, "minAmount" : 80.0, "maxAmount" : 120.0, "shippingAllIndia" : true, "minWeight" : 0.0, "maxWeight" : 0.0, "range" : 0.0, "createDate" : ISODate("2023-08-01T16:21:15.493+0000"), "lastModifiedDate" : ISODate("2023-08-01T16:21:15.493+0000") }, "active" : true, "createDate" : ISODate("2023-07-02T10:58:02.455+0000"), "lastModifiedDate" : ISODate("2023-08-03T15:29:16.697+0000"), "_class" : "com.electrotrade.model.Product" }
该集合仅约200条文档,但使用Spring Data MongoDB的findAll()方法查询时,耗时超62秒:
public interface ProductRepository extends MongoRepository<Product, String> { List<Product> findAll(); }
DBRef关联的集合(如brand)仅包含4-5个简单字符串字段。另外,MongoDB和Java应用部署在同一EC2实例的EBS卷上:通过EC2内部IP访问API响应仅毫秒级,但本地连接EC2上的MongoDB调用同一API耗时超30秒。
优化方案
1. 禁用DBRef自动加载
Spring Data MongoDB默认会自动解析DBRef并发起关联查询,这会产生大量N+1查询(200条product对应200次brand查询,加上user里的roles、categories的DBRef,查询量会指数级增长)。可以在实体类的DBRef字段上添加@DBRef(lazy = true)实现延迟加载,仅在实际使用关联数据时才发起查询;如果业务不需要实时获取关联数据,可直接去掉DBRef,改用手动关联查询。
示例:
@DBRef(lazy = true) private Brand brand;
2. 投影查询,只返回必要字段
findAll()会返回文档的所有字段,包括大文本(如description)、数组(如photos)和嵌套对象(如user),这些数据会占用大量带宽和序列化时间。可以定义投影接口,只返回业务需要的字段:
// 定义投影接口 public interface ProductProjection { String getName(); Double getSellingPrice(); Integer getTotalStockAvailable(); // 按需添加其他字段 } // 在Repository中添加投影查询方法 List<ProductProjection> findAllProjectedBy();
3. 优化EC2网络与存储配置(解决本地访问慢问题)
- 确认EC2安全组已将MongoDB端口(默认27017)开放给本地IP,避免网络阻塞或路由异常。
- 直接使用EC2公网IP连接MongoDB,减少DNS解析耗时。
- 检查EBS卷性能:如果使用通用型EBS卷,可能存在IOPS瓶颈,可临时切换到IOPS优化型卷测试性能。
4. 调整MongoDB连接池参数
在Spring配置中优化连接池,避免频繁创建销毁连接:
spring: data: mongodb: uri: mongodb://localhost:27017/your_database connection-pool: max-size: 20 min-size: 5 max-wait-time: 1000ms
5. 移除冗余嵌套数据
user字段包含大量冗余信息(如password、deviceDetail、roles等),如果业务不需要这些数据,可在投影时排除,或在实体类中用@Transient标记不需要返回的字段;长期方案可拆分数据,仅在product文档中存储必要的卖家信息(如卖家ID、名称),而非整个user对象。
6. 分析MongoDB查询日志
开启MongoDB慢查询日志,定位耗时环节:
- 启用慢查询日志(阈值设为100ms):
db.setProfilingLevel(1, { slowms: 100 })
- 查询日志详情:
db.system.profile.find({ op: "query", ns: "your_database.product" }).sort({ ts: -1 })
通过日志确认是查询本身慢,还是数据传输/序列化耗时。
内容的提问来源于stack exchange,提问作者jarvo69

