如何在JPA中对含复合ID的实体实现批量插入?
向Oracle 19C全局临时表批量插入的JPA性能问题
我在项目中使用JPA 3.1.1,此前运行一切正常,但在向Oracle 19C的全局临时表(GTT)批量插入数据时遇到性能瓶颈。该表无主键(PK),仅包含4列,我需要向其中加载多达百万级的数据,之后在数据库端执行查询以保证效率。
问题在于加载过程非常缓慢,因为Hibernate会逐条执行INSERT语句。我已经通过实现Persistable的isNew()逻辑避免了插入前的SELECT操作,但仍无法实现类似INSERT ALL的批量操作,减少与服务器的通信次数。
日志片段
2024-12-19 18:33:57,594 DEBUG org.hibernate.engine.jdbc.spi.SqlStatementLogger [scheduling-3] insert into XXX.MY_TMP_TABLE (ADDITIONAL_DATA,AN_ENUM,A_BOOLEAN,A_STRING) values (?,?,?,?) 2024-12-19 18:33:57,594 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [1] as [VARCHAR] - [xx1] 2024-12-19 18:33:57,594 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [2] as [VARCHAR] - [I] 2024-12-19 18:33:57,594 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [3] as [BOOLEAN] - [true] 2024-12-19 18:33:57,594 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [4] as [VARCHAR] - [abcd] 2024-12-19 18:33:57,595 DEBUG org.hibernate.engine.jdbc.spi.SqlStatementLogger [scheduling-3] insert into XXX.MY_TMP_TABLE (ADDITIONAL_DATA,AN_ENUM,A_BOOLEAN,A_STRING) values (?,?,?,?) 2024-12-19 18:33:57,595 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [1] as [VARCHAR] - [xx1] 2024-12-19 18:33:57,595 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [2] as [VARCHAR] - [I] 2024-12-19 18:33:57,595 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [3] as [BOOLEAN] - [true] 2024-12-19 18:33:57,596 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [4] as [VARCHAR] - [abce] 2024-12-19 18:33:57,596 DEBUG org.hibernate.engine.jdbc.spi.SqlStatementLogger [scheduling-3] insert into XXX.MY_TMP_TABLE (ADDITIONAL_DATA,AN_ENUM,A_BOOLEAN,A_STRING) values (?,?,?,?) 2024-12-19 18:33:57,596 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [1] as [VARCHAR] - [xx1] 2024-12-19 18:33:57,596 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [2] as [VARCHAR] - [I] 2024-12-19 18:33:57,596 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [3] as [BOOLEAN] - [true] 2024-12-19 18:33:57,597 TRACE org.hibernate.type.descriptor.JdbcBindingLogging [scheduling-3] binding parameter [4] as [VARCHAR] - [abcf]
实体类
@Entity @Table(schema = "XXX", name = "MY_TMP_TABLE") public class MyEntity implements Persistable<MyPK> { @EmbeddedId private MyPK id; @Column(name = "ADDITIONAL_DATA", nullable = false) private String additionalData; @Transient private boolean isNew; /** * Creator with "isNew" flag to avoid SELECT before INSERT in batch treatments * * @param aIsNew true if we are certain the key does not exist in the database */ public MyEntity(boolean aIsNew) { super(); this.isNew = aIsNew; } /** * @return the composite ID */ @Override public MyPK getId() { return id; } /** * Sets the composite ID * * @param anId the ID */ public void setId(MyPK anId) { this.id = anId; } /** * @return the additional data */ public String getAdditionalData() { return additionalData; } /** * @param anAdditionalData the additional data to set */ public void setAdditionalData(String anAdditionalData) { this.additionalData = anAdditionalData; } @Override public boolean isNew() { return isNew; } }
嵌入ID类
@Embeddable public class MyPK { @Column(name = "A_STRING", nullable = false) private String aString; @Column(name = "AN_ENUM", nullable = false) @Convert(converter = AConverter.class) private AnEnum anEnum; @Column(name = "A_BOOLEAN", nullable = false) private boolean aBoolean; /** * @return the String */ public String getAString() { return aString; } /** * @param aString the String to set */ public void setAString(String aString) { this.aString = aString; } /** * @return the enum */ public AnEnum getAnEnum() { return anEnum; } /** * @param anEnum the enum to set */ public void setAnEnum(AnEnum anEnum) { this.anEnum = anEnum; } /** * @return the boolean */ public boolean getABoolean() { return aBoolean; } /** * @param aBoolean the boolean to set */ public void setBoolean(boolean aBoolean) { this.aBoolean = aBoolean; } }
插入逻辑
myEntityRepository.save(myEntity); if (nbTotal % batchSize == 0) { myEntityRepository.flush(); }
(注:也尝试过使用saveAll()批量保存,但无效果)
JPA配置
properties.setProperty("hibernate.jdbc.batch_size", dbOkcJpaBatchSize); --100 properties.setProperty("hibernate.order_inserts", dbOkcJpaOrderInserts); --true properties.setProperty("hibernate.order_updates", dbOkcJpaOrderUpdates); --true
需求:不想使用JDBC原生查询,也不想修改数据库新增看似无用的数值ID和序列,寻求可行的解决方案。
解决方案
1. 补充Hibernate批量插入核心配置
当前配置缺少两个关键参数,需添加:
properties.setProperty("hibernate.jdbc.batch_versioned_data", "true"); properties.setProperty("hibernate.statement_inspector", "org.hibernate.resource.jdbc.internal.StatementInspectorImpl");
hibernate.jdbc.batch_versioned_data:允许Hibernate对复合ID场景的实体执行批量操作;statement_inspector:确保Hibernate正确拦截并批量处理SQL语句。
2. 标注全局临时表属性
告诉Hibernate这是全局临时表,优化操作逻辑:
@Entity @Table(schema = "XXX", name = "MY_TMP_TABLE") @org.hibernate.annotations.Table(appliesTo = "MY_TMP_TABLE", options = @org.hibernate.annotations.TableOptions( temporary = org.hibernate.annotations.TemporaryTableType.GLOBAL ) ) public class MyEntity implements Persistable<MyPK> { // 原有代码不变 }
此配置会减少Hibernate对临时表的不必要缓存和状态跟踪。
3. 优化批量插入逻辑
避免循环调用save(),改用批量收集+saveAll(),配合清空一级缓存:
List<MyEntity> batchList = new ArrayList<>(batchSize); for (/* 数据循环逻辑 */) { MyEntity entity = new MyEntity(true); // 设置实体属性 batchList.add(entity); if (batchList.size() >= batchSize) { myEntityRepository.saveAll(batchList); entityManager.flush(); entityManager.clear(); // 清空一级缓存,确保批量逻辑生效 batchList.clear(); } } // 处理剩余数据 if (!batchList.isEmpty()) { myEntityRepository.saveAll(batchList); entityManager.flush(); entityManager.clear(); }
循环save()会让实体堆积在一级缓存,导致批量插入失效,clear()可强制清空缓存,让Hibernate真正批量处理下一批数据。
4. 配置Oracle JDBC驱动批量支持
在数据源URL中添加默认批量大小,与Hibernate配置保持一致:
jdbc.url=jdbc:oracle:thin:@//your-host:port/service?defaultBatchValue=100
5. 完善复合ID的equals和hashCode方法
Hibernate批量操作依赖实体唯一性判断,需为MyPK补充以下方法:
@Embeddable public class MyPK { // 原有属性和方法 @Override public boolean equals(Object o) { if (this == o) return true; if (o == null || getClass() != o.getClass()) return false; MyPK myPK = (MyPK) o; return aBoolean == myPK.aBoolean && Objects.equals(aString, myPK.aString) && anEnum == myPK.anEnum; } @Override public int hashCode() { return Objects.hash(aString, anEnum, aBoolean); } }
内容的提问来源于stack exchange,提问作者Antoine
相关产品推荐
相关产品推荐

