Akka Projection与Akka Serialization导致JVM栈溢出崩溃排查求助

问题背景 #

刚接触Akka Projection和Akka Serialization，将官方购物车第6教程适配到自有项目时，首次运行就触发JVM崩溃，错误为序列化相关的StackOverflowError。项目中事件溯源用MongoDB插件，Akka Projection基于JDBC。

依赖配置 #

val akkaVersion = "2.6.20"
val AkkaProjectionVersion = "1.2.5"
val ScalikeJdbcVersion = "3.5.0"

libraryDependencies ++= Seq(
  "com.typesafe.akka" %% "akka-cluster-typed" % akkaVersion,
  "com.typesafe.akka" %% "akka-cluster-sharding-typed" % akkaVersion,
  "com.typesafe.akka" %% "akka-stream" % akkaVersion,
  "com.typesafe.akka" %% "akka-actor-testkit-typed" % akkaVersion % Test,
  "com.typesafe.akka" %% "akka-stream-testkit" % akkaVersion % Test,

  // Akka Management powers Health Checks and Akka Cluster Bootstrapping
  "com.lightbend.akka.management" %% "akka-management" % AkkaManagementVersion,
  "com.lightbend.akka.management" %% "akka-management-cluster-http" % AkkaManagementVersion,
  "com.lightbend.akka.management" %% "akka-management-cluster-bootstrap" % AkkaManagementVersion,
  "com.lightbend.akka.discovery" %% "akka-discovery-kubernetes-api" % AkkaManagementVersion,
  "com.typesafe.akka" %% "akka-discovery" % akkaVersion,
  // Common Dependencies
  "com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
  "ch.qos.logback" % "logback-classic" % "1.2.11",
  "org.scalatest" %% "scalatest" % "3.1.0" % Test,
  // Akka Persistence
  "com.typesafe.akka" %% "akka-persistence-typed" % akkaVersion,
  "com.github.scullxbones" %% "akka-persistence-mongo-scala" % "3.0.8",
  "com.github.scullxbones" %% "akka-persistence-mongo-rxmongo" % "3.0.8",
  "com.typesafe.akka" %% "akka-serialization-jackson" % akkaVersion,
  "com.typesafe.akka" %% "akka-persistence-testkit" % akkaVersion % Test,

  // Querying or projecting data from Akka Persistence
  "com.typesafe.akka" %% "akka-persistence-query" % akkaVersion,
  "com.lightbend.akka" %% "akka-projection-eventsourced" % AkkaProjectionVersion,
  "com.lightbend.akka" %% "akka-projection-jdbc" % AkkaProjectionVersion,
  "org.scalikejdbc" %% "scalikejdbc" % ScalikeJdbcVersion,
  "org.scalikejdbc" %% "scalikejdbc-config" % ScalikeJdbcVersion,
  "mysql" % "mysql-connector-java" % "5.1.34",
  "com.lightbend.akka" %% "akka-projection-testkit" % AkkaProjectionVersion % Test
)

错误日志 #

received message: InitF(Kids, Member(5f6e16da,bz16716,None,None,Some(2f55a5),None,MemberStatus(1,Available),DateOnly(2022,12,22),None),Actor[akka://application/temp/FEn-4a4147e0$d#0])
2022-12-22 02:19:11 DEBUG iMadz  actor [akka://application/system/sharding/FEn/336/4a4147e0] received message: WriteMessagesSuccessful
2022-12-22 02:19:11 DEBUG iMadz  actor [akka://application/system/sharding/FamilyEntity/336/4a4147e0] received message: WriteMessageSuccess(PersistentRepr(FEn|4a4147e0,1,2ae2981eabd9,0,None),3)
2022-12-22 02:19:11 INFO  a.a.t.i.LogMessagesInterceptor  actor [akka://application/system/sharding/RegistrationService/795/97db1846570837fce6ff62a408f1c26a] received message: RegistrationDone(UserProfile(572f55a5,barry0433),GHOST,fab07689,Actor[akka://application/temp/RegistrationService-97db1846570837fce6ff62a408f1c26a$a#0])
2022-12-22 02:19:11 INFO  a.a.t.i.LogMessagesInterceptor  actor [akka://application/system/sharding/RegistrationService/795/97db1846570837fce6ff62a408f1c26a] received message: WriteMessagesSuccessful
2022-12-22 02:19:11 INFO  a.a.t.i.LogMessagesInterceptor  actor [akka://application/system/sharding/RegistrationService/795/97db1846570837fce6ff62a408f1c26a] received message: WriteMessageSuccess(PersistentRepr(RegistrationService|97db1846570837fce6ff62a408f1c26a,1,7d87389b-277b-41f2-a9cd-fee3303b1ae6,0,None),1)
Uncaught error from thread [application-akka.projection.jdbc.blocking-jdbc-dispatcher-77]: null, shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[application]
java.lang.StackOverflowError
    at akka.serialization.Serialization.serializerFor(Serialization.scala:305)
    at akka.serialization.Serialization.findSerializerFor(Serialization.scala:292)
    at akka.persistence.query.internal.QuerySerializer.toStorageRepresentation(QuerySerializer.scala:155)
    at akka.persistence.query.internal.QuerySerializer.manifest(QuerySerializer.scala:54)
    at akka.serialization.Serializers$.manifestFor(Serializer.scala:85)
    at akka.persistence.query.internal.QuerySerializer.toStorageRepresentation(QuerySerializer.scala:157)
    at akka.persistence.query.internal.QuerySerializer.manifest(QuerySerializer.scala:54)
    at akka.serialization.Serializers$.manifestFor(Serializer.scala:85)
    at akka.persistence.query.internal.QuerySerializer.toStorageRepresentation(QuerySerializer.scala:157)

事件定义 #

sealed trait FEvent extends CborSerializable
final case class FCreated(id: Id, name: String, ownerId: Id, owner: Profile, createdOn: DateOnly) extends FEvent

排查方向 #

• 检查序列化配置循环依赖：从堆栈看，QuerySerializer在toStorageRepresentation和manifest间无限递归，需确认application.conf中是否为CborSerializable类型指定了正确序列化器，避免QuerySerializer被错误用于事件序列化；排查自定义序列化器是否触发了循环调用Akka默认序列化流程。
• 验证嵌套类序列化兼容性：事件FCreated包含Profile、DateOnly等嵌套类，需确认所有嵌套类都正确实现序列化接口（如CborSerializable），且不存在循环引用（比如Profile引用FEvent相关类型）。
• 核对依赖版本兼容性：确认Akka 2.6.20与Akka Projection 1.2.5、akka-persistence-mongo-scala3.0.8的版本适配性，排查是否存在已知的序列化兼容问题。
• 检查Projection查询配置：确认EventsourcedProvider配置正确，指定了读取MongoDB事件的序列化器；检查Projection偏移存储的序列化方式，避免偏移数据序列化触发循环。
• 辅助定位调整：临时增加JVM栈大小（如-Xss2m）获取完整堆栈，或开启akka.serialization的DEBUG日志，查看具体触发循环的类。

内容的提问来源于stack exchange，提问作者Barry Zhong