JVM生态系统中外置Protobuf文件的实现方法及Akka gRPC教程实践问题咨询

Got it, let's break down how to properly reuse external Protobuf sources in Akka gRPC projects—since your current setup isn't working, I'll walk you through a complete, runnable example and troubleshoot common pitfalls.

1. 先搭建独立的Protobuf源Jar项目（proto-library） #

这个项目的核心职责就是存放.proto文件，并将其打包成带有protobuf-src分类的Jar，供服务端和客户端依赖。

项目结构 #

proto-library/
├── project/
│   └── plugins.sbt
├── build.sbt
└── src/
    └── main/
        └── proto/
            └── hello.proto

配置插件（project/plugins.sbt） #

确保添加Akka gRPC和Protobuf编译所需的插件，版本要兼容：

addSbtPlugin("com.lightbend.akka.grpc" % "sbt-akka-grpc" % "2.3.1")
addSbtPlugin("com.thesamet" % "sbt-protoc" % "1.0.6")
libraryDependencies += "com.thesamet.scalapb" %% "compilerplugin" % "0.11.13"

编写Protobuf定义（src/main/proto/hello.proto） #

syntax = "proto3";

package com.example.grpc;

service HelloService {
  rpc SayHello (HelloRequest) returns (HelloResponse);
}

message HelloRequest {
  string name = 1;
}

message HelloResponse {
  string message = 1;
}

项目构建配置（build.sbt） #

关键是自定义任务生成包含Protobuf源的Jar，并发布到本地仓库：

name := "proto-library"
organization := "com.example"
version := "1.0.0"
scalaVersion := "2.13.12"

// 自定义任务：生成包含Protobuf源的Jar，标记为protobuf-src分类
lazy val protoSrcJar = taskKey[File]("Creates a JAR containing proto sources")
protoSrcJar := {
  val jarPath = baseDirectory.value / s"target/${name.value}-${version.value}-protobuf-src.jar"
  IO.zip(
    (sourceDirectory.value / "main" / "proto").***.pair(relativeTo(sourceDirectory.value / "main" / "proto")),
    jarPath
  )
  jarPath
}

// 发布时自动包含这个自定义Jar
publishLocal := publishLocal.dependsOn(protoSrcJar).value

2. 服务端项目（grpc-server）配置 #

项目结构 #

grpc-server/
├── project/
│   └── plugins.sbt
├── build.sbt
└── src/
    └── main/
        └── scala/
            └── com/
                └── example/
                    └── grpc/
                        └── HelloServer.scala

构建配置（build.sbt） #

添加依赖并配置Akka gRPC插件读取依赖中的Protobuf源：

name := "grpc-server"
organization := "com.example"
version := "1.0.0"
scalaVersion := "2.13.12"

libraryDependencies ++= Seq(
  "com.lightbend.akka.grpc" %% "akka-grpc-runtime" % "2.3.1",
  "com.example" %% "proto-library" % "1.0.0" % "protobuf-src",
  "ch.qos.logback" % "logback-classic" % "1.4.8" // 可选日志依赖
)

// 配置生成Scala服务端存根
akkaGrpcGeneratedLanguages := Seq(AkkaGrpc.Scala)
akkaGrpcGeneratedSources := AkkaGrpc.GeneratedSources.Server

服务端实现（HelloServer.scala） #

package com.example.grpc

import akka.actor.ActorSystem
import akka.grpc.GrpcServerSettings
import scala.concurrent.Await
import scala.concurrent.duration.Duration

class HelloServiceImpl extends HelloService {
  override def sayHello(in: HelloRequest) = {
    println(s"Received request from ${in.name}")
    scala.concurrent.Future.successful(HelloResponse(s"Hello, ${in.name}!"))
  }
}

object HelloServer extends App {
  implicit val system: ActorSystem = ActorSystem("HelloServer")
  import system.dispatcher

  val service = HelloServiceHandler(new HelloServiceImpl())
  val server = GrpcServerSettings.fromConfig("helloService").createServer(service)

  server.bind().onComplete {
    case scala.util.Success(binding) =>
      val address = binding.localAddress
      println(s"Server bound to ${address.getHostName}:${address.getPort}")
    case scala.util.Failure(e) =>
      println(s"Server binding failed: ${e.getMessage}")
      system.terminate()
  }

  Await.result(system.whenTerminated, Duration.Inf)
}

3. 客户端项目（grpc-client）配置 #

项目结构 #

grpc-client/
├── project/
│   └── plugins.sbt
├── build.sbt
└── src/
    └── main/
        └── scala/
            └── com/
                └── example/
                    └── grpc/
                        └── HelloClient.scala

构建配置（build.sbt） #

类似服务端，只是配置生成客户端存根：

name := "grpc-client"
organization := "com.example"
version := "1.0.0"
scalaVersion := "2.13.12"

libraryDependencies ++= Seq(
  "com.lightbend.akka.grpc" %% "akka-grpc-runtime" % "2.3.1",
  "com.example" %% "proto-library" % "1.0.0" % "protobuf-src",
  "ch.qos.logback" % "logback-classic" % "1.4.8"
)

// 配置生成Scala客户端存根
akkaGrpcGeneratedLanguages := Seq(AkkaGrpc.Scala)
akkaGrpcGeneratedSources := AkkaGrpc.GeneratedSources.Client

客户端实现（HelloClient.scala） #

package com.example.grpc

import akka.actor.ActorSystem
import akka.grpc.GrpcClientSettings
import scala.concurrent.Await
import scala.concurrent.duration.Duration

object HelloClient extends App {
  implicit val system: ActorSystem = ActorSystem("HelloClient")
  import system.dispatcher

  val client = HelloServiceClient(GrpcClientSettings.fromConfig("helloService"))
  val response = client.sayHello(HelloRequest("World"))

  response.onComplete {
    case scala.util.Success(res) =>
      println(s"Received response: ${res.message}")
      system.terminate()
    case scala.util.Failure(e) =>
      println(s"Request failed: ${e.getMessage}")
      system.terminate()
  }

  Await.result(system.whenTerminated, Duration.Inf)
}

4. 关键运行步骤与问题排查 #

• 先发布proto-library到本地仓库：在proto-library目录下运行sbt publishLocal，确保服务端和客户端能拉取到依赖。
• 验证Jar内容：解压生成的proto-library-1.0.0-protobuf-src.jar，确认里面包含hello.proto文件。
• 检查生成的存根：运行sbt compile后，查看服务端/客户端的target/scala-2.13/src_managed/main/akka/grpc/scala/com/example/grpc/目录，应该能看到自动生成的HelloService.scala（服务端）或HelloServiceClient.scala（客户端）。
• 版本兼容性：如果还是不生效，检查Akka gRPC、sbt-protoc、scalapb的版本是否匹配，版本不兼容是常见问题。

5. 替代方案：多项目构建 #

如果不想单独发布Jar，可以把三个项目合并成一个sbt多项目：

// 根目录build.sbt
lazy val protoLibrary = project
  .in(file("proto-library"))
  .settings(/* 同之前的proto-library配置 */)

lazy val grpcServer = project
  .in(file("grpc-server"))
  .dependsOn(protoLibrary % "protobuf-src")
  .settings(/* 同之前的服务端配置 */)

lazy val grpcClient = project
  .in(file("grpc-client"))
  .dependsOn(protoLibrary % "protobuf-src")
  .settings(/* 同之前的客户端配置 */)

这样运行sbt compile时，会自动处理Protobuf源的复用，无需单独发布。

内容的提问来源于stack exchange，提问作者Shivam