Apache Thrift序列化至文件性能异常：较Protobuf慢100倍以上求排查

我正在做序列化相关工作，需要对Apache Thrift做基准测试，但发现序列化耗时极长，平均耗时比Protobuf高出100倍以上。请问问题出在哪里？

Thrift定义文件 #

namespace java com.myproject.mybenchmark.thrift

struct StudentThrift {
  1: required double sid
  2: required double grade
  3: required i32 age
  4: required i32 year
}

struct StudentThriftList {
  1: list<StudentThrift> students
}

序列化实现代码 #

private static void serializeToFile(ArrayList<Student> studentList, String outputFile) {
    StudentThriftList thriftStudentList = new StudentThriftList();
    for(Student stu:studentList) {
        StudentThrift thriftStudent = new StudentThrift();
        thriftStudent.setSid(stu.getSid());
        thriftStudent.setGrade(stu.getGrade());
        thriftStudent.setAge(stu.getAge());
        thriftStudent.setYear(stu.getYear());
        thriftStudentList.addToStudents(thriftStudent);
    }

    // Serializing to disk.
    try (FileOutputStream fos = new FileOutputStream(outputFile);
            TTransport transport = new TIOStreamTransport(fos)) {
            TCompactProtocol protocol = new TCompactProtocol(transport);
            thriftStudentList.write(protocol);  
       } catch (Exception e) {
        e.printStackTrace();
    } 
}

反序列化实现代码 #

public static StudentThriftList deserializeFromFile(String dataFile)  {
    StudentThriftList thriftList = new StudentThriftList();
    try (FileInputStream fis = new FileInputStream(dataFile);
         TTransport transport = new TIOStreamTransport(fis)) {
        TCompactProtocol protocol = new TCompactProtocol(transport);
        thriftList.read(protocol);
    } catch (Exception e) {
        e.printStackTrace();
    } 
    return thriftList;
}

基准测试主类代码 #

public static void main(String[] args) {

    ArrayList<Student> studentList = new ArrayList<Student>();

    long serStartTime = 0L;
    long serEndTime = 0L;
    int serCount = 0;
    long serTotalTime = 0L;

    long desStartTime = 0L;
    long desEndTime = 0L;
    int desCount = 0;
    long desTotalTime = 0L;
    
    
    
    // Create a list with 10000 objects:
    for (int i = 1; i <= 10000; i++) {
        Student student = new Student();
        student.setSid(1000 + i * 1.1);  
        student.setGrade((float) (i * 0.5));
        student.setAge(10 + i);
        student.setYear(2000 + i);
        studentList.add(student);
    }

    // benchmark serilization-deserialization 1000 times:
    for(int i=0; i<1000; i++) {
        // serialize...
        String outputFile = "output/thriftStudents_"+System.currentTimeMillis();
        serStartTime = System.currentTimeMillis();
        serializeToFile(studentList, outputFile);
        serEndTime = System.currentTimeMillis();
        serCount++;
        serTotalTime += (serEndTime-serStartTime);
        
        // deserialize...
        desStartTime = System.currentTimeMillis();
        StudentThriftList stuTriftList =  deserializeFromFile(outputFile);
        desEndTime = System.currentTimeMillis();
        desCount++;
        desTotalTime += (desEndTime-desStartTime);
    }
    
    // print report
    System.out.println("--------------REPORT---------------");
    System.out.println("Serializetion count: " + serCount);
    System.out.println("Serializetion avg time (ms): " + serTotalTime/(long)serCount);
    System.out.println("Deserializetion count: " + desCount);
    System.out.println("Deserializetion avg time (ms): " + desTotalTime/(long)desCount);
}

测试结果报告 #

--------------REPORT---------------
Serializetion count: 1000
Serializetion avg time (ms): 408
Deserializetion count: 1000
Deserializetion avg time (ms): 448

性能异常的核心原因 #

1. 磁盘IO被计入序列化耗时：你的基准测试把文件写入/读取的磁盘IO时间完全算进了序列化/反序列化耗时里。Thrift本身的序列化是内存操作，磁盘IO的开销远大于序列化逻辑本身，这才是耗时的主要来源——如果Protobuf的测试没有包含磁盘IO，自然会显得Thrift慢100倍。
2. 重复创建大量临时文件：每次循环都创建新的输出文件，频繁的文件创建、写入、读取会带来额外系统开销，比如文件系统元数据操作、磁盘寻道等。
3. 缺少JVM预热：基准测试直接开始循环，没有给JVM留出即时编译（JIT）的预热时间，Thrift的序列化代码可能还处于解释执行状态，未被优化。

优化建议 #

• 分离序列化与磁盘IO：先在内存中完成序列化（写入TByteArrayTransport），再单独统计磁盘写入时间；反序列化同理，先把文件内容读入内存，再进行反序列化，这样才能得到真实的序列化性能数据。
• 用内存替代磁盘测试：如果仅测试序列化逻辑，完全可以不用写入磁盘，直接用内存传输（TByteArrayTransport）避免IO干扰。
• 添加预热阶段：正式测试前先运行几次序列化/反序列化循环，让JVM完成热点代码的编译优化。

内容的提问来源于stack exchange，提问作者Kuvalya