Scala中使用SizeEstimator计算Case Class对象大小异常求助

Let me break down why you're seeing these numbers and how to get what you actually need:

The Root Cause: JVM Object Overhead & SizeEstimator's Purpose #

First, it's important to understand that SizeEstimator isn't measuring the raw byte count of your string characters—it's estimating the total heap memory occupied by the entire object graph, including all the hidden overhead that the JVM adds to every object. Here's what contributes to those bigger numbers:

1. String Object Overhead: A Java/Scala String isn't just the characters themselves. It includes:
   • Object header (12 bytes on 64-bit JVM with compressed pointers, the default in most environments)
   • A reference to the underlying char[] array (4 bytes)
   • A cached hash value (4 bytes)
   • Padding to align the object to an 8-byte boundary (JVM requires object sizes to be multiples of 8)
2. Char Array Overhead: The actual characters are stored in a char[] array, which has its own object header (12 bytes) plus the characters themselves. Since Java char uses 2 bytes (UTF-16 encoding), even a single "a" takes 2 bytes in the array.
3. Case Class Overhead: Your event case class adds its own object header, plus references to the two String fields (4 bytes each), plus padding. On top of that, SizeEstimator sums up the full size of both the imei and date strings (including their char arrays and overhead) to get the total for the case class instance.

Mapping this to your outputs:

• For "a": String overhead (~20 bytes with padding) + char array overhead (~14 bytes with padding) + 2 bytes for the char = ~48 bytes (matches your result).
• For your 15-digit IMEI: String overhead + char array overhead (15*2=30 bytes for chars + array header + padding) = ~72 bytes (also matches your output).
• For the event instance: Case class overhead + full size of imei + full size of date (assuming your date string has similar overhead) + padding = ~520 bytes.

Solutions for Your Needs #

Depending on what you're actually trying to measure, here are the right approaches:

1. Measure Raw Character/Byte Count (e.g., for serialization) #

If you want the actual number of bytes the string would take when encoded (like UTF-8 for storage or network transfer), use:

// UTF-8 byte count
println("UTF-8 bytes for 'a': " + "a".getBytes("UTF-8").length) // Outputs 1
println("UTF-8 bytes for IMEI: " + imei.getBytes("UTF-8").length) // Outputs 15

// UTF-16 byte count (matches Java's internal char storage)
println("UTF-16 bytes for 'a': " + "a".length * 2) // Outputs 2
println("UTF-16 bytes for IMEI: " + imei.length * 2) // Outputs 30

2. Accurately Estimate Heap Memory Usage #

If you do need to measure heap memory (for Spark tuning, for example):

• Stick with SizeEstimator but remember its output represents the total heap footprint, not raw string bytes. This is exactly what you need for planning Spark executor memory.
• For a more precise breakdown of object memory structure, use the JOL (Java Object Layout) library. Add it to your dependencies, then run:

  import org.openjdk.jol.info.GraphLayout
  println("Heap size of 'a': " + GraphLayout.parseInstance("a").totalSize())
  println("Heap size of event: " + GraphLayout.parseInstance(check).totalSize())
  

  JOL will show you exactly how each part of the object contributes to the total size.

内容的提问来源于stack exchange，提问作者Pinnacle