Avro 1.9.0中Decimal类型赋值优化及可插拔转换器咨询

Great question! I totally get the frustration of dealing with Avro's Decimal logical type hoops when using GenericRecord—those manual conversions with DecimalConversion feel way too clunky for everyday use. Let’s break down two solid solutions for you, including how to use Avro’s pluggable logical type converters in 1.9.0.

Avro 1.9.0 actually includes a DecimalConversion that can handle the heavy lifting—you just need to register it with the global GenericData instance first. Once registered, you can directly assign BigDecimal values to your GenericRecord without manually converting to ByteBuffer or byte arrays.

Step 1: Register the Converter #

Add this one-time setup code early in your application (e.g., during initialization):

import org.apache.avro.generic.GenericData;
import org.apache.avro.conversions.DecimalConversion;

// Register the built-in Decimal converter with GenericData
GenericData genericData = GenericData.get();
genericData.addLogicalTypeConversion(new DecimalConversion());

Step 2: Assign Values Directly #

Now you can assign BigDecimal values to your GenericRecord just like you wanted:

// This works seamlessly now!
genericRecord.put("Quantity", BigDecimal.valueOf(4.0));

Avro will automatically handle converting the BigDecimal to the correct ByteBuffer/byte array based on your schema's Decimal logical type (with precision 20, scale 3 in your case).

If you want to directly assign double values (like 4.0) without wrapping them in BigDecimal, you can create a custom pluggable converter that bridges double to Avro's Decimal type.

Step 1: Create the Custom Converter #

Implement LogicalTypeConversion to handle double ↔ ByteBuffer conversions:

import org.apache.avro.LogicalType;
import org.apache.avro.LogicalTypes;
import org.apache.avro.Schema;
import org.apache.avro.conversions.DecimalConversion;
import org.apache.avro.conversions.LogicalTypeConversion;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.nio.ByteBuffer;

public class DoubleToDecimalConversion extends LogicalTypeConversion<Double, ByteBuffer> {
    private final DecimalConversion decimalConverter = new DecimalConversion();

    public DoubleToDecimalConversion() {
        // Tie this converter to the Decimal logical type (adjust precision/scale if needed)
        super(LogicalTypes.decimal(20, 3), Schema.Type.BYTES);
    }

    @Override
    public ByteBuffer toDouble(Double value, Schema schema, LogicalType type) {
        // Convert double to BigDecimal, matching the schema's scale
        BigDecimal decimalValue = BigDecimal.valueOf(value)
                .setScale(((LogicalTypes.Decimal) type).getScale(), RoundingMode.HALF_UP);
        // Delegate to the built-in converter for byte buffer conversion
        return decimalConverter.toByteBuffer(decimalValue, schema, type);
    }

    @Override
    public Double fromByteBuffer(ByteBuffer value, Schema schema, LogicalType type) {
        // Convert back from Decimal to double if needed
        BigDecimal decimalValue = decimalConverter.fromByteBuffer(value, schema, type);
        return decimalValue.doubleValue();
    }
}

Step 2: Register the Custom Converter #

Add it to GenericData alongside the built-in converter:

GenericData.get().addLogicalTypeConversion(new DoubleToDecimalConversion());

Step 3: Assign Double Values Directly #

Now you can use both double and BigDecimal values interchangeably:

// Both of these work now!
genericRecord.put("Quantity", 4.0);
genericRecord.put("Quantity", BigDecimal.valueOf(4.0));

If you can’t modify the global GenericData configuration (e.g., in a shared library), wrap the conversion logic in a reusable utility method to avoid repeating code:

import org.apache.avro.LogicalTypes;
import org.apache.avro.Schema;
import org.apache.avro.conversions.DecimalConversion;
import org.apache.avro.generic.GenericRecord;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.nio.ByteBuffer;

public class AvroDecimalUtils {
    public static void putDecimal(GenericRecord record, String fieldName, Number value) {
        Schema.Field field = record.getSchema().getField(fieldName);
        if (field == null) {
            throw new IllegalArgumentException("Field '" + fieldName + "' not found in schema");
        }

        LogicalTypes.Decimal decimalType = (LogicalTypes.Decimal) field.schema().getLogicalType();
        if (decimalType == null) {
            throw new IllegalArgumentException("Field '" + fieldName + "' is not a Decimal logical type");
        }

        BigDecimal decimalValue;
        if (value instanceof BigDecimal) {
            decimalValue = (BigDecimal) value;
        } else {
            decimalValue = BigDecimal.valueOf(value.doubleValue());
        }

        // Ensure the value matches the schema's scale (adjust rounding mode as needed)
        decimalValue = decimalValue.setScale(decimalType.getScale(), RoundingMode.HALF_UP);

        // Convert to ByteBuffer and assign
        ByteBuffer byteBuffer = new DecimalConversion().toByteBuffer(decimalValue, field.schema(), decimalType);
        record.put(fieldName, byteBuffer);
    }
}

Use it like this:

AvroDecimalUtils.putDecimal(genericRecord, "Quantity", 4.0);
AvroDecimalUtils.putDecimal(genericRecord, "Quantity", BigDecimal.valueOf(4.0));

Key Notes #

• Always ensure your schema defines the Decimal logical type correctly (e.g., {"type": "bytes", "logicalType": "decimal", "precision": 20, "scale": 3}).
• Using double with Decimal carries a risk of precision loss—stick with BigDecimal if you need exact decimal values.

内容的提问来源于stack exchange，提问作者Werner Daehn