Dataflow写入BigQuery时tempLocation属性@Default注解报错求助

The error you're seeing happens because you're redefining the tempLocation property in your custom Options interface, but this property is already built into the base PipelineOptions interface. The Dataflow annotation processor gets confused by the duplicate declaration—your custom version has @Default, but the base one doesn't, leading to the inconsistency error.

Here are two straightforward solutions:

Solution 1: Remove the custom tempLocation definition (Recommended) #

Since PipelineOptions already includes tempLocation as a standard property, you don't need to redefine it. Just extend PipelineOptions and add only your custom options (like the BigQuery table):

private interface Options extends PipelineOptions {
    @Description("BigQuery table to write to, specified as <project_id>:<dataset_id>.<table_id>")
    @Validation.Required // Optional, if you want to enforce this parameter
    String getBigQueryTable();
    void setBigQueryTable(String value);
}

If you want to set a default value for tempLocation, do it programmatically when initializing your options:

Options options = PipelineOptionsFactory.fromArgs(args).as(Options.class);
// Set default temp location if not provided via command line
if (options.getTempLocation() == null) {
    options.setTempLocation(GCS_TEMP_LOCATION);
}

Solution 2: Override the existing tempLocation property with annotations #

If you prefer to use annotations to set the default, add @Override to clarify you're modifying the existing property (not creating a new one):

private interface Options extends PipelineOptions {
    @Override
    @Description("GCS temp location to store temp files.")
    @Default.String(GCS_TEMP_LOCATION)
    @Validation.Required
    String getTempLocation();

    @Override
    void setTempLocation(String value);

    @Description("BigQuery table to write to, specified as <project_id>:<dataset_id>.<table_id>")
    String getBigQueryTable();
    void setBigQueryTable(String value);
}

This tells the annotation processor you're overriding the base tempLocation property, so it will apply your @Default and @Validation annotations correctly without conflict.

Either approach will resolve the error and let your Dataflow pipeline use the GCS temp location as expected.

内容的提问来源于stack exchange，提问作者Manuel RODRIGUEZ