如何高效从REST API下载文件映射到FileNode并发送至RabbitMQ？

Great question—handling large file streams without buffering to memory or disk is critical for lightweight infrastructure layers, so let’s skip the hacky RestTemplate override and go with clean, maintainable best practices.

The Core Problem #

RestTemplate’s default behavior closes the ClientHttpResponse (and its underlying InputStream) immediately after the ResponseExtractor finishes extracting data. This makes it impossible to pass the stream to another component (like a message queue) without first buffering it.

The Best Solution: Wrap the InputStream to Defer Response Closure #

Instead of modifying RestTemplate’s core doExecute method, use a custom ResponseExtractor that returns a wrapped InputStream. This wrapper will automatically close the original ClientHttpResponse when your code finishes using the stream (e.g., when the message queue client finishes sending the file).

Example Implementation #

public InputStream downloadFileStream(URI fileUri) {
    return restTemplate.execute(
        fileUri,
        HttpMethod.GET,
        null, // No request body needed for GET
        (ClientHttpResponse response) -> {
            // Validate response status first (optional but recommended)
            if (!HttpStatus.OK.equals(response.getStatusCode())) {
                throw new RestClientException("Failed to download file: " + response.getStatusCode());
            }
            
            // Wrap the response input stream to handle proper resource cleanup
            return new FilterInputStream(response.getBody()) {
                @Override
                public void close() throws IOException {
                    try {
                        // Close the underlying stream first
                        super.close();
                    } finally {
                        // Ensure the parent ClientHttpResponse is closed too
                        response.close();
                    }
                }
            };
        }
    );
}

How to Use It with Your FileNode Model #

// Map the wrapped stream to your FileNode
FileNode fileNode = new FileNode();
fileNode.setInputStream(downloadFileStream(fileUri));

// Pass the FileNode directly to your message queue producer
messageQueueProducer.send(fileNode);

Key Advantages of This Approach #

• No RestTemplate modifications: Avoids breaking Spring’s core behavior and reduces maintenance risk during version upgrades.
• True streaming: The file is never buffered in memory or written to disk—data flows directly from the REST API to the message queue.
• Proper resource management: The wrapper ensures the HTTP connection is closed only after the stream is fully processed, preventing connection leaks (critical if using connection pools).

Critical Notes for Your Use Case #

1. Ensure the message queue client supports streaming: Most modern clients (like RabbitMQ’s AMQP client or Kafka’s producer API) can consume an InputStream directly without buffering.
2. Never skip closing the stream: The message queue consumer or producer must call close() on the InputStream when done—otherwise, you’ll leak HTTP connections.
3. Avoid intermediate processing: Don’t read the stream in your infrastructure layer; pass the wrapped stream directly to the message queue to keep the pipeline lightweight.

Alternative: Use Spring's Resource Abstraction #

If you prefer working with Spring’s Resource interface (e.g., InputStreamResource), you can apply the same wrapping pattern:

public Resource downloadFileResource(URI fileUri) {
    return restTemplate.execute(
        fileUri,
        HttpMethod.GET,
        null,
        (ClientHttpResponse response) -> {
            InputStream wrappedStream = new FilterInputStream(response.getBody()) {
                @Override
                public void close() throws IOException {
                    try {
                        super.close();
                    } finally {
                        response.close();
                    }
                }
            };
            return new InputStreamResource(wrappedStream);
        }
    );
}

Then assign resource.getInputStream() to your FileNode’s property.

内容的提问来源于stack exchange，提问作者mhrsalehi