Apache Camel Http组件内存溢出：如何实现无全量内存加载的流式传输？

安全阿岳

2026-7-14

问题

使用Apache Camel Http组件从公共链接下载文件并流式传输到SharePoint时触发OutOfMemoryError，该组件默认会将整个文件一次性加载到内存中。需要实现不加载完整流到内存的传输方案。

当前路由逻辑如下：

/**
 * Submit file to upload
 */
from("direct:fetch-content-data-and-upload")
        .log("** Fetching file content from public link")
        .setHeader(Exchange.HTTP_METHOD, constant("GET"))
        .toD("${header.CamelHttpUri}")
        .process(this.submitFileToUpload)
        .end();

处理器内部使用Graph Client SDK分块提交InputStream上传，但错误在请求到达处理器前就抛出，错误堆栈信息如下：

Aug  3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onIdleExpired(HttpConnectionOverHTTP.java:160)
Aug  3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.ssl.SslConnection.onIdleExpired(SslConnection.java:391)
Aug  3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.AbstractEndPoint.onIdleExpired(AbstractEndPoint.java:402)
Aug  3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:171)
Aug  3 07:38:37 ip-172-31-9-176 web: at org.eclipse.jetty.io.IdleTimeout.idleCheck(IdleTimeout.java:113)
Aug  3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
Aug  3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Aug  3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
Aug  3 07:38:37 ip-172-31-9-176 web: ... 3 common frames omitted
Aug  3 07:38:37 ip-172-31-9-176 web: Caused by: java.lang.OutOfMemoryError: Java heap space
Aug  3 07:38:37 ip-172-31-9-176 web: at java.base/java.util.Arrays.copyOf(Arrays.java:3537) ~[na:na]
Aug  3 07:38:37 ip-172-31-9-176 web: at java.base/java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:100) ~[na:na]
Aug  3 07:38:37 ip-172-31-9-176 web: at java.base/java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:130) ~[na:na]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.converter.stream.CachedOutputStream.write(CachedOutputStream.java:101) ~[camel-support-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:206) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:161) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:156) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.util.IOHelper.copy(IOHelper.java:152) ~[camel-util-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.doExtractResponseBodyAsStream(HttpProducer.java:547) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.extractResponseBody(HttpProducer.java:532) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.populateResponse(HttpProducer.java:332) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.component.http.HttpProducer.process(HttpProducer.java:291) ~[camel-http-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66) ~[camel-support-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.SendDynamicProcessor.lambda$process$0(SendDynamicProcessor.java:197) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.SendDynamicProcessor$$Lambda$1001/0x000000d8016ec350.doInAsyncProducer(Unknown Source) ~[na:na]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.support.cache.DefaultProducerCache.doInAsyncProducer(DefaultProducerCache.java:327) ~[camel-support-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:182) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:477) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:181) ~[camel-base-engine-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.impl.engine.DefaultReactiveExecutor.schedule(DefaultReactiveExecutor.java:54) ~[camel-base-engine-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.lambda$redeliver$2(RedeliveryErrorHandler.java:914) ~[camel-core-processor-3.21.0.jar!/:3.21.0]
Aug  3 07:38:37 ip-172-31-9-176 web: at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask$$Lambda$994/0x000000d8016c1360.done(Unknown Source) ~[na:na]

解决方案

启用Http组件流式传输模式
在Http端点添加streaming=true参数，让组件直接返回原始输入流，避免将内容缓存到内存。修改后的toD端点写法：
```
.toD("${header.CamelHttpUri}?streaming=true")
```
该配置会阻止Camel将响应体写入CachedOutputStream，直接传递原始InputStream到后续处理器。
禁用Exchange流缓存
设置Exchange.STREAM_CACHE头为false，强制Camel不缓存流数据，放在toD之前：
```
.setHeader(Exchange.STREAM_CACHE, constant(false))
```
确保处理器端流式兼容
处理器中获取请求体时，直接使用InputStream类型，避免调用会加载整个流到内存的方法：
```
InputStream inputStream = exchange.getIn().getBody(InputStream.class);
```
调整Jetty客户端超时配置
错误堆栈包含IdleTimeout日志，需延长连接和套接字超时时间，避免大文件传输时断开：
```
.toD("${header.CamelHttpUri}?streaming=true&connectionTimeout=300000&socketTimeout=300000")
```

内容的提问来源于stack exchange，提问作者Jawad Ahmad