使用OneDrive API上传大文件遇Java堆内存溢出问题求助

尝试通过OneDrive API上传largefile.zip时失败，报错：

Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space.

上传会话创建成功，但上传文件字节数据环节执行失败。已尝试以下操作但未解决：

• 配置spring.servlet.multipart.max-file-size=3000MB、spring.servlet.multipart.max-request-size=3000MB
• 将RestTemplate替换为HttpURLConnection
• 调整JVM堆内存大小

推测是创建了超出JVM堆内存的对象导致，相关实现代码如下：

@RequestMapping(value = "/UploadFile", method = RequestMethod.POST)
public void UpFileOD( @RequestBody RequestUploadFile f ) {
    device_managementResponse response = new device_managementResponse();

    // GET https://graph.microsoft.com/v1.0/users/+mail to get user_id
    final String user_id = "2ada8e58-8d32-42d0-9928-9cbc3ae"; // Can change (depend on the app)

    RestTemplate restTemplate = new RestTemplate();

    try {
        // Create Upload Session
        //File file = new File(file_name.getAbsoluteFile().toString());
        File file = new File(f.file_name);
        
        if (!file.isFile())
            throw new Exception();

        long file_size = file.length();

        //final String session_url = "https://graph.microsoft.com/v1.0/users/"+user_id+"/drive/items/root"+path+"/"+file.getName()+":/createUploadSession";
        final String session_url = "https://graph.microsoft.com/v1.0/users/"+user_id+"/drive/items/root:/"+file.getName()+":/createUploadSession";
        // Create request header
        HttpHeaders session_header = new HttpHeaders();
        session_header.setContentType(MediaType.APPLICATION_JSON);
        session_header.add("Authorization", "Bearer " +AccessToken());

        // Create request body
        JSONObject  json_request_body_nested = new JSONObject();
        json_request_body_nested.put("@odata.type","microsoft.graph.driveItemUploadableProperties");
        json_request_body_nested.put("@microsoft.graph.conflictBehavior","rename");
        json_request_body_nested.put("name",file.getName());

        JSONObject json_request_body = new JSONObject();
        json_request_body.put("item", json_request_body_nested);

        // Create HttpEntity
        HttpEntity<JSONObject> session_entity = new HttpEntity<>(json_request_body,session_header );

        // Call Session Upload API
        ResponseEntity<String> session_upload = restTemplate.exchange(session_url, HttpMethod.PUT,session_entity,String.class );

        // Map response body of Session Upload
        ObjectMapper session_upload_map = new ObjectMapper();
        SessionUpload sessionUpload_obj = session_upload_map.readValue(session_upload.getBody(),SessionUpload.class);

        // Upload file
        // URL of upload file depends on the response of the upload session


        FileSystemResource fileSystemResource = new FileSystemResource(file);
//       Create request header
        HttpHeaders upload_header = new HttpHeaders();
        upload_header.setContentType(MediaType.APPLICATION_OCTET_STREAM);
        upload_header.setContentLength(file_size);
//      upload_header.add("Content-Range","bytes 0-"+(file_size-1)+"/"+file_size);
        upload_header.add("Content-Range","*/"+file_size);
        // Create entity
        HttpEntity<FileSystemResource> upload_entity = new HttpEntity<>(fileSystemResource,upload_header);

        // Call API
        ResponseEntity<String> uploading_file = restTemplate.exchange(sessionUpload_obj.getUploadURL(),HttpMethod.PUT,upload_entity,String.class);
//      System.out.println("Upload Success "+ "File Name: "+ file.getName() +" Path: "+path);

        System.out.println("Upload Success "+ "File Name: "+ file.getName());
        response.setRespcode(ResponseMessage.KEY_RC_SUCCESS);
        response.setMessage(ResponseMessage.getMessage(ResponseMessage.KEY_RC_SUCCESS));

    } catch (Exception e) {
        e.printStackTrace();
        response.setRespcode(ResponseMessage.KEY_RC_EXCEPTION);
        response.setMessage(e.getMessage());
    }
}

问题根源是你当前代码直接一次性上传整个大文件，FileSystemResource会把文件全部加载到内存，触发OOM。OneDrive API本身支持分块上传，这是解决大文件上传内存溢出的核心方案，具体实现步骤如下：

1. 实现分块上传逻辑 #

OneDrive要求分块大小为320KB的倍数，建议选择5MB-100MB之间的分块大小（示例用10MB），平衡内存占用和请求次数：

// 替换原有的上传文件代码块
long chunkSize = 10 * 1024 * 1024; // 10MB分块
long fileSize = file.length();
long offset = 0;

try (FileInputStream fis = new FileInputStream(file)) {
    while (offset < fileSize) {
        long end = Math.min(offset + chunkSize - 1, fileSize - 1);
        byte[] chunk = new byte[(int)(end - offset + 1)];
        fis.read(chunk);

        // 构造分块请求头
        HttpHeaders uploadHeader = new HttpHeaders();
        uploadHeader.setContentType(MediaType.APPLICATION_OCTET_STREAM);
        uploadHeader.add("Content-Range", String.format("bytes %d-%d/%d", offset, end, fileSize));

        // 仅将当前分块加载到内存
        ByteArrayResource chunkResource = new ByteArrayResource(chunk);
        HttpEntity<ByteArrayResource> uploadEntity = new HttpEntity<>(chunkResource, uploadHeader);

        // 上传分块
        ResponseEntity<String> response = restTemplate.exchange(
                sessionUpload_obj.getUploadURL(),
                HttpMethod.PUT,
                uploadEntity,
                String.class
        );

        // 201状态码表示所有分块上传完成
        if (response.getStatusCode() == HttpStatus.CREATED) {
            System.out.println("Upload Success "+ "File Name: "+ file.getName());
            response.setRespcode(ResponseMessage.KEY_RC_SUCCESS);
            response.setMessage(ResponseMessage.getMessage(ResponseMessage.KEY_RC_SUCCESS));
            break;
        }
        // 更新偏移量，准备下一块上传
        offset = end + 1;
    }
} catch (IOException e) {
    e.printStackTrace();
    response.setRespcode(ResponseMessage.KEY_RC_EXCEPTION);
    response.setMessage(e.getMessage());
}

2. 额外优化建议 #

• 彻底放弃FileSystemResource上传大文件：默认实现会把文件全量加载到内存，改用流式读取+分块上传的方式
• 合理调整JVM堆内存：如果分块后仍有OOM，可适当调大堆内存（例如-Xmx4g），但分块才是根本解决办法
• 添加断点续传逻辑：记录每次成功上传的偏移量，若上传中断可从断点继续，避免重复上传已完成的分块
• 及时释放对象引用：每次分块上传完成后，手动将chunk数组置为null，帮助GC快速回收内存

内容的提问来源于stack exchange，提问作者user23886729