使用commons-fileupload上传40M大文件耗时20秒，是否正常？

Hey there! Let's unpack your question—uploading a 40MB file taking 20 seconds can be totally normal or a sign of room for improvement, depending on your environment and setup. Let's break this down:

First, calculate the transfer speed #

40MB translates to roughly 320 megabits. At 20 seconds, that works out to ~16 Mbps. Here's how that stacks up:

• Local test (server + client on the same machine): This speed is definitely slower than expected. Even on a regular mechanical HDD, writing 40MB locally should take just a few seconds max.
• Network upload:
  • On a 10 Mbps connection: Theoretical max speed is ~1.25 MB/s, so 40MB would take ~32 seconds—meaning 20 seconds is actually pretty solid.
  • On a 100 Mbps connection: Theoretical max is ~12.5 MB/s, so 40MB should take ~3 seconds. In this case, 20 seconds is way slower than normal.

Possible reasons for slow local uploads #

If you're testing locally, here are the most common culprits:

• Small default buffer size: The DiskFileItemFactory uses a tiny default size threshold (usually 10KB). This forces the library to write data to disk in tiny chunks, which is super inefficient for large files. Try bumping it up to 1MB or higher:

  DiskFileItemFactory factory = new DiskFileItemFactory();
  // Set buffer size to 1MB to reduce frequent disk writes
  factory.setSizeThreshold(1024 * 1024);
  

• Inefficient IO methods: While Commons-IO's FileUtils.copyInputStreamToFile works, you might get better results with NIO's FileChannel for direct byte transfers (it skips some intermediate buffer steps).
• Disk bottlenecks: Check if your disk is under heavy load (e.g., other processes writing large files) or if it's an older mechanical drive with slow read/write speeds.
• Server configuration: For servlet containers like Tomcat, connector settings like socketBuffer size or unused compression can drag down performance. Double-check these settings in your server config.

Steps to troubleshoot #

1. Isolate network vs. local: Run the server and client on the same machine first. If it's still slow, focus on code, disk, or server config.
2. Tweak buffer settings: Adjust the sizeThreshold in DiskFileItemFactory to cut down on disk I/O overhead.
3. Monitor disk usage: Use tools like Task Manager (Windows) or iostat (Linux) to see if disk I/O is the bottleneck.
4. Try NIO for faster transfers: Replace stream-based copy with NIO for potentially better speed. Example snippet:

   InputStream inputStream = fileItem.getInputStream();
   FileChannel outChannel = new FileOutputStream(uploadFile).getChannel();
   outChannel.transferFrom(Channels.newChannel(inputStream), 0, Long.MAX_VALUE);
   outChannel.close();
   inputStream.close();
   

Final takeaway #

If you're working with a slow network, 20 seconds might be totally normal. But for local transfers, it's worth optimizing your buffer sizes and checking for disk or server bottlenecks to speed things up.

内容的提问来源于stack exchange，提问作者aeontree