Android多文件下载中InputStream.read()读取缓慢问题求助

原本实现多文件下载到Android存储时速度很快，相关日志：

2023-10-06 13:02:26.412  D  File downloaded: v6:file_1696590146411 | 288 ms
2023-10-06 13:02:26.412  D  File downloaded: v6:file_1696590146412 | 288 ms
2023-10-06 13:02:26.419  D  File downloaded: v6:file_1696590146418 | 295 ms
2023-10-06 13:02:26.572  D  File downloaded: v6:file_1696590146572 | 448 ms
2023-10-06 13:02:26.603  D  File downloaded: v6:file_1696590146602 | 480 ms

但添加缓冲区读取写入逻辑后，下载耗时大幅增加，日志如下：

2023-10-06 12:59:14.602  D  File downloaded: v5:file_1696589953363 | 1452 ms
2023-10-06 12:59:16.299  D  File downloaded: v5:file_1696589953401 | 3148 ms
2023-10-06 12:59:16.796  D  File downloaded: v5:file_1696589953400 | 3646 ms
2023-10-06 12:59:17.088  D  File downloaded: v5:file_1696589953399 | 3938 ms
2023-10-06 12:59:18.882  D  File downloaded: v5:file_1696589953398 | 5731 ms

目前核心读写逻辑为：

while ((bytesRead = inputStream.read(buffer)) != -1) { outputStream.write(buffer, 0, bytesRead); }

附上项目代码：

MainActivity.java #

package com.kerman.testapp;

import androidx.appcompat.app.AppCompatActivity;

import android.os.Bundle;

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);


        String[] urls = {
                "https://fastly.picsum.photos/id/16/2500/1667.jpg?hmac=uAkZwYc5phCRNFTrV_prJ_0rP0EdwJaZ4ctje2bY7aE",
                "https://fastly.picsum.photos/id/19/2500/1667.jpg?hmac=7epGozH4QjToGaBf_xb2HbFTXoV5o8n_cYzB7I4lt6g",
                "https://fastly.picsum.photos/id/22/4434/3729.jpg?hmac=fjZdkSMZJNFgsoDh8Qo5zdA_nSGUAWvKLyyqmEt2xs0",
                "https://fastly.picsum.photos/id/21/3008/2008.jpg?hmac=T8DSVNvP-QldCew7WD4jj_S3mWwxZPqdF0CNPksSko4",
                "https://fastly.picsum.photos/id/25/5000/3333.jpg?hmac=yCz9LeSs-i72Ru0YvvpsoECnCTxZjzGde805gWrAHkM"
        };

        DownloadTask downloadTask = new DownloadTask(this); 
        downloadTask.execute(urls);
    }
}

DownloadTask.java #

package com.kerman.testapp;

import android.content.Context;
import android.os.AsyncTask;
import android.util.Log;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;

public class DownloadTask extends AsyncTask<String, Void, Void> {
    private static final String TAG = "DownloadTask";
    private final Context context;

    public DownloadTask(Context context) {
        this.context = context;
    }

    @Override
    protected Void doInBackground(String... urls) {
        // Use ExecutorService to download files concurrently
        ExecutorService executor = Executors.newFixedThreadPool(urls.length);

        try {
            for (String url : urls) {
                executor.submit(() -> downloadFile(url));
            }
        } finally {
            // Shut down the executor when done
            executor.shutdown();
        }

        return null;
    }

    private void downloadFile(String urlString) {
        Log.d(TAG, "downloadFile(): " + urlString);
        long unixTime = System.currentTimeMillis();

        try {
            URL url = new URL(urlString);
            HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();

            try {
                InputStream inputStream = urlConnection.getInputStream();

                // Get the app-specific external files directory
                String externalFilesDir = context.getExternalFilesDir(null).getPath();

                // Create a timestamped filename
                String fileName = "file_" + System.currentTimeMillis();

                // Create a FileOutputStream pointing to the external files directory and the filename
                FileOutputStream outputStream = new FileOutputStream(externalFilesDir + "/" + fileName);

                byte[] buffer = new byte[1024];
                int bytesRead;
                while ((bytesRead = inputStream.read(buffer)) != -1) {
                    outputStream.write(buffer, 0, bytesRead);
                }


                Log.d(TAG, "File downloaded: v6:" + fileName+" | "+Long.toString((System.currentTimeMillis()) - unixTime)+" ms");

                // Close the streams
                outputStream.close();
                inputStream.close();
            } finally {
                urlConnection.disconnect();
            }
        } catch (IOException e) {
            Log.e(TAG, "Error downloading file from " + urlString, e);
        }
    }
}

优化建议 #

• 增大缓冲区大小：当前使用的1024字节缓冲区过小，建议调整为8192（8KB）、16384（16KB）或32768（32KB），减少IO操作的频次。
• 使用带缓冲的流包装类：用BufferedInputStream包装原始输入流，BufferedOutputStream包装原始输出流，借助内置的缓冲机制提升读写效率，示例代码：

  InputStream bufferedInput = new BufferedInputStream(urlConnection.getInputStream());
  OutputStream bufferedOutput = new BufferedOutputStream(outputStream);
  

• 利用已引入的Apache Commons IO工具类：项目已导入IOUtils，直接调用IOUtils.copy(inputStream, outputStream)替代手动循环读写，该方法内部已做缓冲区优化，代码更简洁高效。
• 优化线程池配置：当前线程池大小设为文件总数（5个），对于网络IO密集型任务可行；若后续文件数量增多，建议设置为Runtime.getRuntime().availableProcessors() * 2，避免过多线程引发资源竞争。
• 使用File对象构建输出路径：避免字符串拼接路径的潜在错误，改为File outputFile = new File(context.getExternalFilesDir(null), fileName)，再通过new FileOutputStream(outputFile)创建输出流，更安全可靠。

内容的提问来源于stack exchange，提问作者Vitalii