PHP分片上传超100MB视频合并后文件损坏问题排查

实现方案 #

实现从Unity3D编辑器向本地WAMP服务器上传1MB~2GB不等的视频文件，方案逻辑为：

• 将每个视频拆分为单块固定大小的字节数组分片
• 按队列顺序为分片标注序列号后串行上传，仅当上一个分片上传成功后才启动下一分片上传
• 所有分片上传完成后，在服务端合并为完整文件

现有服务端PHP代码 #

define("CHUNK_FILE_EXTENSION", ".part");

if($_SERVER['REQUEST_METHOD'] == "POST") 
{
    $folder_name = isset($_POST['folder_name']) ? trim($_POST['folder_name']) : '';
    $target_file_name = isset($_POST['target_file_name']) ? trim($_POST['target_file_name']) : '';
    $chunkByteArray = isset($_FILES['chunk_byte_array']) ? $_FILES['chunk_byte_array'] : '';
    $currentChunkNumber = isset($_POST['current_chunk_number']) ? trim($_POST['current_chunk_number']) : '';
    $totalChunksNumber = isset($_POST['total_chunks_number']) ? trim($_POST['total_chunks_number']) : '';
    $startMerge = isset($_POST['start_merge']) ? trim($_POST['start_merge']) : '';
    $totalFileSize = isset($_POST['total_file_size']) ? trim($_POST['total_file_size']) : '';
    $startRollback = isset($_POST['start_rollback']) ? trim($_POST['start_rollback']) : '';
    
    function targetFileDirectoryPath($folder_name) {
        //$tempDir = $_SERVER['DOCUMENT_ROOT']."\\media\\temp\\test\\%s";
        $tempDir = $_SERVER['DOCUMENT_ROOT']."\\media\\temp\\test";
        return sprintf($tempDir, $folder_name);
    }

    function chunksFileDirectoryPath($folder_name) {
        return CombinePath(targetFileDirectoryPath($folder_name), "chunks");
    }

    function mergeChunkFiles($targetFileName, $chunkFileDir, $targetFileTempPath) {
        $files = array_diff(scandir($chunkFileDir), array('.','..',$targetFileName));
        sort($files);
        
        $final = fopen($targetFileTempPath, 'w');
        foreach ($files as $file) {
            $filePath = CombinePath($chunkFileDir, $file);
            if(($filePath != $targetFileTempPath) && (filesize($filePath) > 0)) {
                $myfile = fopen($filePath, "r");
                $buff = fread($myfile,filesize($filePath));
                $write = fwrite($final, $buff);
                fclose($myfile);
            }
        }
        fclose($final);
    }

    if (!empty($currentChunkNumber) && !empty($totalChunksNumber) && !empty($chunkByteArray)) {
        $chunkFileDir = chunksFileDirectoryPath($folder_name);
        $chunkFilePath = CombinePath($chunkFileDir, $currentChunkNumber.CHUNK_FILE_EXTENSION);
        $tempPath = $chunkByteArray['tmp_name'];
        if (createDirectory($chunkFileDir)) {
            if(move_uploaded_file($tempPath, $chunkFilePath)) {
                $responseJson = array(
                    "status" => 1, 
                    "message" => $currentChunkNumber." uploaded successfully"
                );
            }
            else {
                $responseJson = array(
                    "status" => 2, 
                    "message" => $currentChunkNumber." not uploaded to ".$chunkFilePath." from ".$tempPath, 
                    "uploaded_chunk_file" => $chunkByteArray, 
                    "is_uploaded_file" => is_uploaded_file($tempPath)
                );
            }
        }
        else {
            $responseJson = array(
                "status" => 3, 
                "message" => "Chunk file user directory not created @ ".$chunkFileDir
            );
        }
    }
    else if (!empty($startMerge) && !empty($totalFileSize)) {
        $targetFileName = $target_file_name;
        $chunkFileDir = chunksFileDirectoryPath($folder_name);
        $targetFileTempDir = NormalizePath(targetFileDirectoryPath($folder_name));
        $targetFileTempPath = CombinePath($targetFileTempDir, $targetFileName);

        if(createDirectory($targetFileTempDir)) {
            mergeChunkFiles($targetFileName, $chunkFileDir, $targetFileTempPath);
            removeFolder($chunkFileDir);
            if (filesize($targetFileTempPath) == $totalFileSize) {
                $responseJson = array(
                    "status" => 1, 
                    "message" => "Target file saved successfully!"
                );
            }
            else {
                $responseJson = array(
                    "status" => 2, 
                    "message" => "Target file size doesn't match with actual file size. ".
                        "Please try again! Target File Size: ".filesize($targetFileTempPath).
                        " & Input File Size: ".$totalFileSize);
            }   
        }
        else {
            $responseJson = array(
                "status" => 3, 
                "message" => "Unable to create target directory for merging chunks @ ".$targetFileTempDir
            );
        }
    }
    else if (!empty($startRollback)) {
        $responseJson = array(
            "status" => 4, 
            "message" => "Rollback successful!"
        );
    }
    else {
        $responseJson = array(
            "status" => 0, 
            "message" => "Invalid request parameters!!"
        );
    }
}
else {
    $responseJson = array(
        "status" => 0, 
        "message" => "Invalid request method!!"
    );
}

/* Output header */
header('Content-type: application/json;charset=utf-8');
echo json_encode($responseJson, JSON_UNESCAPED_UNICODE);

//Remove folder and its inner folder and files at the input path
function removeFolder($folder) {
    if (empty($folder)) {
        return;
    }
    $folder = NormalizePath($folder);
    if(is_file($folder)) {
        unlink($folder);
    }
    else if(is_dir($folder)) {
        $files = scandir($folder);
        foreach($files as $file) {
            if (( $file != '.' ) && ( $file != '..' )) {
                $file = CombinePath($folder, $file);
                if(is_dir($file)) {
                    removeFolder($file);
                }
                else {
                    unlink($file);n                }
            }
        }
        rmdir($folder);
    }
}

//Check if directory is exist return true, else create new directory and returns bool
function createDirectory($directoryPath) {
    $directoryPath = NormalizePath($directoryPath);
    if(!is_dir($directoryPath)) {
        return mkdir($directoryPath, 0775, true);
    }
    else {
        return true;
    }
}

//Method to combine local file or folder paths using a DIRECTORY_SEPARATOR
function NormalizePath($path) 
{
    //normalize
    $path = str_replace('/', DIRECTORY_SEPARATOR, $path);
    $path = str_replace('\\', DIRECTORY_SEPARATOR, $path);

    //remove leading/trailing dir separators
    if(!empty($path) && substr($path, -1) == DIRECTORY_SEPARATOR) {
        $path = substr($path, 0, -1);
    }
    return $path;
}

//Method to combine local file or folder paths using a DIRECTORY_SEPARATOR
function CombinePath($one, $other, $normalize = true) 
{
    //normalize
    if($normalize) {
        $one = NormalizePath($one);
        $other = NormalizePath($other);
    }
    
    //remove leading/trailing dir separators
    if(!empty($one)) {
        $one = rtrim($one, DIRECTORY_SEPARATOR);
    }
    if(!empty($other)) {
        $other = ltrim($other, DIRECTORY_SEPARATOR);
    }
    
    //return combined path
    if(empty($one)) {
        return $other;
    } elseif(empty($other)) {
        return $one;
    } else {
        return $one.DIRECTORY_SEPARATOR.$other;
    }
}

?>

故障现象 #

• 小于100MB的视频经上述流程上传合并后可正常播放，效果与原文件一致
• 大于100MB的视频接口返回上传成功，服务器上最终合并的文件大小与客户端源文件大小完全一致，但视频损坏无法正常播放
• 本地WAMP环境php.ini中upload_max_filesize、post_max_size参数均设置为20M，将分片大小从10MB调整为5MB后问题仍然存在
• 使用ffmpeg对106MB大小的损坏视频执行错误检测命令ffmpeg -v error -i {video_file_path} -f null - 2>{error_log_file_path}，确认文件存在字节流顺序错误导致的结构损坏

根因分析 #

1. 分片排序逻辑错误（核心原因）
   合并分片时使用sort()对分片文件名字符串做字典序排序，分片命名规则为[分片序号].part。字典序下，10.part会排在2.part前面，当文件分片数小于10（即10MB分片下文件小于100MB）时排序结果正确，一旦分片数≥10，合并顺序就会错乱。由于所有分片内容都被写入最终文件，总文件大小和源文件完全一致，但字节流顺序错误，导致视频结构损坏无法播放，和100MB的故障阈值完全吻合。
2. 文件读写模式非二进制安全
   Windows环境下fopen()默认使用文本模式读写，会自动对换行符做转义处理，可能导致二进制文件内容被篡改。
3. 代码语法笔误
   removeFolder函数中unlink($file);后多了多余的字符n，会触发语法错误，导致分片上传完成后临时分片目录无法正常删除，残留垃圾文件。
4. 大文件内存溢出风险
   合并分片时一次性读取整个分片内容写入目标文件，当分片较大时会超出PHP内存限制，导致分片读写不完整。

修复方案 #

1. 修复分片排序逻辑 #

替换原有的字符串sort()逻辑，提取分片文件名中的序号转为整数，按数值大小排序，确保分片合并顺序和上传顺序一致。

2. 使用二进制安全的文件读写模式 #

打开文件时统一使用rb（二进制只读）、wb（二进制写入）模式，避免Windows环境下的换行符转义问题。

3. 分块读写文件 #

读写分片时使用8KB缓冲区循环读取，避免一次性加载大分片导致内存溢出。

4. 修复语法笔误 #

删除removeFolder函数中unlink($file);后多余的n字符。

5. （可选优化）增加校验逻辑 #

• 保存分片时将传入的分片序号转为整数后再命名，避免非法参数导致的排序异常、路径遍历问题
• 合并完成后除了校验文件大小，可增加文件MD5校验，彻底避免顺序错误、内容篡改导致的损坏文件漏判

修复后的mergeChunkFiles函数代码如下：

function mergeChunkFiles($targetFileName, $chunkFileDir, $targetFileTempPath) {
    $files = array_diff(scandir($chunkFileDir), array('.','..',$targetFileName));
    // 按分片序号数值排序，替换原字符串字典序排序
    usort($files, function($a, $b) {
        $numA = intval(pathinfo($a, PATHINFO_FILENAME));
        $numB = intval(pathinfo($b, PATHINFO_FILENAME));
        return $numA - $numB;
    });
    
    // 二进制写入模式打开目标文件
    $final = fopen($targetFileTempPath, 'wb');
    foreach ($files as $file) {
        $filePath = CombinePath($chunkFileDir, $file);
        if(($filePath != $targetFileTempPath) && (filesize($filePath) > 0)) {
            // 二进制只读模式打开分片
            $myfile = fopen($filePath, "rb");
            // 分块读取写入，避免内存溢出
            while (!feof($myfile)) {
                $buff = fread($myfile, 8192);
                fwrite($final, $buff);
            }
            fclose($myfile);
        }
    }
    fclose($final);
}

修复分片保存路径的代码片段，将原分片路径生成逻辑：

$chunkFilePath = CombinePath($chunkFileDir, $currentChunkNumber.CHUNK_FILE_EXTENSION);

替换为：

$chunkFilePath = CombinePath($chunkFileDir, intval($currentChunkNumber).CHUNK_FILE_EXTENSION);

内容的提问来源于stack exchange，提问作者Sanket Kale