优化循环中运行Curl的PHP代码以避免阻塞网站访问

我有一段在for循环中执行大量Curl请求的PHP代码，这是一个后台系统任务，但运行这段代码时，网站会一直加载无法访问，直到该任务/函数执行完毕。

原代码：

public function endcount() {

        $order_list = $this->order_model->get_rows([
            'where' => [
              "status IN ('Processing', 'In Progress') AND service_id = 1"
            ],
            'order_by' => 'quantity ASC',
            'limit' => '200'
        ]);

    
        foreach ($order_list as $key => $value) {

            $start_count = $value['start_count'];
            
            
            //service id = 1 is for followers
            if ($value['service_id'] == '1' || $value['service_id'] == '3' || $value['service_id'] == '5') {
                
                
                $end_count = 0;

                
                try {
                    
                
                    $username = $value['target'];
                    $proxy = '139.99.54.49:10163';
                    $proxyauth = 'username:password';

                    $url = 'https://www.instagram.com/'. $username .'/?__a=1&__d=dis';
                    $ch = curl_init();
                    curl_setopt($ch, CURLOPT_URL, $url);
                    curl_setopt($ch, CURLOPT_PROXY, $proxy);     // PROXY details with port
                    curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);   // Use if proxy have username and password
                    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
                    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
                    curl_setopt($ch, CURLOPT_HTTPHEADER, array('Accept: application/json') );
                    $data = curl_exec($ch);
                    if (curl_errno($ch)) {
                        $error_msg = curl_error($ch);
                        var_dump($error_msg);
                    }

                    $json_data = json_decode($data, true);
                    if ($json_data && array_key_exists('graphql', $json_data)) {
                        $end_count = $json_data['graphql']['user']['edge_followed_by']['count'];
                    } else {
                        var_dump($username . ' data ga bisa');
                        $end_count = 0;
                    }
                    

                    curl_close($ch);


                } catch(Exception $e) {
                    var_dump($e->getMessage());
                    $end_count = 0;
                }

                if ($end_count == -666) {
                    $end_count = $start_count + 5;
                }
                

                $total_followers = $start_count + $value['quantity'];
                $remains = $total_followers - $end_count;

                if ($remains <= 10) {
                    $status = 'Success';
                } else {
                    $status = 'Partial';
                }

            
                var_dump('end count of ' . $value['target'] . ' ' . $end_count . ' WITH REMAINS: ' . $remains);

                if ($status == 'Success') {
                    $update_order = [
                    'remains' => $remains,
                    'status' => $status,
                    'updated_at' => date('Y-m-d H:i:s'),
                    ];
                    $update_order = $this->order_model->update($update_order, ['id' => $value['id']]);
                    if ($update_order == true) {
                        print('ID '.$value['id'].' => ['.$status.'] - [FINAL COUNT : '.$end_count.'] - [REMAINS : '.$remains.'] - [TARGET COUNT : '.$total_followers.']<br>');
                    } else {
                        print('Error..');
                    }
                }
                
                //var_dump($value['target']);
            } else if ($value['service_id'] == '2') {
                //service id = 2 is for likers

            }
            
        }
        
}

优化方案 #

1. 改为后台异步执行（最彻底的解决方法） #

不要在Web请求上下文里运行这个耗时任务，把逻辑提取到独立的CLI脚本，通过系统定时任务（Cron）或消息队列触发执行：

• 步骤1：创建独立脚本process_orders.php，引入框架依赖后实现订单处理逻辑：

<?php
// 引入你的框架初始化文件（比如CodeIgniter的index.php或自定义入口）
require_once './index.php';

$order_model = new OrderModel(); // 根据你的框架调整实例化方式
$order_list = $order_model->get_rows([
    'where' => [
        "status IN ('Processing', 'In Progress') AND service_id = 1"
    ],
    'order_by' => 'quantity ASC',
    'limit' => '200'
]);

foreach ($order_list as $value) {
    // 原endcount函数中的处理逻辑（复制过来即可）
}

• 步骤2：用Cron定期执行脚本，比如每5分钟执行一次：

*/5 * * * * php /path/to/your/project/process_orders.php >> /var/log/order_process.log 2>&1

或者用消息队列（比如Redis），在Web请求中只把任务丢进队列，后台启动Worker进程持续处理队列任务，彻底隔离Web请求和耗时任务。

2. 使用Curl多并发请求减少总耗时 #

如果必须在Web环境临时运行，把串行的Curl请求改成并行批量处理，用curl_multi_*系列函数：

public function endcount() {
    $order_list = $this->order_model->get_rows([
        'where' => [
            "status IN ('Processing', 'In Progress') AND service_id = 1"
        ],
        'order_by' => 'quantity ASC',
        'limit' => '200'
    ]);

    $multiHandle = curl_multi_init();
    $handles = [];
    $orderMap = [];

    // 批量创建Curl句柄并添加到多句柄
    foreach ($order_list as $key => $value) {
        if (!in_array($value['service_id'], ['1','3','5'])) continue;

        $username = $value['target'];
        $proxy = '139.99.54.49:10163';
        $proxyauth = 'username:password';
        $url = 'https://www.instagram.com/'. $username .'/?__a=1&__d=dis';

        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_PROXY, $proxy);
        curl_setopt($ch, CURLOPT_PROXYUSERPWD, $proxyauth);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
        curl_setopt($ch, CURLOPT_HTTPHEADER, ['Accept: application/json']);
        curl_setopt($ch, CURLOPT_TIMEOUT, 10); // 设置超时避免卡住

        $handles[$key] = $ch;
        $orderMap[$key] = $value;
        curl_multi_add_handle($multiHandle, $ch);
    }

    // 执行多并发请求
    $running = null;
    do {
        curl_multi_exec($multiHandle, $running);
        curl_multi_select($multiHandle);
    } while ($running > 0);

    // 处理返回结果
    foreach ($handles as $key => $ch) {
        $value = $orderMap[$key];
        $end_count = 0;
        try {
            $data = curl_multi_getcontent($ch);
            if (curl_errno($ch)) {
                var_dump(curl_error($ch));
            } else {
                $json_data = json_decode($data, true);
                if ($json_data && isset($json_data['graphql']['user']['edge_followed_by']['count'])) {
                    $end_count = $json_data['graphql']['user']['edge_followed_by']['count'];
                } else {
                    var_dump($value['target'] . ' data ga bisa');
                }
            }
        } catch(Exception $e) {
            var_dump($e->getMessage());
        }

        // 后续的订单状态更新逻辑（和原代码一致）
        if ($end_count == -666) {
            $end_count = $value['start_count'] + 5;
        }
        $total_followers = $value['start_count'] + $value['quantity'];
        $remains = $total_followers - $end_count;
        $status = $remains <= 10 ? 'Success' : 'Partial';

        var_dump('end count of ' . $value['target'] . ' ' . $end_count . ' WITH REMAINS: ' . $remains);

        if ($status == 'Success') {
            $update_order = [
                'remains' => $remains,
                'status' => $status,
                'updated_at' => date('Y-m-d H:i:s'),
            ];
            $update_result = $this->order_model->update($update_order, ['id' => $value['id']]);
            if ($update_result) {
                print('ID '.$value['id'].' => ['.$status.'] - [FINAL COUNT : '.$end_count.'] - [REMAINS : '.$remains.'] - [TARGET COUNT : '.$total_followers.']<br>');
            } else {
                print('Error..');
            }
        }

        curl_multi_remove_handle($multiHandle, $ch);
        curl_close($ch);
    }

    curl_multi_close($multiHandle);
}

3. 立即返回响应并后台继续执行 #

如果必须通过Web请求触发，先给前端返回任务启动成功的响应，然后断开连接继续处理任务：

public function endcount() {
    // 关闭输出缓冲，立即返回响应
    ignore_user_abort(true); // 忽略客户端断开连接
    set_time_limit(0); // 取消脚本执行时间限制
    ob_start();
    echo json_encode(['status' => 'success', 'message' => '任务已启动，后台处理中']);
    $size = ob_get_length();
    header("Content-Length: $size");
    header('Connection: close');
    ob_end_flush();
    flush();

    // 后续的订单处理逻辑（原代码全部放在这里）
    $order_list = $this->order_model->get_rows([...]);
    foreach ($order_list as $value) {
        // ... 原循环逻辑
    }
}

4. 分批次处理并添加延迟 #

如果以上方法暂时无法实现，减少单次处理的订单数量，并在循环中加入延迟，避免瞬间占用过多资源：

public function endcount() {
    // 把limit从200改成20，分批次处理
    $order_list = $this->order_model->get_rows([
        'where' => [
            "status IN ('Processing', 'In Progress') AND service_id = 1"
        ],
        'order_by' => 'quantity ASC',
        'limit' => '20'
    ]);

    foreach ($order_list as $value) {
        // 原处理逻辑...
        
        // 每次请求后延迟0.5秒，减少请求频率
        usleep(500000);
    }
}

内容的提问来源于stack exchange，提问作者aherlambang