如何用PHP或Perl实现向多个URL端点发送数据？

针对共享服务器的限制，核心思路是用队列+异步非阻塞处理替代fork与高频cron轮询，既保证重试机制，又避免资源过载。

核心优化逻辑 #

1. 把待转发任务存入队列（文件/轻量数据库），记录目标URL、POST数据、重试次数、下次重试时间（模仿ClickBank的指数退避间隔：1分钟→5分钟→15分钟→30分钟→1小时→6小时）
2. 接收ClickBank INS请求的端点直接返回200，不用等待转发完成（避免ClickBank对你的端点重试）
3. 用定时脚本批量处理队列任务，限制并发数，非阻塞发送请求，成功则删除任务，失败则更新重试时间放回队列

PHP 实现方案 #

1. 队列存储（SQLite示例） #

创建SQLite数据库表存储任务：

CREATE TABLE IF NOT EXISTS notification_tasks (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    target_url TEXT NOT NULL,
    post_data TEXT NOT NULL,
    retry_count INTEGER DEFAULT 0,
    next_retry INTEGER NOT NULL -- Unix时间戳
);

2. 接收ClickBank INS的端点脚本 #

<?php
// 验证ClickBank请求（可选，按官方文档做签名验证）
// ...

// 获取POST数据
$postData = file_get_contents('php://input');

// 要转发的目标URL列表（比如从配置或数据库读取）
$targetUrls = ['https://your-site.com/handler', 'https://stripe.com/webhook'];

// 写入队列
$db = new SQLite3('notification_queue.db');
foreach ($targetUrls as $url) {
    $db->exec("INSERT INTO notification_tasks (target_url, post_data, next_retry) 
               VALUES ('" . $db->escapeString($url) . "', '" . $db->escapeString($postData) . "', " . time() . ")");
}
$db->close();

// 立即返回200给ClickBank
http_response_code(200);
echo "OK";
?>

3. 队列处理脚本（配合cron每5分钟运行一次） #

<?php
$db = new SQLite3('notification_queue.db');
// 取出到重试时间的任务，限制每次处理10个
$tasks = $db->query("SELECT * FROM notification_tasks WHERE next_retry <= " . time() . " LIMIT 10");

if (!$tasks) exit;

// 初始化curl多句柄（非阻塞批量请求）
$mh = curl_multi_init();
$handles = [];

// 批量创建curl请求
while ($task = $tasks->fetchArray(SQLITE3_ASSOC)) {
    $ch = curl_init($task['target_url']);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $task['post_data']);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_TIMEOUT, 10);
    curl_multi_add_handle($mh, $ch);
    $handles[(int)$ch] = $task['id'];
}

// 执行非阻塞请求
$running = null;
do {
    curl_multi_exec($mh, $running);
    curl_multi_select($mh);
} while ($running > 0);

// 处理请求结果
foreach ($handles as $chId => $taskId) {
    $httpCode = curl_getinfo($chId, CURLINFO_HTTP_CODE);
    if ($httpCode === 200) {
        // 成功，删除任务
        $db->exec("DELETE FROM notification_tasks WHERE id = $taskId");
    } else {
        // 失败，更新重试次数与下次重试时间
        $task = $db->querySingle("SELECT retry_count FROM notification_tasks WHERE id = $taskId", true);
        $retryCount = $task['retry_count'] + 1;
        // 指数退避设置延迟
        $delay = match($retryCount) {
            1 => 60, 2 => 300, 3 => 900, 4 => 1800, 5 => 3600,
            default => 21600 // 超过5次后每6小时重试
        };
        $nextRetry = time() + $delay;
        $db->exec("UPDATE notification_tasks SET retry_count = $retryCount, next_retry = $nextRetry WHERE id = $taskId");
    }
    curl_multi_remove_handle($mh, $chId);
    curl_close($chId);
}

curl_multi_close($mh);
$db->close();
?>

Perl 实现方案 #

1. 队列存储（SQLite示例） #

同PHP的数据库表结构，用Perl的DBI模块操作。

2. 接收ClickBank INS的CGI脚本 #

#!/usr/bin/perl
use CGI;
use DBI;
use JSON;

my $cgi = CGI->new;
my $post_data = $cgi->param('POSTDATA');

# 目标URL列表
my @target_urls = ('https://your-site.com/handler', 'https://stripe.com/webhook');

# 写入队列
my $dbh = DBI->connect("dbi:SQLite:dbname=notification_queue.db", "", "");
foreach my $url (@target_urls) {
    $dbh->do("INSERT INTO notification_tasks (target_url, post_data, next_retry) VALUES (?, ?, ?)", undef, $url, $post_data, time());
}
$dbh->disconnect;

# 返回200
print "Content-Type: text/plain\n";
print "Status: 200 OK\n\n";
print "OK";

3. 队列处理脚本（配合cron每5分钟运行一次） #

#!/usr/bin/perl
use AnyEvent;
use AnyEvent::HTTP;
use DBI;

my $dbh = DBI->connect("dbi:SQLite:dbname=notification_queue.db", "", "");
# 取出到时间的任务，限制每次处理10个
my $tasks = $dbh->selectall_arrayref("SELECT * FROM notification_tasks WHERE next_retry <= ? LIMIT 10", undef, time());
exit unless @$tasks;

my $cv = AnyEvent->condvar;
my $max_concurrent = 10; # 限制并发数
my $current_concurrent = 0;

foreach my $task (@$tasks) {
    $cv->begin;
    $current_concurrent++;
    # 控制并发，超过上限则等待
    if ($current_concurrent >= $max_concurrent) {
        my $wait_cv = AnyEvent->condvar;
        my $timer = AnyEvent->timer(after => 0.1, cb => sub { $wait_cv->send });
        $wait_cv->recv;
        $current_concurrent--;
    }

    # 非阻塞POST请求
    http_post $task->[1], $task->[2], timeout => 10, sub {
        my ($body, $hdr) = @_;
        if ($hdr->{Status} == 200) {
            $dbh->do("DELETE FROM notification_tasks WHERE id = ?", undef, $task->[0]);
        } else {
            my $retry_count = $task->[3] + 1;
            my $delay = $retry_count == 1 ? 60 : $retry_count == 2 ? 300 : $retry_count ==3 ?900 : $retry_count ==4?1800 : $retry_count ==5?3600 : 21600;
            my $next_retry = time() + $delay;
            $dbh->do("UPDATE notification_tasks SET retry_count = ?, next_retry = ? WHERE id = ?", undef, $retry_count, $next_retry, $task->[0]);
        }
        $current_concurrent--;
        $cv->end;
    };
}

$cv->recv;
$dbh->disconnect;

关键注意事项 #

• 并发控制：根据共享服务器的配置调整每次处理的任务数（比如5-10个），避免触发主机商的资源限制
• 请求超时：设置合理的超时时间（10秒以内），避免单个请求阻塞整个处理流程
• 指数退避：严格按照递增间隔重试，避免频繁请求目标服务器导致被封禁
• 任务清理：可定期清理重试超过N次（比如10次）的任务，避免队列无限膨胀

内容的提问来源于stack exchange，提问作者Cristofayre