如何通过Proc::Simple传递含JSON数据的Perl数组至子进程？

核心思路 #

1. 将设备数组@DEVS拆分为n个近似等份的子数组
2. 用序列化工具将子数组转为可传递的字符串（无需依赖文件）
3. 通过Proc::Simple启动子进程，将序列化字符串通过标准输入传递给子进程
4. 子进程反序列化字符串恢复数组，执行Redfish查询逻辑
5. 主进程等待所有子进程完成后结束

具体实现步骤 #

1. 拆分设备数组 #

手动计算每份元素数量，通过数组切片生成子数组：

my $num_workers = 4; # 自定义子进程数量
my @DEVS = @{$DECODED_JSON->{'value'}};
my $total_devs = scalar @DEVS;
my $devs_per_worker = int($total_devs / $num_workers);
$devs_per_worker++ if $total_devs % $num_workers != 0; # 处理余数

my @worker_chunks;
for my $i (0..$num_workers-1) {
    my $start = $i * $devs_per_worker;
    last if $start >= $total_devs;
    my $end = ($i+1)*$devs_per_worker - 1;
    $end = $total_devs - 1 if $end >= $total_devs;
    push @worker_chunks, [ @DEVS[$start..$end] ];
}

2. 序列化与反序列化 #

因为原数据来自JSON，推荐用JSON模块直接处理：

# 主进程序列化子数组
use JSON;
my $json = JSON->new->utf8;
my $chunk_str = $json->encode($worker_chunk);

# 子进程反序列化恢复数组
my $devices = $json->decode($chunk_str);

若追求性能，也可使用Storable模块的freeze/thaw处理Perl原生结构：

use Storable qw(freeze thaw);
my $chunk_str = freeze($worker_chunk); # 主进程
my $devices = thaw($chunk_str); # 子进程

3. 启动子进程并传递数据 #

通过标准输入传递序列化数据（避免命令行参数长度限制）：

use Proc::Simple;

my @workers;
foreach my $chunk (@worker_chunks) {
    my $proc = Proc::Simple->new();
    $proc->start(sub {
        # 子进程读取标准输入的序列化数据
        my $input = do { local $/; <STDIN> };
        my $json = JSON->new->utf8;
        my $devices = $json->decode($input);
        
        # 执行Redfish查询逻辑
        foreach my $dev (@$devices) {
            my $redfish_url = $dev->{'RedfishUri'}; # 假设设备数据包含Redfish地址
            # 此处编写你的Redfish API调用代码
            print "Processing device: " . $dev->{'Id'} . "\n";
            # ... 具体查询逻辑
        }
    });
    
    # 主进程向子进程写入序列化数据
    print {$proc->stdin} $json->encode($chunk);
    close($proc->stdin);
    
    push @workers, $proc;
}

# 等待所有子进程完成
foreach my $proc (@workers) {
    $proc->wait();
}

完整示例代码 #

use strict;
use warnings;
use JSON;
use Proc::Simple;

# 模拟从OME API获取设备数据
my $DECODED_JSON = {
    'value' => [
        { Id => 'dev1', RedfishUri => 'https://dev1.example.com/redfish/v1' },
        { Id => 'dev2', RedfishUri => 'https://dev2.example.com/redfish/v1' },
        { Id => 'dev3', RedfishUri => 'https://dev3.example.com/redfish/v1' },
        { Id => 'dev4', RedfishUri => 'https://dev4.example.com/redfish/v1' },
        { Id => 'dev5', RedfishUri => 'https://dev5.example.com/redfish/v1' },
        # ... 更多设备数据
    ]
};
my @DEVS = @{$DECODED_JSON->{'value'}};

# 配置子进程数量
my $num_workers = 4;
my $total_devs = scalar @DEVS;
my $devs_per_worker = int($total_devs / $num_workers);
$devs_per_worker++ if $total_devs % $num_workers != 0;

# 拆分数组
my @worker_chunks;
for my $i (0..$num_workers-1) {
    my $start = $i * $devs_per_worker;
    last if $start >= $total_devs;
    my $end = ($i+1)*$devs_per_worker - 1;
    $end = $total_devs - 1 if $end >= $total_devs;
    push @worker_chunks, [ @DEVS[$start..$end] ];
}

# 启动子进程
my $json = JSON->new->utf8;
my @workers;
foreach my $chunk (@worker_chunks) {
    my $proc = Proc::Simple->new();
    $proc->start(sub {
        my $input = do { local $/; <STDIN> };
        my $devices = $json->decode($input);
        
        foreach my $dev (@$devices) {
            # 替换为你的Redfish查询逻辑
            print "Processed device " . $dev->{Id} . " via " . $dev->{RedfishUri} . "\n";
            # 模拟API调用延迟
            sleep int(rand(3));
        }
    });
    
    print {$proc->stdin} $json->encode($chunk);
    close($proc->stdin);
    push @workers, $proc;
}

# 等待所有子进程结束
print "Waiting for all workers to finish...\n";
foreach my $proc (@workers) {
    $proc->wait();
}
print "All tasks completed.\n";

注意事项 #

• 若设备数量小于子进程数量，自动将子进程数调整为设备总数，避免空进程
• 子进程中需做好Redfish API的错误捕获（超时、连接失败等），避免异常退出
• 大体积数据优先用标准输入传递，规避命令行参数长度限制

内容的提问来源于stack exchange，提问作者Denis.A