Azure Pipeline执行Yocto构建报runqueue未捕获BrokenPipe异常

DevOps 小满

2026-8-28

问题背景

在Windows系统上通过Hyper-V创建Linux虚拟机，手动执行Yocto构建全流程可正常生成产物，但将近似相同的构建步骤配置到Azure Pipeline中执行时，构建在Initialising tasks阶段失败并抛出未捕获异常。当前无Azure构建机器的访问权限，无法直接查看Yocto源码开展调试，无法明确故障触发原因，需要对应的调试思路与解决方案。

报错信息

2022-06-23T00:57:52.6893303Z **Initialising tasks...ERROR: An uncaught exception occurred in runqueue**
2022-06-23T00:57:52.6894429Z Traceback (most recent call last):
2022-06-23T00:57:52.6896192Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1524, in RunQueue.execute_runqueue():
2022-06-23T00:57:52.6896660Z              try:
2022-06-23T00:57:52.6896927Z     >            return self._execute_runqueue()
2022-06-23T00:57:52.6897250Z              except bb.runqueue.TaskFailure:
2022-06-23T00:57:52.6898244Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1474, in RunQueue._execute_runqueue():
2022-06-23T00:57:52.6899084Z                  self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6899368Z     >            self.start_worker()
2022-06-23T00:57:52.6899675Z                  self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6900270Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1295, in RunQueue.start_worker():
2022-06-23T00:57:52.6900860Z              for mc in self.rqdata.dataCaches:
2022-06-23T00:57:52.6901154Z     >            self.worker[mc] = self._start_worker(mc)
2022-06-23T00:57:52.6901387Z      
2022-06-23T00:57:52.6902344Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1268, in RunQueue._start_worker(mc='', fakeroot=False, rqexec=None):
2022-06-23T00:57:52.6902932Z              worker.stdin.write(b"<extraconfigdata>" + pickle.dumps(self.cooker.extraconfigdata) + b"</extraconfigdata>")
2022-06-23T00:57:52.6903397Z     >        worker.stdin.write(b"<workerdata>" + pickle.dumps(workerdata) + b"</workerdata>")
2022-06-23T00:57:52.6904226Z              worker.stdin.flush()
2022-06-23T00:57:52.6904497Z BrokenPipeError: [Errno 32] Broken pipe
2022-06-23T00:57:52.6904644Z 
2022-06-23T00:57:52.6905449Z ERROR: Running idle function
2022-06-23T00:57:52.6906161Z Traceback (most recent call last):
2022-06-23T00:57:52.6908003Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/cooker.py", line 1438, in buildTargetsIdle(server=<ProcessServer(ProcessServer-1, started)>, rq=<bb.runqueue.RunQueue object at 0x7fd16f08e4a8>, abort=False):
2022-06-23T00:57:52.6908771Z                  try:
2022-06-23T00:57:52.6910081Z     >                retval = rq.execute_runqueue()
2022-06-23T00:57:52.6910509Z                  except runqueue.TaskFailure as exc:
2022-06-23T00:57:52.6911385Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1524, in RunQueue.execute_runqueue():
2022-06-23T00:57:52.6911836Z              try:
2022-06-23T00:57:52.6912141Z     >            return self._execute_runqueue()
2022-06-23T00:57:52.6912497Z              except bb.runqueue.TaskFailure:
2022-06-23T00:57:52.6913487Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1474, in RunQueue._execute_runqueue():
2022-06-23T00:57:52.6913882Z                  self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6914155Z     >            self.start_worker()
2022-06-23T00:57:52.6914634Z                  self.rqdata.init_progress_reporter.next_stage()
2022-06-23T00:57:52.6915413Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1295, in RunQueue.start_worker():
2022-06-23T00:57:52.6915991Z              for mc in self.rqdata.dataCaches:
2022-06-23T00:57:52.6916309Z     >            self.worker[mc] = self._start_worker(mc)
2022-06-23T00:57:52.6916551Z      
2022-06-23T00:57:52.6917175Z   File "/__w/1/s/msocs-rackmanager-bsp/yocto/bitbake/lib/bb/runqueue.py", line 1268, in RunQueue._start_worker(mc='', fakeroot=False, rqexec=None):
2022-06-23T00:57:52.6917793Z              worker.stdin.write(b"<extraconfigdata>" + pickle.dumps(self.cooker.extraconfigdata) + b"</extraconfigdata>")
2022-06-23T00:57:52.6918746Z     >        worker.stdin.write(b"<workerdata>" + pickle.dumps(workerdata) + b"</workerdata>")
2022-06-23T00:57:52.6919101Z              worker.stdin.flush()
2022-06-23T00:57:52.6919963Z BrokenPipeError: [Errno 32] Broken pipe
2022-06-23T00:57:52.6920269Z 
2022-06-23T00:57:52.8226862Z 
2022-06-23T00:57:52.8228092Z Summary: There were 11 WARNING messages shown.
2022-06-23T00:57:52.8229138Z Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

BSP相关配置

<remote fetch="https://git.yoctoproject.org/git" name="yocto"/>  
<remote fetch="git://git.openembedded.org" name="oe"/>
<remote> fetch="https://git.yoctoproject.org" name="ti"/>

调试思路

该错误核心是BrokenPipeError: [Errno 32] Broken pipe，出现在bitbake启动worker进程、向worker标准输入写入序列化配置数据阶段，本质是bitbake拉起的worker进程提前异常退出，导致管道连接断开，并非序列化逻辑本身故障，可按以下优先级排查：

优先排查环境资源差异：本地Hyper-V虚拟机和Azure Pipeline构建环境的资源配额差异是最高发诱因。Azure Pipeline默认托管代理的内存、CPU、磁盘空间、tmpfs容量通常远低于本地手动构建的虚拟机，Yocto启动worker时如果内存不足会触发系统OOM机制直接杀掉worker进程，直接触发该断管错误。可在Pipeline中构建步骤前增加free -h、df -h、ulimit -a命令输出环境资源配置，和本地正常构建的环境参数做对比，同时检索构建日志中是否存在OOM相关的进程被杀记录。
排查源码拉取协议兼容性问题：当前BSP配置中OpenEmbedded层使用git://协议拉取代码，Azure Pipeline托管构建代理的出站规则默认封禁git协议使用的9418端口，拉取对应代码层时如果出现静默失败、文件残缺，会导致worker加载配置时直接崩溃退出。
排查配置与权限差异：Azure Pipeline每次构建默认在全新工作目录执行，如果本地构建复用了sstate缓存、下载目录缓存但Pipeline中未做对应缓存配置，或者构建目录权限异常（比如执行用户和本地不一致、工作目录被挂载为noexec权限），也会导致worker启动失败。可在Pipeline中执行bitbake -e输出完整环境配置，和本地正常构建的bitbake -e输出做diff，定位差异配置项。
无构建机权限时的日志捕获方法：在Pipeline构建步骤开头增加set -x打印所有执行的命令，同时给bitbake增加-vv参数开启最高等级调试日志，捕获worker启动前的所有输出；如果默认日志未捕获worker的标准错误输出，可将bitbake的所有输出重定向到单独日志文件，构建结束后无论成功失败都将日志文件作为Pipeline产物发布，即可拿到worker退出前打印的完整错误栈。

可落地的解决方案

修正BSP配置问题：将所有git://协议源替换为https://协议源，同时修正第三行remote标签的语法错误，修正后配置如下：

<remote fetch="https://git.yoctoproject.org/git" name="yocto"/>  
<remote fetch="https://git.openembedded.org" name="oe"/>
<remote fetch="https://git.yoctoproject.org" name="ti"/>

调整构建资源配置：如果使用微软托管代理，选择至少8核16G内存规格的代理节点，将构建目录下的tmp、sstate、downloads目录挂载到非tmpfs的持久化磁盘目录，避免tmpfs空间不足导致进程被杀。
限制并发资源占用：在bitbake构建命令中增加-j 4 --set-threads=4参数（数值可根据代理实际规格调整），避免启动过多worker瞬间占用过高内存触发OOM。
增加构建前预检查：在执行bitbake构建前先执行依赖安装命令，确保所有Yocto构建依赖包都已正确安装，和本地构建环境的依赖版本保持一致。

内容的提问来源于stack exchange，提问作者Annie Poh