使用uproot3生成的ROOT文件无法正常读取的技术问询

我使用uproot3（3.14.4）编写脚本读取ROOT文件并生成精简版本，用于创建分析所需直方图，但部分生成文件无法正常读取。错误随机出现，处理子集文件时脚本正常，全量处理后问题出现，甚至包含之前正常的文件。经验证原始ROOT文件无异常，推测部分变量未正确填充，导致调用pandas.df()或array()方法时触发错误。进一步检查发现，有问题的变量其basket_compressedbytes()和basket_uncompressedbytes()值为0，正常变量无此情况，说明写入tree时存在异常。

报错回溯 #

Traceback (most recent call last):
  File "Create_histos.py", line 1690, in <module>
    Frame = calculate_yields(process,name,region,Type,paths)
  File "Create_histos.py", line 1486, in calculate_yields
    FF=reading_BIG_and_CUTS(ff,region)
  File "Create_histos.py", line 42, in reading_BIG_and_CUTS
    foo=FF.pandas.df(entrystart=i_min,entrystop=i_max,flatten = False)
  File "/afs/cern.ch/user/a/atsiamis/test2/venv_3_8/lib64/python3.8/site-packages/uproot3/_connect/_pandas.py", line 32, in df
    return self._tree.arrays(branches=branches, outputtype=pandas.DataFrame, namedecode=namedecode, entrystart=entrystart, entrystop=entrystop, flatten=flatten, flatname=flatname, awkwardlib=awkwardlib, cache=cache, basketcache=basketcache, keycache=keycache, executor=executor, blocking=blocking)
  File "/afs/cern.ch/user/a/atsiamis/test2/venv_3_8/lib64/python3.8/site-packages/uproot3/tree.py", line 563, in arrays
    futures = [(branch.name if namedecode is None else branch.name.decode(namedecode), interpretation, branch.array(interpretation=interpretation, entrystart=entrystart, entrystop=entrystop, flatten=(flatten and not ispandas), awkwardlib=awkward0, cache=cache, basketcache=basketcache, keycache=keycache, executor=executor, blocking=False)) for branch, interpretation in branches]
  File "/afs/cern.ch/user/a/atsiamis/test2/venv_3_8/lib64/python3.8/site-packages/uproot3/tree.py", line 563, in <listcomp>
    futures = [(branch.name if namedecode is None else branch.name.decode(namedecode), interpretation, branch.array(interpretation=interpretation, entrystart=entrystart, entrystop=entrystop, flatten=(flatten and not ispandas), awkwardlib=awkward0, cache=cache, basketcache=basketcache, keycache=keycache, executor=executor, blocking=False)) for branch, interpretation in branches]
  File "/afs/cern.ch/user/a/atsiamis/test2/venv_3_8/lib64/python3.8/site-packages/uproot3/tree.py", line 1475, in array
    _delayedraise(fill(j))
  File "/afs/cern.ch/user/a/atsiamis/test2/venv_3_8/lib64/python3.8/site-packages/uproot3/tree.py", line 59, in _delayedraise
    raise err.with_traceback(trc)
  File "/afs/cern.ch/user/a/atsiamis/test2/venv_3_8/lib64/python3.8/site-packages/uproot3/tree.py", line 1443, in fill
    source = self._basket(i, interpretation, local_entrystart, local_entrystop, awkward0, basketcache, keycache)
  File "/afs/cern.ch/user/a/atsiamis/test2/venv_3_8/lib64/python3.8/site-packages/uproot3/tree.py", line 1247, in _basket
    byteoffsets = awkward0.numpy.empty((key._fObjlen - key.border - 4) // 4, dtype=awkward0.numpy.int32)  # native endian
ValueError: negative dimensions are not allowed

生成ROOT文件的代码示例 #

with uproot.recreate(main_path+REGION+tag+'/'+'*'+DIR+'*'+'_'+FILE+'.root') as f:
    tree = uproot.newtree({br:'float64' for br in cols})
    f["tree"] = tree
    f['tree'].extend({br:Data[br] for br in cols})

问题分析与解决建议 #

1. uproot3 recreate的已知bug #

uproot3的recreate模块确实存在历史稳定性问题，尤其是处理大文件或批量数据时，容易出现basket写入不完整的情况。报错中的negative dimensions are not allowed以及basket字节数为0的现象，本质是写入时basket元数据（如_fObjlen和border）计算错误导致的，属于uproot3的已知缺陷。

2. 压缩方案的影响 #

未指定压缩方案可能是诱因之一，但不是根本原因。uproot3默认使用zlib压缩，若数据列全为空值或数据量极小，可能触发压缩逻辑中的异常，导致basket字节数为0。可以尝试显式关闭压缩验证：

with uproot.recreate(output_path, compression=None) as f:
    tree = uproot.newtree({br:'float64' for br in cols})
    f["tree"] = tree
    f['tree'].extend({br:Data[br] for br in cols})

3. 临时修复与长期解决方案 #

• 临时修复：提前检查并过滤Data dataframe中的全空列，这类列写入时极易触发异常。
• 长期方案：升级到uproot4（当前稳定版本），uproot4重写了文件写入逻辑，彻底解决了uproot3中存在的basket写入问题，兼容性和稳定性大幅提升。

内容的提问来源于stack exchange，提问作者maevius3