安装Torch执行install.sh时遇luajit内存不足错误如何解决？

问题描述 #

我正按照Torch官方指南安装Torch，执行以下命令：

git clone https://github.com/torch/distro.git ~/torch --recursive
cd ~/torch; bash install-deps;
./install.sh

执行./install.sh时出现内存不足错误，第一次报错内容：

...
-- Generating done
-- Build files have been written to: /home/myUser/torch/pkg/torch/build
[  5%] Building C object lib/luaT/CMakeFiles/luaT.dir/luaT.c.o
[  5%] Building C object lib/TH/CMakeFiles/TH.dir/THHalf.c.o
[  7%] Building C object lib/TH/CMakeFiles/TH.dir/THGeneral.c.o
[ 10%] Building C object lib/TH/CMakeFiles/TH.dir/THFile.c.o
[ 12%] Building C object lib/TH/CMakeFiles/TH.dir/THLapack.c.o
[ 15%] Building C object lib/TH/CMakeFiles/TH.dir/THAllocator.c.o
[ 17%] Building C object lib/TH/CMakeFiles/TH.dir/THSize.c.o
[ 20%] Building C object lib/TH/CMakeFiles/TH.dir/THStorage.c.o
[ 23%] Building C object lib/TH/CMakeFiles/TH.dir/THRandom.c.o
[ 28%] Building C object lib/TH/CMakeFiles/TH.dir/THMemoryFile.c.o
[ 25%] Building C object lib/TH/CMakeFiles/TH.dir/THTensor.c.o
[ 30%] Building C object lib/TH/CMakeFiles/TH.dir/THBlas.c.o
[ 33%] Building C object lib/TH/CMakeFiles/TH.dir/THLogAdd.c.o
[ 35%] Building C object lib/TH/CMakeFiles/TH.dir/THDiskFile.c.o
[ 38%] Building C object lib/TH/CMakeFiles/TH.dir/THAtomic.c.o
[ 41%] Building C object lib/TH/CMakeFiles/TH.dir/THVector.c.o
[ 43%] Building C object lib/TH/CMakeFiles/TH.dir/generic/simd/convolve.c.o
[ 46%] Building C object lib/TH/CMakeFiles/TH.dir/generic/simd/convolve5x5_sse.c.o
[ 48%] Building C object lib/TH/CMakeFiles/TH.dir/vector/AVX2.c.o
[ 51%] Building C object lib/TH/CMakeFiles/TH.dir/generic/simd/convolve5x5_avx.c.o
[ 53%] Building C object lib/TH/CMakeFiles/TH.dir/vector/AVX.c.o
[ 56%] Linking C shared library libluaT.so
[ 56%] Built target luaT
[ 58%] Linking C shared library libTH.so
[ 58%] Built target TH
[ 64%] Generating random.c
[ 64%] Generating TensorMath.c
/home/myUser/torch/install/bin/luajit: not enough memory
make[2]: *** [CMakeFiles/torch.dir/build.make:74: TensorMath.c] Error 1
make[2]: *** Deleting file 'TensorMath.c'
make[1]: *** [CMakeFiles/Makefile2:133: CMakeFiles/torch.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

再次运行时的报错：

-- Generating done
-- Build files have been written to: /home/myUser/torch/pkg/torch/build
Consolidate compiler generated dependencies of target luaT
Consolidate compiler generated dependencies of target TH
[  5%] Built target luaT
[ 58%] Built target TH
[ 61%] Generating TensorMath.c
PANIC: unprotected error in call to Lua API (not enough memory)
make[2]: *** [CMakeFiles/torch.dir/build.make:74: TensorMath.c] Error 1
make[1]: *** [CMakeFiles/Makefile2:133: CMakeFiles/torch.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

Error: Build error: Failed building.

解决方法 #

这个错误源于LuaJIT默认内存限制较低，生成TensorMath.c这类大文件时内存不足，可通过以下几种方式解决：

方法1：临时提升LuaJIT内存限制 #

在执行安装脚本前，设置环境变量增大LuaJIT的内存上限（示例设置为4GB）：

export LUAJIT_MAX_MEM=4000000000

随后运行安装脚本：

./install.sh

方法2：修改Torch构建配置 #

若临时环境变量不生效，可直接修改Torch的CMake配置：

1. 进入Torch的build目录：

cd ~/torch/pkg/torch/build

2. 编辑CMakeCache.txt文件，找到CMAKE_C_FLAGS行，添加-DLUAJIT_MAX_MEM=4000000000参数，修改后类似：

CMAKE_C_FLAGS:STRING=... -DLUAJIT_MAX_MEM=4000000000

3. 保存文件后回到Torch根目录，重新运行安装脚本：

cd ~/torch
./install.sh

方法3：增加系统可用内存 #

如果是虚拟机或云服务器环境，可临时添加swap分区补充内存：

1. 创建并启用swap文件：

sudo fallocate -l 4G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

2. 安装完成后，若不需要swap可关闭并删除：

sudo swapoff /swapfile
sudo rm /swapfile

内容的提问来源于stack exchange，提问作者Christian O.