WSL Ubuntu环境下Ruby sqlite3 1.4.2版本gem原生扩展编译失败问题

极简总结 #

sqlite3在WSL Ubuntu环境下编译原生扩展失败，相同配置的非WSL原生Ubuntu环境可正常编译安装。

问题概述 #

用户在Windows 10专业版（版本号19042.1288）上使用WSL2运行Ubuntu 20.04.1 LTS，已配置可用的rbenv环境，同时安装了Ruby 2.7.4和Ruby 3.0.2两个版本。执行gem原生扩展编译命令gem install sqlite3 -v '1.4.2' --source 'https://rubygems.org/'时输出大量警告和错误，首个核心报错如下：

sqlite3.c:33:37: error: invalid type argument of ‘->’ (have ‘int’)
   33 | #   define RBIGNUM_LEN(x) RBIGNUM(x)->len
      |                                     ^~

已尝试解决方案 #

更新相关依赖包 #

已更新确认相关的依赖包，执行命令如下：

$ sudo apt-get update
$ sudo apt-get install sqlite3 libsqlite3-dev libxslt-dev libxml2-dev

指定安装参数 #

尝试添加--with-sqlite3-include参数执行安装，命令如下：

$ gem install sqlite3 -v '1.4.2' --source 'https://rubygems.org/' -- --with-sqlite3-include=/usr/include

源码编译安装sqlite3 #

通过源码编译安装的sqlite3可正常运行且已正确加入$PATH，但未解决gem原生扩展编译报错问题，安装命令如下：

$ wget https://www.sqlite.org/2021/sqlite-autoconf-3360000.tar.gz
$ tar xvf sqlite-autoconf-3360000.tar.gz
$ cd sqlite-autoconf-3360000
$ ./configure
$ make
$ make install

原生Ubuntu真机测试 #

相同操作在原生Ubuntu设备上可正常执行，由此判断问题与WSL环境相关；新建干净的WSL Ubuntu实例复现了相同问题，排除本地环境配置异常的可能性。

完整报错日志 #

$ gem install sqlite3 -v '1.4.2' --source 'https://rubygems.org/'
Fetching sqlite3-1.4.2.gem
Building native extensions. This could take a while...
ERROR:  Error installing sqlite3:
        ERROR: Failed to build gem native extension.

    current directory: /home/wwboynton/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/sqlite3-1.4.2/ext/sqlite3
/home/wwboynton/.rbenv/versions/3.0.2/bin/ruby -I /home/wwboynton/.rbenv/versions/3.0.2/lib/ruby/3.0.0 -r ./siteconf20211105-14591-ztfaj2.rb extconf.rb
checking for sqlite3.h... yes
checking for pthread_create() in -lpthread... yes
checking for -ldl... yes
checking for sqlite3_libversion_number() in -lsqlite3... yes
checking for rb_proc_arity()... no
checking for rb_integer_pack()... no
checking for sqlite3_initialize()... yes
checking for sqlite3_backup_init()... yes
checking for sqlite3_column_database_name()... no
checking for sqlite3_enable_load_extension()... yes
checking for sqlite3_load_extension()... yes
checking for sqlite3_open_v2()... yes
checking for sqlite3_prepare_v2()... yes
checking for sqlite3_int64 in sqlite3.h... yes
checking for sqlite3_uint64 in sqlite3.h... yes
creating Makefile

current directory: /home/wwboynton/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/sqlite3-1.4.2/ext/sqlite3
make DESTDIR\= clean

current directory: /home/wwboynton/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/sqlite3-1.4.2/ext/sqlite3
make DESTDIR\=
compiling aggregator.c
compiling backup.c
compiling database.c
database.c: In function ‘rb_sqlite3_open_v2’:
database.c:38:9: warning: unused variable ‘flags’ [-Wunused-variable]
   38 |   VALUE flags;
      |         ^~~~~
database.c: In function ‘exec_batch’:
database.c:726:57: warning: passing argument 3 of ‘sqlite3_exec’ from incompatible pointer type [-Wincompatible-pointer-types]
  726 |     status = sqlite3_exec(ctx->db, StringValuePtr(sql), hash_callback_function, callback_ary, &errMsg);
      |                                                         ^~~~~~~~~~~~~~~~~~~~~~
      |                                                         |
      |                                                         int (*)(VALUE,  int,  char **, char **) {aka int (*)(long unsigned int,  int,  char **, char **)}
In file included from ./sqlite3_ruby.h:25,
                 from database.c:1:
/usr/local/include/sqlite3.h:405:9: note: expected ‘int (*)(void *, int,  char **, char **)’ but argument is of type ‘int (*)(VALUE,  int,  char **, char **)’ {aka ‘int (*)(long unsigned int,  int,  char **, char **)’}
  405 |   int (*callback)(void*,int,char**,char**),  /* Callback function */
      |   ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
database.c:726:81: warning: passing argument 4 of ‘sqlite3_exec’ makes pointer from integer without a cast [-Wint-conversion]
  726 |     status = sqlite3_exec(ctx->db, StringValuePtr(sql), hash_callback_function, callback_ary, &errMsg);
      |                                                                                 ^~~~~~~~~~~~
      |                                                                                 |
      |                                                                                 VALUE {aka long unsigned int}
In file included from ./sqlite3_ruby.h:25,
                 from database.c:1:
/usr/local/include/sqlite3.h:406:3: note: expected ‘void *’ but argument is of type ‘VALUE’ {aka ‘long unsigned int’}
  406 |   void *,                                    /* 1st argument to callback */
      |   ^~~~~~
database.c:728:57: warning: passing argument 3 of ‘sqlite3_exec’ from incompatible pointer type [-Wincompatible-pointer-types]
  728 |     status = sqlite3_exec(ctx->db, StringValuePtr(sql), regular_callback_function, callback_ary, &errMsg);
      |                                                         ^~~~~~~~~~~~~~~~~~~~~~~~~
      |                                                         |
      |                                                         int (*)(VALUE,  int,  char **, char **) {aka int (*)(long unsigned int,  int,  char **, char **)}
In file included from ./sqlite3_ruby.h:25,
                 from database.c:1:
/usr/local/include/sqlite3.h:405:9: note: expected ‘int (*)(void *, int,  char **, char **)’ but argument is of type ‘int (*)(VALUE,  int,  char **, char **)’ {aka ‘int (*)(long unsigned int,  int,  char **, char **)’}
  405 |   int (*callback)(void*,int,char**,char**),  /* Callback function */
      |   ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
database.c:728:84: warning: passing argument 4 of ‘sqlite3_exec’ makes pointer from integer without a cast [-Wint-conversion]
  728 |     status = sqlite3_exec(ctx->db, StringValuePtr(sql), regular_callback_function, callback_ary, &errMsg);
      |                                                                                    ^~~~~~~~~~~~
      |                                                                                    |
      |                                                                                    VALUE {aka long unsigned int}
In file included from ./sqlite3_ruby.h:25,
                 from database.c:1:
/usr/local/include/sqlite3.h:406:3: note: expected ‘void *’ but argument is of type ‘VALUE’ {aka ‘long unsigned int’}
  406 |   void *,                                    /* 1st argument to callback */
      |   ^~~~~~
database.c: At top level:
cc1: warning: unrecognized command line option ‘-Wno-self-assign’
cc1: warning: unrecognized command line option ‘-Wno-parentheses-equality’
cc1: warning: unrecognized command line option ‘-Wno-constant-logical-operand’
compiling exception.c
compiling sqlite3.c
sqlite3.c: In function ‘bignum_to_int64’:
sqlite3.c:33:27: warning: implicit declaration of function ‘RBIGNUM’; did you mean ‘T_BIGNUM’? [-Wimplicit-function-declaration]
   33 | #   define RBIGNUM_LEN(x) RBIGNUM(x)->len
      |                           ^~~~~~~
sqlite3.c:35:20: note: in expansion of macro ‘RBIGNUM_LEN’
   35 |   const long len = RBIGNUM_LEN(value);
      |                    ^~~~~~~~~~~
sqlite3.c:33:37: error: invalid type argument of ‘->’ (have ‘int’)
   33 | #   define RBIGNUM_LEN(x) RBIGNUM(x)->len
      |                                     ^~
sqlite3.c:35:20: note: in expansion of macro ‘RBIGNUM_LEN’
   35 |   const long len = RBIGNUM_LEN(value);
      |                    ^~~~~~~~~~~
sqlite3.c:40:19: error: ‘SIZEOF_BDIGITS’ undeclared (first use in this function); did you mean ‘SIZEOF_INT’?
   40 |   if (len > 63 / (SIZEOF_BDIGITS * CHAR_BIT) + 1) return 0;
      |                   ^~~~~~~~~~~~~~
      |                   SIZEOF_INT
sqlite3.c:40:19: note: each undeclared identifier is reported only once for each function it appears in
sqlite3.c:42:11: error: unknown type name ‘BDIGIT’
   42 |     const BDIGIT *digits = RBIGNUM_DIGITS(value);
      |           ^~~~~~
sqlite3.c:42:28: warning: implicit declaration of function ‘RBIGNUM_DIGITS’; did you mean ‘RBIGNUM_SIGN’? [-Wimplicit-function-declaration]
   42 |     const BDIGIT *digits = RBIGNUM_DIGITS(value);
      |                            ^~~~~~~~~~~~~~
      |                            RBIGNUM_SIGN
sqlite3.c:42:28: warning: initialization of ‘const int *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
sqlite3.c:43:5: error: unknown type name ‘BDIGIT’; did you mean ‘ISDIGIT’?
   43 |     BDIGIT blast = digits[len-1];
      |     ^~~~~~
      |     ISDIGIT
sqlite3.c:44:5: error: unknown type name ‘BDIGIT’; did you mean ‘ISDIGIT’?
   44 |     BDIGIT bmax = (BDIGIT)1UL << (63 % (CHAR_BIT * SIZEOF_BDIGITS));
      |     ^~~~~~
      |     ISDIGIT
sqlite3.c:44:20: error: ‘BDIGIT’ undeclared (first use in this function); did you mean ‘ISDIGIT’?
   44 |     BDIGIT bmax = (BDIGIT)1UL << (63 % (CHAR_BIT * SIZEOF_BDIGITS));
      |                    ^~~~~~
      |                    ISDIGIT
sqlite3.c:44:27: error: expected ‘,’ or ‘;’ before numeric constant
   44 |     BDIGIT bmax = (BDIGIT)1UL << (63 % (CHAR_BIT * SIZEOF_BDIGITS));
      |                           ^~~
sqlite3.c: At top level:
cc1: warning: unrecognized command line option ‘-Wno-self-assign’
cc1: warning: unrecognized command line option ‘-Wno-parentheses-equality’
cc1: warning: unrecognized command line option ‘-Wno-constant-logical-operand’
make: *** [Makefile:245: sqlite3.o] Error 1

make failed, exit code 2

Gem files will remain installed in /home/wwboynton/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/sqlite3-1.4.2 for inspection.
Results logged to /home/wwboynton/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/extensions/x86_64-linux/3.0.0/sqlite3-1.4.2/gem_make.out

问题根源 #

本次报错的核心原因是sqlite3 1.4.2版本与Ruby 3.0+的内部大数处理API不兼容，WSL环境下的头文件加载逻辑差异触发了该问题，原生Ubuntu环境因为系统补丁、头文件优先级等差异未复现报错。

解决方法 #

• 方案一（推荐）：升级sqlite3 gem到1.4.3及以上版本，该版本已修复Ruby 3.0兼容性问题：

  gem install sqlite3 -v '>=1.4.3' --source 'https://rubygems.org/'
  

  若为项目依赖，直接修改Gemfile中的sqlite3版本约束后执行bundle update sqlite3即可。
• 方案二：若必须使用1.4.2版本，切换到Ruby 2.7版本完成安装：

  rbenv local 2.7.4
  gem install sqlite3 -v '1.4.2' --source 'https://rubygems.org/'
  

内容的提问来源于stack exchange，提问作者Wesley Boynton