Webpack Module Federation在GitLab构建时出现JavaScript堆内存溢出问题

构建阶段失败，报错信息如下：

<--- Last few GCs --->
[16:0x7f50fd1533b0]   388576 ms: Mark-sweep 1820.0 (1888.8) -> 1805.4 (1890.6) MB, 5141.7 / 0.0 ms  (average mu = 0.112, current mu = 0.056) allocation failure scavenge might not succeed
[16:0x7f50fd1533b0]   393919 ms: Mark-sweep 1811.2 (1890.6) -> 1806.4 (1891.6) MB, 4570.0 / 0.0 ms  (average mu = 0.128, current mu = 0.145) allocation failure GC in old space requested
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
npm ERR! path /app
npm ERR! command failed
npm ERR! signal SIGABRT
npm ERR! command sh -c rimraf dist && webpack build --config webpack.production.js
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-08-11T21_50_36_294Z-debug-0.log

相关背景 #

• 本地未使用Docker环境下构建成功
• 无Module Federation的同款配置在GitLab上可成功构建
• 本地构建产物大小不超过12MB，应用运行正常

构建正常的Webpack配置（无Module Federation） #

const path = require('path')
const HTMLWebpackPlugin = require('html-webpack-plugin')

require('dotenv').config({ path: './.env' })

module.exports = {
  output: {
    publicPath: '.',
    path: path.resolve(__dirname, 'dist'),
    filename: '[name]-[contenthash].js'
  },
  mode: 'production',
  entry: {
    index: './src/index.tsx'
  },
  performance: {
    hints: false,
    maxEntrypointSize: 512000,
    maxAssetSize: 512000
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        exclude: /node_modules/,
        use: [
          {
            loader: require.resolve('ts-loader')
          }
        ]
      },
      {
        test: /\.scss$/,
        use: [
          {
            loader: 'style-loader'
          },
          {
            loader: 'css-modules-typescript-loader'
          },
          {
            loader: 'css-loader',
            options: {
              modules: true
            }
          },
          {
            loader: 'sass-loader'
          }
        ]
      },
      {
        test: /\.css$/,
        use: [
          {
            loader: 'style-loader'
          },
          {
            loader: 'css-loader'
          }
        ]
      },
      {
        test: /\.(png|jpg|gif|env|glb|gltf|stl)$/i,
        use: [
          {
            loader: 'url-loader',
            options: {
              limit: 8192
            }
          }
        ]
      }
    ]
  },
  resolve: {
    extensions: ['.tsx', '.ts', '.js']
  },
  plugins: [
    new HTMLWebpackPlugin({ template: './public/index.html' }),
    new Webpack.ProvidePlugin({ process: 'process/browser' }),
    new Webpack.DefinePlugin({ 'process.env': JSON.stringify(process.env) })
  ]
}

GitLab构建失败的Module Federation Webpack配置 #

const Webpack = require('webpack')
const path = require('path')
const ModuleFederationPlugin = require('webpack/lib/container/ModuleFederationPlugin')
const HTMLWebpackPlugin = require('html-webpack-plugin')

require('dotenv').config({ path: './.env' })

const deps = require('./package.json').dependencies

module.exports = {
  output: {
    publicPath: 'auto',
    path: path.resolve(__dirname, 'dist'),
    filename: '[name]-[contenthash].js'
  },
  mode: 'production',
  entry: {
    index: './src/index.tsx'
  },
  performance: {
    hints: false,
    maxEntrypointSize: 512000,
    maxAssetSize: 512000
  },
  module: {
    rules: [
      {
        test: /\.tsx?$/,
        exclude: /node_modules/,
        use: [
          {
            loader: require.resolve('ts-loader')
          }
        ]
      },
      {
        test: /\.scss$/,
        use: [
          {
            loader: 'style-loader'
          },
          {
            loader: 'css-modules-typescript-loader'
          },
          {
            loader: 'css-loader',
            options: {
              modules: true
            }
          },
          {
            loader: 'sass-loader'
          }
        ]
      },
      {
        test: /\.css$/,
        use: [
          {
            loader: 'style-loader'
          },
          {
            loader: 'css-loader'
          }
        ]
      },
      {
        test: /\.(png|jpg|gif|env|glb|gltf|stl)$/i,
        use: [
          {
            loader: 'url-loader',
            options: {
              limit: 8192
            }
          }
        ]
      }
    ]
  },
  resolve: {
    extensions: ['.tsx', '.ts', '.js']
  },
  plugins: [
    new ModuleFederationPlugin({
      name: 'game',
      filename: 'game.js',
      exposes: {
        './Game': './src/App.tsx'
      },
      remoteType: 'var',
      shared: {
        // ...deps,
        react: {
          singleton: true,
          eager: true,
          requiredVersion: deps.react
        },
        'react-dom': {
          singleton: true,
          requiredVersion: deps['react-dom']
        }
      }
    }),
    new HTMLWebpackPlugin({ template: './public/index.html' }),
    new Webpack.ProvidePlugin({ process: 'process/browser' }),
    new Webpack.DefinePlugin({ 'process.env': JSON.stringify(process.env) })
  ]
}

问题原因 #

Module Federation会增加Webpack构建时的内存消耗：它需要分析模块依赖、处理共享模块的版本兼容性、生成容器入口等额外逻辑，GitLab Runner默认分配的Node.js堆内存不足以支撑这些额外计算，导致内存溢出。本地环境通常有更大的内存配额，所以能成功构建。

解决方法 #

1. 增加Node.js堆内存限制 #

修改package.json中的构建命令，通过--max-old-space-size参数提升Node.js的堆内存上限，比如设置为4GB：

{
  "scripts": {
    "build": "node --max-old-space-size=4096 node_modules/webpack/bin/webpack.js build --config webpack.production.js"
  }
}

或者直接在GitLab CI配置中修改构建命令：

build:
  script:
    - npm install
    - node --max-old-space-size=4096 node_modules/webpack/bin/webpack.js build --config webpack.production.js

2. 优化Module Federation配置 #

• 关闭eager模式：如果不需要共享模块在入口立即加载，去掉react配置中的eager: true，减少初始构建时的内存占用。
• 精确控制共享模块：保持只共享必要的核心库（你已经注释了...deps，这步已完成）。

3. 优化Webpack构建流程 #

• 替换style-loader为mini-css-extract-plugin：生产环境下style-loader会把CSS注入JS中，增加构建内存占用，mini-css-extract-plugin直接生成独立CSS文件，降低内存压力。
  修改CSS相关loader配置：

  const MiniCssExtractPlugin = require('mini-css-extract-plugin');
  
  // ...
  module: {
    rules: [
      {
        test: /\.scss$/,
        use: [
          MiniCssExtractPlugin.loader, // 替换style-loader
          { loader: 'css-modules-typescript-loader' },
          { loader: 'css-loader', options: { modules: true } },
          { loader: 'sass-loader' }
        ]
      },
      {
        test: /\.css$/,
        use: [
          MiniCssExtractPlugin.loader, // 替换style-loader
          { loader: 'css-loader' }
        ]
      }
    ]
  },
  plugins: [
    new MiniCssExtractPlugin({ filename: '[name]-[contenthash].css' }),
    // ...其他插件
  ]
  

• 升级Webpack版本：新版本Webpack对Module Federation的内存使用有优化，尝试升级到最新稳定版。

4. 调整GitLab Runner资源配置 #

如果GitLab Runner是自托管的，可以修改Runner配置（比如config.toml中的memory参数），给构建容器分配更多内存。

内容的提问来源于stack exchange，提问作者Max Kachanov