使用代理时open-graph-scraper(NodeJs)在Docker容器中无法返回响应体的求助

Let's break down why your open-graph-scraper works locally but returns only basic fields when running in a Docker container with PM2, and fix it step by step.

1. Fix Your Dockerfile (Dual FROM Issue) #

Your current Dockerfile uses two FROM commands, which means the second keymetrics/pm2:latest-alpine image completely replaces the first node:14 image. This can lead to missing dependencies or Node.js version mismatches that break the scraper.

Replace your Dockerfile with this corrected version, which builds on Node.js 14 Alpine and installs PM2 explicitly:

FROM node:14-alpine

# Install PM2 globally
RUN npm install pm2 -g

# Install CA certificates (critical for HTTPS requests in Alpine)
RUN apk add --no-cache ca-certificates

WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD [ "pm2-runtime", "start", "server.js" ]

2. Fix Proxy Access from Inside the Container #

If your proxy runs on your host machine (e.g., localhost:3128), the container's localhost refers to itself—not your host. This means the scraper can't reach the proxy.

Solution A: Use Host Gateway (Docker 20.10+) #

Run your container with --add-host to map a hostname to your host's IP:

docker run -it --rm -p 3000:3000 --add-host=host.docker.internal:host-gateway <container-name>

Then update your scraper's proxy config to use http://host.docker.internal:3128 instead of localhost:3128.

Solution B: Use Host Network Mode #

If you don't need port mapping isolation, run the container in host network mode (shares your host's network stack):

docker run -it --rm --network host <container-name>

This lets the container use your host's localhost directly, so your original proxy config will work.

3. Ensure Proxy Environment Variables Are Passed #

If your scraper relies on HTTP_PROXY/HTTPS_PROXY environment variables, make sure they're passed to the container:

docker run -it --rm -p 3000:3000 \
  -e HTTP_PROXY=http://your-proxy-address:port \
  -e HTTPS_PROXY=http://your-proxy-address:port \
  <container-name>

Or if you use a .env file, use --env-file:

docker run -it --rm -p 3000:3000 --env-file .env <container-name>

4. Debug Network Connectivity Inside the Container #

To verify if the issue is network-related, run a shell inside the container and test the proxy/URL directly:

docker exec -it <container-id> sh

Then run:

# Test if you can reach the proxy
curl -I http://your-proxy-address:port

# Test if you can fetch the target URL via proxy
curl -x http://your-proxy-address:port https://www.npmjs.com/package/open-graph-scraper

If curl returns the full HTML of the npm page, the network is working—check your scraper's configuration. If not, fix the proxy/network access first.

5. Verify PM2 Is Inheriting Environment Variables #

PM2 Runtime should inherit container environment variables by default, but double-check your scraper code:

• Make sure you're not hardcoding proxy values that only work locally.
• If using a .env file, confirm it's copied into the container (your current Dockerfile does COPY . ., so this should be okay, but verify).

内容的提问来源于stack exchange，提问作者Susampath