Nginx反向代理Mosquitto配置问题求助

我参考了Stack Overflow上的MQTT反向代理配置帖子，在虚拟机中用Nginx监听1885端口，希望实现消息在多个Mosquitto容器间分发，构建MQTT集群。

初始配置及问题 #

docker-compose.yml #

version: '3'
services:
  nginx:
    image: nginx:latest
    ports:
      - "1885:1885"
      - "443:443"
    volumes:
      - ./nginx.conf:/var/lib/docker/volumes/nginx-conf/_data/nginx.conf
    networks:
      - backend

  mosquitto1:
    image: eclipse-mosquitto
    ports:
      - "1886:1886"
    environment:
      - listener=1886:1886
      - allow_anonymous=true
    networks:
      - backend
    volumes:
      - ./mosquitto.conf:/etc/mosquitto/mosquitto.conf

networks:
  backend:
    driver: bridge

nginx.conf #

events {
}

http {
    upstream mosquitto_backend {
        
    zone tcp_mem 64k;
    proxy_protocol on;
    }

    server {
        listen 1885;
        server_name localhost;
    log_format mqtt '$remote_addr [$time_local] $protocol $status $bytes_sent $bytes_received $session_time "$request" "$http_referer" "$http_user_agent" "$upstream_addr"';
    access_log /var/log/nginx/mqtt_access.log mqtt;
    error_log  /var/log/nginx/mqtt_error.log; # Health check notifications
        location / {
            proxy_pass http://mosquitto_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}

初始问题 #

• 能订阅1885端口，但无法从其他终端发布消息，也无法用MQTT Explorer连接
• 启动docker-compose时，Mosquitto容器提示1883端口地址不可用，但宿主机通过lsof/netstat检查该端口未被占用

更新后配置及当前问题 #

docker-compose.yml #

version: '3'
services:
  nginx:
    image: nginx:latest
    container_name: nginx
    ports:
      - "1885:1885"
      - "443:443"
    volumes:
      - /var/lib/docker/volumes/nginx-conf/_data:/etc/nginx
    depends_on:
      - mosquitto1
      - mosquitto2
    networks:
      - backend

  mosquitto1:
    image: eclipse-mosquitto
    container_name: mosquitto1
    ports:
      - "1886:1883"
    networks:
      - backend
    volumes:
      - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/config:/mosquitto/config    
  mosquitto2:
    image: eclipse-mosquitto
    container_name: mosquitto2
    ports:
      - "1887:1883"
    networks:
      - backend
    volumes:
      - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/config:/mosquitto/config

networks:
  backend:
    driver: bridge

nginx.conf #

events {
  worker_connections  4096;  ## Default: 1024
}


stream {

        log_format mqtt '$remote_addr [$time_local] $protocol $status $bytes_received ' 
                '$bytes_sent $upstream_addr';
    
    upstream mqtt_cluster {
        server 192.168.210.36:1886; #mosquitto1 | 192.168.210.35
        server 192.168.210.36:1887; #mosquitto2 | 192.168.210.39
    }


    server {
        listen 1885;
        proxy_pass mqtt_cluster;
       

        access_log /var/log/nginx/mqtt_access.log mqtt;
        
    }
}   

mosquitto.conf #

# =================================================================
# Default listener
# =================================================================

# Port to use for the default listener.
listener 1883
allow_anonymous true
persistence true
persistence_location /mosquitto/data/
log_dest file /mosquitto/log/mosquitto.log

当前问题 #

所有容器正常启动，Nginx负载均衡能将消息分发到1886和1887端口的容器，但订阅1885端口的客户端只能收到半数消息——仅保留单个Mosquitto容器时，所有消息能正常接收，怀疑是配置导致两个容器存在覆盖冲突。

核心问题排查与修复 #

1. docker-compose.yml的共享卷冲突（直接导致消息丢失） #

两个Mosquitto容器挂载了同一宿主机目录到容器内的/mosquitto/config，同时默认的/mosquitto/data和/mosquitto/log也会因为基础镜像的配置，最终共享宿主机的同一目录：

• 当两个容器同时运行时，它们的配置文件、持久化数据、日志会互相覆盖，导致只有一个容器能正常处理消息，客户端自然只能收到半数消息。

修复方案：为每个Mosquitto容器分配独立的挂载目录：

mosquitto1:
  image: eclipse-mosquitto
  container_name: mosquitto1
  ports:
    - "1886:1883"
  networks:
    - backend
  volumes:
    - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/config1:/mosquitto/config
    - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/data1:/mosquitto/data
    - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/log1:/mosquitto/log    
mosquitto2:
  image: eclipse-mosquitto
  container_name: mosquitto2
  ports:
    - "1887:1883"
  networks:
    - backend
  volumes:
    - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/config2:/mosquitto/config
    - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/data2:/mosquitto/data
    - /var/lib/docker/volumes/docker-compose-conf/_data/mosquitto/log2:/mosquitto/log

确保每个configX目录下都有独立的mosquitto.conf（内容可一致，但路径必须分开）。

2. Nginx负载均衡策略优化 #

当前Nginx使用宿主机IP+端口的方式连接Mosquitto容器，存在两个问题：

• 依赖宿主机端口映射，增加网络转发损耗
• 默认轮询策略会导致同一客户端的订阅、发布请求被分到不同节点，而独立Mosquitto节点间没有消息同步，客户端收不到跨节点的消息

修复方案：

• 使用Docker服务名直接通信（桥接网络内可直接通过服务名访问）
• 添加ip_hash策略，保证同一客户端始终连接到同一节点：

upstream mqtt_cluster {
    server mosquitto1:1883;
    server mosquitto2:1883;
    ip_hash;
}

3. 补充Mosquitto节点消息同步（集群核心） #

仅靠Nginx负载均衡只是实现了流量分发，两个Mosquitto节点仍是独立个体，跨节点的消息无法互通。需要添加桥接配置实现消息同步：

在mosquitto1的mosquitto.conf中添加：

connection mosquitto2
address mosquitto2:1883
topic # both 0

在mosquitto2的mosquitto.conf中添加：

connection mosquitto1
address mosquitto1:1883
topic # both 0

该配置会让两个节点同步所有主题的消息，确保客户端无论连接到哪个节点，都能收到所有发布的消息。

内容的提问来源于stack exchange，提问作者adrian