OpenWRT设备上Mosquitto高消息量时脚本失效问题求助

Let's break down what's likely causing your script to fail under high message volume and walk through actionable fixes:

Root Cause Analysis #

The core issue is likely synchronous processing blocking your MQTT client:

• When your script processes each message one-by-one (waiting for curl and mosquitto_pub to finish), the pipe between mosquitto_sub and your script fills up.
• mosquitto_sub gets stuck waiting for the pipe to clear, which prevents it from sending keepalive packets to the MQTT server. The server eventually drops the connection, closing the pipe and causing your loop to exit.

Additionally, your background execution attempt had a subtle flaw: background processes inherited the pipe's stdin, leading to race conditions where messages were consumed by background processes instead of the main loop.

Fix 1: Proper Background Processing (No Queue Needed) #

This modification ensures your main loop reads messages quickly (so mosquitto_sub never blocks) and processes them in the background without stdin conflicts:

#!/bin/sh
TOPIC="${1}"
PRINTER="${2}"
USERNAME="${3}"
PASSWORD="${4}"
# Replace with your actual MQTT server host
MQTT_HOST="your_mqtt_server_address"

PrintOne(){
    local MSG="${1}"
    echo "[$(date)] Processing message: ${MSG}" >> /root/log
    # Execute print request
    RES=$(curl -m 2 \
        --header "Content-Type: text/xml;charset=UTF-8" \
        --header "SOAPAction: ''" \
        --header "If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT" \
        --data "${MSG}" \
        "http://${PRINTER}/cgi-bin/epos/service.cgi?devid=local_printer&timeout=5000")
    # Send response back to MQTT
    mosquitto_pub -h "${MQTT_HOST}" -p 8883 -t "${TOPIC}-response" -m "${RES}" -u "${USERNAME}" -P "${PASSWORD}"
    echo "[$(date)] Print result: ${RES}" >> /root/log
}

# Read entire lines (including spaces) without escaping backslashes
while read -r msg ; do
    # Run processing in background, redirect stdin to avoid pipe conflicts
    PrintOne "$msg" </dev/null &
done

Key improvements:

• read -r ensures full lines (including spaces) are captured correctly
• </dev/null prevents background processes from accessing the MQTT pipe
• Added timestamps to logs for easier debugging
• Used local to avoid variable scope issues in the function

Fix 2: Decouple Reception and Processing with a FIFO Queue #

For extremely high message volumes, use a named pipe (FIFO) to completely separate message reception from processing. This guarantees mosquitto_sub never blocks, even if printing is slow:

#!/bin/sh
TOPIC="${1}"
PRINTER="${2}"
USERNAME="${3}"
PASSWORD="${4}"
MQTT_HOST="your_mqtt_server_address"

# Create temporary FIFO queue
FIFO="/tmp/printer_queue.fifo"
mkfifo "${FIFO}" || { echo "Failed to create FIFO queue" >&2; exit 1; }

# Background process to handle printing from the queue
process_queue() {
    while read -r MSG; do
        echo "[$(date)] Processing queued message: ${MSG}" >> /root/log
        RES=$(curl -m 2 \
            --header "Content-Type: text/xml;charset=UTF-8" \
            --header "SOAPAction: ''" \
            --header "If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT" \
            --data "${MSG}" \
            "http://${PRINTER}/cgi-bin/epos/service.cgi?devid=local_printer&timeout=5000")
        mosquitto_pub -h "${MQTT_HOST}" -p 8883 -t "${TOPIC}-response" -m "${RES}" -u "${USERNAME}" -P "${PASSWORD}"
        echo "[$(date)] Queued print result: ${RES}" >> /root/log
    done < "${FIFO}"
    rm "${FIFO}" # Clean up queue when done
}

# Start queue processor in background
process_queue &
PROCESSOR_PID=$!

# Clean up on script exit
trap 'kill ${PROCESSOR_PID}; rm -f ${FIFO}' EXIT INT TERM

# Main loop: read MQTT messages and write to queue
while read -r msg ; do
    echo "$msg" > "${FIFO}" & # Write to queue in background
done

# Wait for all background processes to finish (optional)
wait

Fix 3: Parallel Processing with Xargs #

If you need to control the number of concurrent print jobs (to avoid overwhelming your printer), use xargs to manage parallelism:

First, create a standalone processing script printer_single.sh:

#!/bin/sh
TOPIC="${1}"
PRINTER="${2}"
USERNAME="${3}"
PASSWORD="${4}"
MSG="${5}"
MQTT_HOST="your_mqtt_server_address"

echo "[$(date)] Processing message: ${MSG}" >> /root/log
RES=$(curl -m 2 \
    --header "Content-Type: text/xml;charset=UTF-8" \
    --header "SOAPAction: ''" \
    --header "If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT" \
    --data "${MSG}" \
    "http://${PRINTER}/cgi-bin/epos/service.cgi?devid=local_printer&timeout=5000")
mosquitto_pub -h "${MQTT_HOST}" -p 8883 -t "${TOPIC}-response" -m "${RES}" -u "${USERNAME}" -P "${PASSWORD}"
echo "[$(date)] Print result: ${RES}" >> /root/log

Then update your mosquitto_sub command:

mosquitto_sub -h "${HOST}" -k 30 -c -p 8883 -t "${TOPIC}" -u "${USERNAME}" -P "${PASSWORD}" --id "${ID}" | xargs -d '\n' -I {} -P 3 /bin/sh /bin/printer_single.sh "${TOPIC}" "${PRINTER}" "${USERNAME}" "${PASSWORD}" "{}"

• -P 3: Limits to 3 concurrent print jobs (adjust based on your printer's capacity)
• -d '\n': Splits input by newlines (assumes each MQTT message is a single line)

Additional Debugging Tips #

1. Enable MQTT logging: Add -v to your mosquitto_sub command to log all received messages, confirming whether the client is disconnecting under load.
2. Check printer limits: Verify your printer can handle concurrent HTTP requests—some low-end printers may crash if overwhelmed.
3. Monitor system resources: Use top or htop on OpenWRT to check for high CPU/memory usage that could be killing your script.

内容的提问来源于stack exchange，提问作者carmo