使用Python subprocess调用mosquitto_sub结合grep过滤内容遇错求助

Let's break down what's going wrong in each approach you tried, then walk through two solid solutions to filter your mosquitto_sub output for lines containing "Online".

Why Your First Attempt Failed #

When you set shell=False (the default safe option), subprocess doesn't use a shell to parse your command. That means the | pipe character gets passed directly as an argument to mosquitto_sub—which has no idea what to do with it, hence the "Unknown option '|'" error. The split() method treats every part of your command as a separate argument to the first program, not as shell syntax.

Why Your Second Attempt (shell=True) Failed #

When you switched to shell=True, you still used split() on your command string, which caused two problems:

1. When shell=True receives a list of arguments, it treats the first element as the command to run in the shell, and the rest as arguments to that shell (not the command you want). So your pipe and grep parts were being passed incorrectly.
2. The # in ITT/# is a comment character in shells. Without quoting the topic, the shell ignores everything after the #, leaving mosquitto_sub without a valid topic parameter—hence the "You must specify a topic" error.

Why Your Third Attempt (Two Popen Calls) Had No Output #

You were close here, but two key issues blocked output:

1. You used iter(p2.stdout.readline, '') as the terminator, but stdout.readline() returns bytes by default (not strings). The iterator never hit the empty string terminator, so it hung waiting for input that never signaled completion.
2. You didn't enable line buffering or close p1.stdout after linking it to p2. Without this, output gets stuck in buffers instead of being passed to grep immediately, and p2 never knows when p1 has finished sending data.

Solution 1: Use shell=True (With Correct Command Formatting) #

If you're comfortable using shell=True (note: only use this with trusted commands to avoid security risks), you can pass the full command as a single string, with proper quoting to protect the topic:

import subprocess

def run_command(command):
    # text=True makes output return as strings instead of bytes
    p = subprocess.Popen(
        command,
        shell=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        text=True,
        bufsize=1  # Enable line buffering for real-time output
    )
    return iter(p.stdout.readline, '')

# Quote the topic to protect the # from being treated as a shell comment
command = 'mosquitto_sub -u example -P example -t "ITT/#" -v | grep "Online"'
for line in run_command(command):
    print(line, end='')  # readline includes a newline, so avoid double-printing

Solution 2: Chain Two subprocess Calls (No Shell Needed) #

For a safer approach (no shell involved), fix the buffer and terminator issues in your original two-Popen setup:

import subprocess

def run_command(sub_command, filter_command):
    # text=True converts output to strings; bufsize=1 enables line buffering
    p1 = subprocess.Popen(
        sub_command,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        text=True,
        bufsize=1
    )
    p2 = subprocess.Popen(
        filter_command,
        stdin=p1.stdout,
        stdout=subprocess.PIPE,
        text=True,
        bufsize=1
    )
    # Close p1's stdout so p2 knows when input ends
    p1.stdout.close()
    # Use empty string as terminator since text=True returns strings
    return iter(p2.stdout.readline, '')

# Pass commands as lists (no shell syntax needed)
sub_command = ['mosquitto_sub', '-u', 'example', '-P', 'example', '-t', 'ITT/#', '-v']
filter_command = ['grep', 'Online']

for line in run_command(sub_command, filter_command):
    print(line, end='')

Both solutions will give you real-time output of lines containing "Online" from your MQTT subscription. The second approach is more secure if you're working with untrusted input, while the first is simpler for quick scripts with trusted commands.

内容的提问来源于stack exchange，提问作者Godhaze