如何通过Python调用w64devkit直接向llama.cpp传递Prompt并获取结果？

问题原因 #

w64devkit.exe是终端模拟器程序，直接调用它会启动独立窗口，导致Python无法捕获其标准输出/错误流。你不需要通过w64devkit.exe来运行llama.cpp的main程序，直接调用编译后的可执行文件即可，或者用w64devkit内置的bash以无窗口模式执行。

解决方案 #

方法一：直接调用llama.cpp的Windows可执行文件（推荐） #

如果已经在w64devkit中编译好llama.cpp，会生成main.exe，直接调用这个可执行文件即可，无需经过w64devkit.exe。

修改后的代码：

import subprocess
import os
import mysql.connector
from dotenv import load_dotenv


def run_llama_cpp(prompt):
    # 直接使用llama.cpp编译后的main.exe路径
    executable_path = 'D:\\Code\\llama_cpp\\llama.cpp\\main.exe'
    current_directory = 'D:\\Code\\llama_cpp\\llama.cpp'
    model_path = 'D:\\Code\\llama_cpp\\llama.cpp\\models\\llama-2-7b.Q8_0.gguf'

    os.chdir(current_directory)

    # 修正命令参数：-p的prompt作为单个元素传入，无需额外引号
    command = [
        executable_path,
        '-ins',
        '--color',
        '-c', '1024',
        '--temp', '0.7',
        '--repeat_penalty', '1.1',
        '-s', '42',
        '-n', '-1',
        '-m', model_path,
        '-p', prompt
    ]

    try:
        print("Command: ", ' '.join(command))
        # 使用subprocess.run简化调用，同时捕获输出
        result = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=True)
        
        if result.stderr:
            print("Standard Error Output:", result.stderr)

        title_start = result.stdout.find('[TITLE_START]') + len('[TITLE_START]')
        title_end = result.stdout.find('[TITLE_END]')
        title = result.stdout[title_start:title_end].strip() if (title_start > len('[TITLE_START]')-1 and title_end != -1) else None
        description = ""

        return title, description
    except subprocess.CalledProcessError as e:
        print(f"Error running llama_cpp: {e.stderr}")
        return None, None
    except FileNotFoundError as e:
        print(f"File not found: {e}")
        return None, None
    except Exception as e:
        print(f"An error occurred: {e}")
        return None, None

load_dotenv()
db_host = os.getenv('DB_HOST')
db_port = int(os.getenv('DB_PORT'))
db_database = os.getenv('DB_DATABASE')
db_username = os.getenv('DB_USERNAME')
db_password = os.getenv('DB_PASSWORD')

connection = mysql.connector.connect(host=db_host, user=db_username, password=db_password, database=db_database, port=db_port)
cursor = connection.cursor(dictionary=True)
cursor.execute("SELECT * FROM posts")
prompts = cursor.fetchall()
cursor.close()
connection.close()

for prompt_entity in prompts:
    content_str = prompt_entity['content']
    # 修正prompt的引号转义，避免语法问题
    content = f"Generate headline for this content: '{content_str}'. Put the headline between [TITLE_START] and [TITLE_END]"

    title, description = run_llama_cpp(content)
    if title:
        print("Content:", content)
        print("Title:", title)
        print("####################################")

方法二：用w64devkit的bash无窗口执行 #

如果必须通过w64devkit的环境运行，调用其内置的bash.exe并设置隐藏窗口参数：

修改run_llama_cpp函数中的命令部分：

def run_llama_cpp(prompt):
    bash_path = 'D:\\Code\\llama_cpp\\w64devkit\\bin\\bash.exe'
    current_directory = 'D:\\Code\\llama_cpp\\llama.cpp'
    model_path = 'D:\\Code\\llama_cpp\\llama.cpp\\models\\llama-2-7b.Q8_0.gguf'

    os.chdir(current_directory)

    # 构造bash执行的命令字符串，转义prompt中的单引号
    escaped_prompt = prompt.replace("'", "'\\''")
    cmd_str = f"./main -ins --color -c 1024 --temp 0.7 --repeat_penalty 1.1 -s 42 -n -1 -m '{model_path}' -p '{escaped_prompt}'"
    
    try:
        print("Command: ", cmd_str)
        # CREATE_NO_WINDOW 隐藏终端窗口
        result = subprocess.run(
            [bash_path, '-c', cmd_str],
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True,
            check=True,
            creationflags=subprocess.CREATE_NO_WINDOW
        )
        
        # 提取标题逻辑
        title_start = result.stdout.find('[TITLE_START]') + len('[TITLE_START]')
        title_end = result.stdout.find('[TITLE_END]')
        title = result.stdout[title_start:title_end].strip() if (title_start > len('[TITLE_START]')-1 and title_end != -1) else None
        description = ""

        return title, description
    except subprocess.CalledProcessError as e:
        print(f"Error running llama_cpp: {e.stderr}")
        return None, None
    except FileNotFoundError as e:
        print(f"File not found: {e}")
        return None, None
    except Exception as e:
        print(f"An error occurred: {e}")
        return None, None

关键修正点 #

• 移除w64devkit.exe的直接调用，改用编译后的main.exe或w64devkit的bash.exe
• 修正-p参数传递：subprocess列表模式下无需手动加引号；bash命令字符串模式下需转义prompt中的单引号
• 添加CREATE_NO_WINDOW标志（仅Windows有效），避免弹出终端窗口
• 移除原代码中无效的description提取逻辑（原prompt未要求生成描述）

内容的提问来源于stack exchange，提问作者Carol.Kar