Python未用print命令却自动输出category_list2的原因排查

问题描述 #

运行以下Python代码时，未显式调用print命令，但控制台自动输出了categorization_llm函数返回的category_list2，需排查原因。

代码 #

text_list = ["text1", 
             "text2", 
             "text3"]

def loop_categories(lang):
    
    ###This is a function to loop through categories & categories description by chunk of 5 
    ###Every chunk is added with category "Else" as a back up for future matching
    
    import pandas as pd
    
    df = pd.read_csv("category_list.csv", sep = ";")
    df = df[df['lang'] == lang]
    
    #Find the "Else" row (if not found, add it)
    else_row = df[df['Category'] == 'Else'].iloc[0]
    else_string = else_row['Category']
    else_description = f"{else_row['Category']}: {else_row['Description']}"  
    
    #If the category size is below 5, then keep adding them 
    i = 0
    while i < len(df):
        subset = df[i:i+5]
        
        # If the subset is empty, break the loop
        if subset.empty:
            break

        # If "Else" is not in the subset, add it
        if 'Else' not in subset['Category'].values:
            theme_string = ', '.join(subset['Category'].values.tolist() + [else_string])
            theme_descriptions = ', '.join([f"{row['Category']}: {row['Description']}" for idx, row in subset.iterrows()] + [else_description])
        else:
            theme_string = ', '.join(subset['Category'].values.tolist())
            theme_descriptions = ', '.join([f"{row['Category']}: {row['Description']}" for idx, row in subset.iterrows()])
        
        #Return the last 5 categories & corresponding description including the "Else" one
        yield theme_string, theme_descriptions
        i += 5
        if i >= len(df):
            break



def categorization_llm (lang):
    
    from langchain.llms import OpenAI
    from module_categorization import loop_categories
    from module_smalltools import clean_string, category_stripper
    
    def llm_query (prompt):  #To query the LLM model with the prompt
        
        llm = OpenAI(model_name="text-davinci-003",openai_api_key="xxxxx")
        response = llm(prompt)

        return response
    
    
    category_list2 = []
    for texts in text_list:

        category_list = [] #Set a list where all categories attributed will be stored temporarily
        
        ###Go through all chunks of categories & assess to which the piece of text belong
        
        for theme_string, theme_descriptions in loop_categories(lang): #Loop through chunks of 5 categories
            prompt_categorization = f'''
                myprompt
                '''
            
            response = llm_query (prompt_categorization)  #Get the category for each piece of text x 5 cat. chunk 
            response = clean_string(response) #Clean the output from potential unusual formating
            response = category_stripper(response)

            category_list.append(response)
            
        category_list2.append(list(set(category_list)))
                    
    return category_list2    
        

x = categorization_llm ("en")
print("")

排查原因 #

• 交互式环境自动输出：如果在Jupyter Notebook、IPython或Python交互式控制台运行代码，最后一行的赋值语句x = categorization_llm("en")会被解释器自动评估并打印返回值。这是交互式环境的默认特性，目的是方便查看变量内容。
  解决：在赋值语句末尾加;抑制输出，或者将代码保存为.py文件用python命令运行（非交互式环境无此自动打印行为）。

• LangChain LLM的verbose输出：LangChain的OpenAI类若开启verbose=True，会打印模型响应日志。检查llm_query中llm实例的配置，显式设置verbose=False可关闭该输出：

  llm = OpenAI(model_name="text-davinci-003", openai_api_key="xxxxx", verbose=False)
  

• 第三方工具函数隐式打印：检查module_smalltools中的clean_string或category_stripper函数，确认内部是否包含print语句。若有，移除或注释即可停止不必要的输出。

内容的提问来源于stack exchange，提问作者Raphaël Ambit