Python中TfidfVectorizer拟合数据遇%time语法错误如何解决？

Hey there! That syntax error is happening because %time isn’t standard Python syntax—it’s an IPython magic command, meaning it only works in IPython environments like Jupyter Notebooks or the IPython shell, not in regular Python scripts run with python your_script.py.

Here are two easy fixes depending on your use case: #

1. Use Standard Python's time Module (Best for Regular Scripts)

Replace the %time line with standard timing code using the built-in time module. This works everywhere, not just in IPython:

import time
from sklearn.feature_extraction.text import TfidfVectorizer

# Make sure your tokenize_and_stem function and synopses data are defined first!
def tokenize_and_stem(text):
    # Add your tokenization/stemming logic here
    pass

synopses = ["Sample synopsis text", "Another example synopsis"]

# Define vectorizer parameters
tfidf_vectorizer = TfidfVectorizer(
    max_df=0.8, 
    max_features=200000, 
    min_df=0.2, 
    stop_words='english', 
    use_idf=True, 
    tokenizer=tokenize_and_stem, 
    ngram_range=(1,3)
)

# Timing with standard Python
start_time = time.time()
tfidf_matrix = tfidf_vectorizer.fit_transform(synopses)
end_time = time.time()

print(f"Processing took {end_time - start_time:.4f} seconds")
print(tfidf_matrix.shape)

2. If You're Using IPython/Jupyter

If you’re running this code in a Jupyter Notebook or IPython shell and still getting an error, ensure the %time command is on its own line (not mixed with other code) and that you’re in a valid IPython environment. For example:

from sklearn.feature_extraction.text import TfidfVectorizer

# Define vectorizer and data as before
tfidf_vectorizer = TfidfVectorizer(...)

# %time needs to be on its own line in IPython
%time tfidf_matrix = tfidf_vectorizer.fit_transform(synopses)

print(tfidf_matrix.shape)

The key takeaway: %time is a convenience tool for IPython workflows, but standard Python requires using the time module for timing operations.

内容的提问来源于stack exchange，提问作者fosho