如何禁用StanfordTokenizer的DeprecationWarning？已尝试相关链接无效

Hey there! I get it, that persistent deprecation warning can be super annoying when you're trying to run your code smoothly. Let's walk through a few solid ways to silence it, plus a quick note on the long-term fix just in case you're interested down the line.

Method 1: Filter the Exact Warning Globally #

If you want to mute only this specific warning (and not all deprecation alerts across your code), use Python's warnings module to target it precisely. This way you won't miss other important deprecation messages elsewhere.

Add this code before you initialize your tokenizer:

import warnings

# Match the exact warning message to avoid ignoring unrelated deprecations
warnings.filterwarnings(
    "ignore",
    category=DeprecationWarning,
    message=r"The StanfordTokenizer will be deprecated in version 3\.2\.5\. Please use nltk\.parse\.corenlp\.CoreNLPTokenizer instead\."
)

# Initialize your tokenizer as usual
self.tokenizer = StanfordTokenizer(jar_path)

The backslashes in the message are key—they escape the dots, which have special meaning in regular expressions, ensuring we match the warning text perfectly.

Method 2: Temporary Warning Suppression #

If you'd rather not disable the warning globally, you can suppress it only during the tokenizer setup using a context manager. This keeps all other warnings visible:

import warnings

# Only ignore deprecation warnings inside this block
with warnings.catch_warnings():
    warnings.filterwarnings("ignore", category=DeprecationWarning)
    self.tokenizer = StanfordTokenizer(jar_path)

Bonus: Long-Term Fix (Switch to the Recommended Tokenizer) #

While disabling the warning works for now, the real solution is to transition to CoreNLPTokenizer since StanfordTokenizer will be removed in future NLTK versions. Here's a quick setup example:

from nltk.parse.corenlp import CoreNLPTokenizer

# Option 1: Use a local CoreNLP server (default port 9000)
self.tokenizer = CoreNLPTokenizer(url='http://localhost:9000')

# Option 2: Use local CoreNLP jars if you have them downloaded
# self.tokenizer = CoreNLPTokenizer(
#     path_to_jar='/path/to/stanford-corenlp.jar',
#     path_to_models_jar='/path/to/stanford-corenlp-models.jar'
# )

If your previous attempts to disable the warning failed, double-check that you're targeting the correct DeprecationWarning category and that your message match is exact (including punctuation and version numbers). The regex approach in Method 1 should eliminate any mismatches that might have caused issues before.

内容的提问来源于stack exchange，提问作者Abdul Rahman