使用Snowpark框架在Snowflake中调用NLTK包报错求助

问题 #

在Snowflake中使用Snowpark框架结合Python+NLTK做文本挖掘时，运行报错：

Traceback (most recent call last):
  File "nltk/corpus/util.py", line 84, in __load
    root = nltk.data.find(f"{self.subdir}/{zip_name}")
  File "nltk/data.py", line 583, in find
    raise LookupError(resource_not_found)
LookupError: 
**********************************************************************
  Resource [93mwordnet[0m not found.
  Please use the NLTK Downloader to obtain the resource:

  [31m>>> import nltk
  >>> nltk.download('wordnet')
  [0m
  For more information see: https://www.nltk.org/data.html

  Attempted to load [93mcorpora/wordnet.zip/wordnet/[0m

  Searched in:
    - '/home/udf/nltk_data'
    - '/usr/lib/python_udf/60bdac8bd3ab49160e68d1c230dbee4e9fe700de7ef0f3c065830b86356ce449/nltk_data'
    - '/usr/lib/python_udf/60bdac8bd3ab49160e68d1c230dbee4e9fe700de7ef0f3c065830b86356ce449/share/nltk_data'
    - '/usr/lib/python_udf/60bdac8bd3ab49160e68d1c230dbee4e9fe700de7ef0f3c065830b86356ce449/lib/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
**********************************************************************


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  Worksheet, line 19, in main
  Worksheet, line 68, in tokenize
  Worksheet, line 69, in <listcomp>
  File "nltk/stem/wordnet.py", line 45, in lemmatize
    lemmas = wn._morphy(word, pos)
  File "nltk/corpus/util.py", line 121, in __getattr__
    self.__load()
  File "nltk/corpus/util.py", line 86, in __load
    raise e
  File "nltk/corpus/util.py", line 81, in __load
    root = nltk.data.find(f"{self.subdir}/{self.__name}")
  File "nltk/data.py", line 583, in find
    raise LookupError(resource_not_found)
LookupError: 
**********************************************************************

但代码中已经包含了NLTK的stopwords下载：

import nltk
nltk.download('stopwords')
from nltk.corpus import stopwords

请问问题出在哪里？

解答 #

• 问题核心：报错明确提示缺少wordnet资源，你当前只下载了stopwords，但代码中实际用到了WordNet相关功能（比如WordNetLemmatizer），所以必须下载对应的资源。
• 解决步骤：
  1. 在代码中添加wordnet的下载命令：

     nltk.download('wordnet')
     

  2. 考虑到Snowpark运行环境的特殊性，直接用nltk.download()可能存在资源无法持久化或路径问题，更稳妥的方式是：
     • 提前下载好wordnet和stopwords的数据包
     • 将数据包上传到Snowflake的Stage中
     • 在UDF或存储过程中指定NLTK数据路径指向Stage位置，示例代码：

       import nltk
       nltk.data.path.append('/path/to/your/stage/nltk_data')
       

• 补充说明：如果代码中用到词性标注等其他NLTK功能，还要对应下载所需的资源包，比如averaged_perceptron_tagger。

内容的提问来源于stack exchange，提问作者Shen Henry