DBR 13.3 Standard中导入Mosaic库触发DEPRECATION ERROR求助

问题原因 #

你安装的Mosaic 0.3系列版本与DBR 13.3不兼容，该版本仅适配较低版本的Databricks Runtime（如DBR 11.x），在DBR 13.x环境中会触发废弃错误。

解决方案 #

安装适配DBR 13.x的Mosaic版本（0.4及以上系列），修改安装代码如下：

# Installation
%pip install "databricks-mosaic<0.5,>=0.4" --quiet # 适配DBR 13.x的Mosaic 0.4系列
dbutils.library.restartPython()

# Setup libs
# -- configure AQE for more compute heavy operations
#  - choose option-1 or option-2 below, essential for REPARTITION!
# spark.conf.set("spark.databricks.optimizer.adaptive.enabled", False) # <- option-1: turn off completely for full control
spark.conf.set("spark.sql.adaptive.coalescePartitions.enabled", False) # <- option-2: just tweak partition management
spark.conf.set("spark.sql.shuffle.partitions", 1_024)                  # <-- default is 200

# -- import databricks + spark functions
from pyspark.sql import functions as F
from pyspark.sql.functions import col, udf, explode, to_json, lit
from pyspark.sql.types import *

# -- setup mosaic
import mosaic as mos

mos.enable_mosaic(spark, dbutils)
# mos.enable_gdal(spark) # <- not needed for this example

# --other imports
import os
import pathlib
import requests
import warnings

warnings.simplefilter("ignore")

额外检查项 #

• 执行dbutils.library.restartPython()后，需重新运行整个Notebook，避免环境缓存导致的版本冲突
• 确认DBR 13.3默认的Python 3.9环境与Mosaic 0.4系列兼容

内容的提问来源于stack exchange，提问作者slothPete