如何在Python中像SD WebUI一样为DreamShaper加载LORA

错误分析与修复步骤 #

1. 加载Safetensors模型失败（KeyError: 'safetensors'） #

出现该错误是因为diffusers版本未内置safetensors支持，或缺少对应依赖包。

• 安装依赖包：

pip install safetensors

• 更新diffusers及相关库到最新版：

pip install --upgrade diffusers transformers accelerate

• 完成后可直接使用safetensors格式模型文件，无需转换为ckpt。

2. 加载LORA时的MemoryError #

该错误源于torch.load加载大文件时内存不足，改用safetensors专属加载方法即可解决，同时注意多LORA的加载逻辑：

• 不要直接用pipeline.unet.load_attn_procs加载safetensors文件，先通过safetensors.torch.load_file读取权重再传入。
• 多LORA依次加载时注意内存管理。

3. CLIP权重警告说明 #

CLIP未使用权重的警告是正常现象——SD模型仅用到CLIP的文本编码器部分，视觉模块权重不会被加载，可直接忽略。

修正后的完整代码 #

import torch
from safetensors.torch import load_file
from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler

# 加载内置VAE的DreamShaper模型（safetensors格式）
pipeline = StableDiffusionPipeline.from_ckpt(
    "./DreamShaper_5_beta2_BakedVae.safetensors",
    torch_dtype=torch.float16,
    use_safetensors=True
)

# 设置DPMSolverMultistep调度器
pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config)

# 加载第一个LORA：Flat Illustration
lora_weights1 = load_file("./flat illustration.safetensors")
pipeline.unet.load_attn_procs(lora_weights1)

# 加载第二个LORA：Improve Backgrounds
lora_weights2 = load_file("./improve_backgrounds.safetensors")
pipeline.unet.load_attn_procs(lora_weights2)

# 移至GPU并启用xformers内存优化
pipeline.to("cuda")
pipeline.enable_xformers_memory_efficient_attention()

# 提示词（diffusers无需<lora:xxx:xx>格式，权重通过参数控制）
prompt = "Flat vector illustration of a scary and ominous grassy landscape with five or more trees, a large crack in the ground, and a gigantic monster sticking up high above the crack. The monster is based on an oak tree and made up of all kinds of litter and debris, including cans and bottles. The landscape is scattered with lots of litter and debris, especially tipped over garbage cans. There are hundreds of people running away from the monster, and the environment is dusty with no texture or shading. The color scheme of the grassy landscape is green and brown."
negative_prompt = "(deformed iris, deformed pupils, anime:1.4), text, close up, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck, extremely focused on people"

# 生成图片，设置对应LORA权重（顺序与加载顺序一致）
image = pipeline(
    prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=40,
    guidance_scale=7.5,
    cross_attention_kwargs={"scale": [1.0, 0.85]}
).images[0]

image.save("monster_landscape.png")

关键补充说明 #

• 加载safetensors模型必须指定use_safetensors=True参数。
• 多LORA的权重通过cross_attention_kwargs={"scale": [权重1, 权重2]}设置，顺序需与LORA加载顺序对应。
• 若仍存在内存问题，可尝试启用pipeline.enable_attention_slicing()进一步降低内存占用。

内容的提问来源于stack exchange，提问作者Day Trip