为何Ollama未充分利用可用内存？如何配置提速？

环境与问题背景 #

• 部署环境：Windows笔记本WSL2环境，Intel Core i5处理器，主机16GB内存，WSL默认分配7.6GB内存
• 技术栈：通过Ollama拉取LLM模型，基于LangChain框架开发，代码运行在本地Jupyter Notebook
• 性能现状：
  • 使用gemma:2b模型时，chain.invoke({"input": "how can langsmith help with testing?"})请求响应耗时约30秒
  • 使用mistral (7.3B)模型时，响应耗时约1分15秒
• 核心疑惑：Ollama运行期间free -mh显示仅占用1.2GB内存，仍有6.4GB可用内存未被利用，但其他资源密集型项目可正常占用7-8GB内存；Ollama日志显示未检测到GPU，回退至CPU运行

内存使用情况输出 #

➜  ~ free -mh -s 10
               total       used        free        shared      buff/cache  available
Mem:           7.6Gi       1.2Gi       1.7Gi       2.3Mi       4.9Gi       6.4Gi
Swap:          2.0Gi          0B       2.0Gi

Ollama运行日志 #

➜  ~ ollama serve
time=2024-02-27T13:53:29.377+01:00 level=INFO source=images.go:710 msg="total blobs: 5"
time=2024-02-27T13:53:29.378+01:00 level=INFO source=images.go:717 msg="total unused blobs removed: 0"
time=2024-02-27T13:53:29.380+01:00 level=INFO source=routes.go:1019 msg="Listening on 127.0.0.1:11434 (version 0.1.27)"
time=2024-02-27T13:53:29.382+01:00 level=INFO source=payload_common.go:107 msg="Extracting dynamic libraries..."
time=2024-02-27T13:53:34.146+01:00 level=INFO source=payload_common.go:146 msg="Dynamic LLM libraries [rocm_v6 cpu cpu_avx2 cpu_avx cuda_v11 rocm_v5]"
time=2024-02-27T13:53:34.146+01:00 level=INFO source=gpu.go:94 msg="Detecting GPU type"
time=2024-02-27T13:53:34.146+01:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-02-27T13:53:38.249+01:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-02-27T13:53:38.249+01:00 level=INFO source=gpu.go:265 msg="Searching for GPU management library librocm_smi64.so"
time=2024-02-27T13:53:38.249+01:00 level=INFO source=gpu.go:311 msg="Discovered GPU libraries: []"
time=2024-02-27T13:53:38.249+01:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-27T13:53:38.249+01:00 level=INFO source=routes.go:1042 msg="no GPU detected"
[GIN] 2024/02/27 - 13:55:32 | 200 |      37.084µs |       127.0.0.1 | HEAD     "/"
[GIN] 2024/02/27 - 13:55:32 | 200 |     910.269µs |       127.0.0.1 | POST     "/api/show"
[GIN] 2024/02/27 - 13:55:32 | 200 |     945.017µs |       127.0.0.1 | POST     "/api/show"
time=2024-02-27T13:55:32.502+01:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-27T13:55:32.502+01:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-02-27T13:55:32.502+01:00 level=INFO source=llm.go:77 msg="GPU not available, falling back to CPU"
loading library /tmp/ollama930891318/cpu_avx2/libext_server.so
time=2024-02-27T13:55:32.503+01:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama930891318/cpu_avx2/libext_server.so"
time=2024-02-27T13:55:32.503+01:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
llama_model_loader: loaded meta data with 24 key-value pairs and 291 tensors from /home/unix/.ollama/models/blobs/sha256:e8a35b5937a5e6d5c35d1f2a15f161e07eefe5e5bb0a3cdd42998ee79b057730 (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = mistralai
llama_model_loader: - kv   2:                       llama.context_length u32              = 32768
llama_model_loader: - kv   3:                     llama.embedding_length u32              = 4096
llama_model_loader: - kv   4:                          llama.block_count u32              = 32
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                       llama.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  11:                          general.file_type u32              = 2
llama_model_loader: - kv  12:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv  16:                      tokenizer.ggml.merges arr[str,58980]   = ["▁ t", "i n", "e r", "▁ a", "h e...
llama_model_loader: - kv  17:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  18:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  19:            tokenizer.ggml.unknown_token_id u32              = 0
llama_model_loader: - kv  20:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  21:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  22:                    tokenizer.chat_template str              = {{ bos_token }}{% for message in mess...
llama_model_loader: - kv  23:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type q4_0:  225 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 32768
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 4
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 14336
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 32768
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: model type       = 7B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 7.24 B
llm_load_print_meta: model size       = 3.83 GiB (4.54 BPW)
llm_load_print_meta: general.name     = mistralai
llm_load_print_meta: BOS token        = 1 '<s>'
llm_load_print_meta: EOS token        = 2 '</s>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: LF token         = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.11 MiB
llm_load_tensors:        CPU buffer size =  3917.87 MiB
..................................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 1000000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:        CPU KV buffer size =   256.00 MiB
llama_new_context_with_model: KV self size  =  256.00 MiB, K (f16):  128.00 MiB, V (f16):  128.00 MiB
llama_new_context_with_model:        CPU input buffer size   =    13.02 MiB
llama_new_context_with_model:        CPU compute buffer size =   160.00 MiB
llama_new_context_with_model: graph splits (measure): 1
time=2024-02-27T13:55:46.285+01:00 level=INFO source=dyn_ext_server.go:161 msg="Starting llama main loop"
[GIN] 2024/02/27 - 13:55:46 | 200 | 13.934248657s |       127.0.0.1 | POST     "/api/chat"
[GIN] 2024/02/27 - 14:00:03 | 200 |         1m14s |       127.0.0.1 | POST     "/api/generate"

配置方法 #

1. 增大Ollama上下文窗口 #

默认上下文窗口（n_ctx）为2048，增大该值会扩展KV缓存的内存占用，减少磁盘交换频率，同时提升长文本推理效率：

• 临时生效：启动Ollama时指定环境变量

  OLLAMA_N_CTX=8192 ollama serve
  

  或运行模型时直接设置：

  ollama run mistral --ctx 8192
  

• 永久生效：在WSL中创建~/.ollama/config.json文件，添加以下内容后重启Ollama服务

  {
    "n_ctx": 8192
  }
  

2. 启用CPU大页内存 #

WSL2默认可能未开启透明大页内存，启用后可提升CPU内存访问效率，让Ollama更高效利用内存：

• 临时启用：

  echo always | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
  

• 永久启用：编辑/etc/sysctl.conf添加配置，然后生效

  echo "vm.nr_hugepages=1024" | sudo tee -a /etc/sysctl.conf
  sudo sysctl -p
  

3. 调高WSL2内存分配上限 #

手动提升WSL2的内存分配，避免内存瓶颈：
在Windows用户目录下创建.wslconfig文件（路径示例：C:\Users\<你的用户名>\.wslconfig），添加以下配置后执行wsl --shutdown重启WSL：

[wsl2]
memory=12GB
swap=4GB

4. 使用更高精度的模型版本 #

当前使用的mistral是Q4_0量化版本，内存占用小但推理速度慢。可以尝试Q8_0或FP16版本，这类版本会占用更多内存，但CPU推理速度更快：

ollama pull mistral:8b-q8_0

5. 调整Ollama线程数 #

手动指定线程数匹配CPU核心数，提升并行