Ubuntu 16.04下如何单独统计Python线程的内存占用情况？

Great question—this is a common pain point when working with Python threads, since they share the same process address space (thanks to how Python implements threads on Linux via Lightweight Processes, LWPs). This makes per-thread memory tracking less straightforward than with separate processes, but there are ways to get meaningful insights both at the Linux system level and within your Python code.

Since Python threads map to kernel LWPs, you can leverage Linux's /proc filesystem and specialized tools to isolate per-thread memory metrics (note: most of these focus on thread-specific regions like the stack, since heap memory is shared across all threads in the process):

• Inspect thread-specific stack memory
  Each thread has its own stack, which you can query via the /proc filesystem. For a given process ID <pid> and thread ID <tid> (get TIDs with ps -L <pid>), run:

  cat /proc/<pid>/task/<tid>/status | grep VmStk
  

  This will return the size of the thread's stack in kilobytes. You can also use statm for more granular memory page stats:

  cat /proc/<pid>/task/<tid>/statm
  

  The first value is total virtual memory pages, the second is resident set size (RSS) pages—but keep in mind RSS includes shared pages, so it won't give you a true "per-thread" heap count.

• View all LWPs in top
  Run top -H to launch top in thread mode. This will list every LWP under your process, showing their TIDs and CPU usage. While the %MEM column still reflects the entire process's memory usage, you can cross-reference TIDs with the /proc stack stats above to correlate thread activity with stack growth.

• Use perf for advanced memory tracking
  The perf tool can help you trace memory allocations and accesses per thread. To record memory events for your process:

  perf mem record -p <pid>
  

  Then analyze the output with:

  perf mem report
  

  This will show you which threads are accessing memory most heavily, which can hint at memory growth sources.

Since heap memory is shared across Python threads, you can't get a strict "per-thread heap size"—but you can track which threads are allocating objects, which is critical for identifying leaks. Here are two practical approaches:

• Use tracemalloc (built-in, Python 3.4+)
  tracemalloc lets you track memory allocations and attribute them to specific threads. You can use threading.local() to tag allocations with thread identifiers, then take snapshots to compare usage:

  import tracemalloc
  import threading
  import time
  
  # Thread-local storage to track thread names
  thread_ctx = threading.local()
  
  def worker_thread(thread_name):
      thread_ctx.name = thread_name
      tracemalloc.start()
      
      # Replace this with your actual thread logic
      large_list = [i for i in range(1_000_000)]
      time.sleep(2)
      
      # Take a snapshot and filter by thread's allocations
      snapshot = tracemalloc.take_snapshot()
      # Filter out irrelevant system frames
      thread_stats = snapshot.filter_traces((
          tracemalloc.Filter(False, "<frozen importlib._bootstrap>"),
          tracemalloc.Filter(False, "<unknown>"),
      ))
      
      print(f"\n--- Memory stats for {thread_name} ---")
      for stat in thread_stats.statistics('lineno')[:5]:
          print(stat)
  
  # Launch 8 threads
  threads = []
  for i in range(8):
      t = threading.Thread(target=worker_thread, args=(f"Thread-{i}",))
      threads.append(t)
      t.start()
  
  for t in threads:
      t.join()
  

  This will show you the top memory-consuming lines for each thread, helping you spot where memory is being allocated.

• Track object ownership with objgraph
  The third-party objgraph library lets you visualize and track object creation. You can use it to check which threads are creating persistent objects that might be causing leaks:

  pip install objgraph
  

  Then in your code, add checks at key points to see object counts per thread (you'll need to tag objects with thread IDs when creating them, since Python doesn't track object ownership by thread natively).

• Remember: Python threads share the same heap, so any object created by a thread remains in the process's memory unless explicitly deleted. A "memory leak" here usually means objects are being retained in global variables, thread-local storage, or other shared references.
• If your RAM is growing continuously, combine the system-level stack checks with Python-level allocation tracking to see if threads are accumulating large stacks or leaving behind unused heap objects.

内容的提问来源于stack exchange，提问作者Unknown