跨C、Python语言使用mmap共享内存的有效性及读写问题咨询

Hey there! Let's tackle your two questions about using mmap across C and Python—this is a common gotcha, but the good news is mmap absolutely works across languages when set up correctly.

1. Does mmap really use shared memory across C and Python? Can it work across different languages? #

Short answer: Yes, 100%. Mmap is an operating system feature, not a language-specific tool. When you map the same file from different processes (regardless of the language they're written in), the OS maps that file to the same physical memory region. Any changes made to that memory by one process will be visible to others—provided you handle cache synchronization properly (more on that later).

The language just provides an interface to the OS's mmap system call. As long as both processes are running on the same OS kernel, mapping the same file with compatible parameters, you're sharing the same underlying memory.

2. Why can't my C code see changes from Python? #

If your C reader isn't picking up Python's writes, it's almost certainly a configuration or synchronization issue, not a problem with mmap itself. Let's walk through the most common mistakes:

Cache Synchronization is Key #

Operating systems use caching to optimize mmap performance, which means changes might not immediately show up across processes.

• In Python: After writing to your mmap object, call mm.flush() to force the changes to be written to disk (and thus visible to other processes). Also, make sure you're creating the mmap with access=mmap.ACCESS_WRITE to ensure write permissions are correctly set.
• In C: If you've already mapped the file, call msync(addr, size, MS_SYNC) to synchronize the mapped memory with the disk contents. This ensures you're reading the latest version, not a cached copy.

Mismatched Mapping Parameters #

Double-check that both processes are mapping the exact same file region:

• Use absolute file paths instead of relative ones to avoid accidentally mapping different files (e.g., "/home/user/shared.bin" instead of "shared.bin").
• Ensure the start offset and length of the mmap are identical in both C and Python. If Python maps the first 1024 bytes and C maps the next 1024, they won't see each other's changes.
• File open modes matter: In C, open the file with O_RDWR (even if you're only reading) instead of O_RDONLY—this helps the OS handle synchronization correctly. In Python, use 'r+b' mode when opening the file for mmap.

Example Correct Code Snippets #

Let's fix your "Reading from C Code: Writing from Python" scenario with properly configured code:

Writing from Python

import mmap

# Open the file in read-write binary mode
with open("shared_file.bin", "r+b") as f:
    # Map the entire file with write access
    mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_WRITE)
    # Write your data
    mm.write(b"Updated content from Python!")
    # Force changes to sync to disk
    mm.flush()
    mm.close()

Reading from C Code

#include <stdio.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <unistd.h>

int main() {
    int fd = open("shared_file.bin", O_RDWR);
    if (fd == -1) {
        perror("Failed to open file");
        return 1;
    }

    // Get the file size to map the entire thing
    off_t file_size = lseek(fd, 0, SEEK_END);
    lseek(fd, 0, SEEK_SET);

    // Map the file with shared access and read/write permissions
    char *mapped_addr = mmap(NULL, file_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if (mapped_addr == MAP_FAILED) {
        perror("mmap failed");
        close(fd);
        return 1;
    }

    // Sync to get the latest changes
    msync(mapped_addr, file_size, MS_SYNC);

    printf("Read from mmap: %s\n", mapped_addr);

    // Cleanup
    munmap(mapped_addr, file_size);
    close(fd);
    return 0;
}

Other Things to Check #

• Are both processes running on the same machine/OS kernel? Mmap doesn't work across separate machines or virtualized environments with different kernels.
• Is the file locked by another process? Tools like flock can prevent write synchronization, so make sure no other process is holding an exclusive lock on the file.

Final Takeaway #

Mmap is fully cross-language compatible—it's all about how you interface with the OS's underlying mechanism. Your issue is almost certainly a synchronization or parameter mismatch problem. Try the fixes above, and you should see the Python changes show up in your C code.

内容的提问来源于stack exchange，提问作者viki