跨进程读取fallocate创建文件的写入边界问题及解决方案

我有两个进程：writer进程先通过fallocate(2)创建固定长度的空文件，再向其中写入数据；reader进程需要读取到writer最后写入的位置（即“边界”），之后等待writer写入新数据。

我编写了简单的示例程序，但reader进程读取速度略快，会在某一时刻触及文件边界。此时reader应回退一个数据元素并重试，直到读取到非零数据，但该逻辑仅成功一次，后续始终读取到0。不过通过终端复制文件时，所有预期数据都存在，说明数据已写入但无法被可靠读取。

最终通过在reader进程的fopen后添加setvbuf(fp, NULL, _IONBF, 0)禁用stdio缓冲解决了问题。

WRITER进程代码 #

/*
  g++ fWriter.cpp -o fWriter -g -pg
*/

#include <stdio.h>
#include <fcntl.h>
#include <stdint.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <unistd.h>

int
main(int argc, char **argv)
{
    printf("WRITER LAUNCHED\n");

    const int N = 100;

    FILE *fp = fopen("/tmp/someFile", "w");
    if (!fp)
    {
        perror("fopen");
        exit(1);
    }

    int   itemLen = sizeof(uint32_t);
    off_t     len = itemLen * N;

    int fd = fileno(fp);
    if (fallocate(fd, 0, 0, len) == -1)
    {
        fprintf(stderr, "fallocate failed: %s\n", strerror(errno));
        exit(1);
    }

    for (uint32_t i = 1; i <= N; i++)
    {
        int numBytes = fwrite((void*)&i, 1, itemLen, fp);
        if (numBytes == itemLen)
            printf("[%u] Wrote %u\n", i, i);
        else
            printf("Write error\n");

        fflush(fp);

        sleep(1);
    }

    fclose(fp);

    printf("WRITER COMPLETE\n");
}

READER进程代码 #

/*
  g++ fReader.cpp -o fReader -g -pg
*/

#include <stdio.h>
#include <fcntl.h>
#include <stdint.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <unistd.h>

int
main(int argc, char **argv)
{
    printf("READER LAUNCHED\n");

    FILE *fp = fopen("/tmp/someFile", "r");
    if (!fp)
    {
        perror("fopen");
        exit(1);
    }

    
    // 新增：禁用stdio缓冲
    setvbuf(fp, NULL, _IONBF, 0);

    int   i = 1;
    int   itemLen = sizeof(uint32_t);

    while (!feof(fp))
    {
        uint32_t val;

        int numBytes = fread((void*)&val, 1, itemLen, fp);

        if (numBytes != itemLen)
        {
            printf("Short read\n");
            continue;
        }

        printf("[%d] Read %u\n", i, val);

        // 等待新数据，回退并重试
        if (val == 0)
        {
            off_t curPos = ftello(fp);
            off_t newPos = curPos - itemLen;
            printf("Seek and try again. newPos %d\n", newPos);
            fseeko(fp, newPos, SEEK_SET);
        }

        i++;

        //usleep(1100000);
        usleep(900000);
    }

    fclose(fp);
}

READER进程日志 #

READER LAUNCHED
[1] Read 1
[2] Read 2
[3] Read 3
[4] Read 4
[5] Read 5
[6] Read 6
[7] Read 0
Seek and try again. newPos 24
[8] Read 7
[9] Read 8
[10] Read 9
[11] Read 10
[12] Read 11
[13] Read 0
Seek and try again. newPos 44
[14] Read 0
Seek and try again. newPos 44
[15] Read 0
Seek and try again. newPos 44
[16] Read 0
Seek and try again. newPos 44
[17] Read 0
Seek and try again. newPos 44
[18] Read 0
Seek and try again. newPos 44
[19] Read 0
Seek and try again. newPos 44
[20] Read 0
Seek and try again. newPos 44
[21] Read 0
Seek and try again. newPos 44
[22] Read 0
Seek and try again. newPos 44

问题核心原因 #

出现该问题的关键是stdio默认缓冲机制：reader使用fread时，默认采用全缓冲/行缓冲模式，当它提前读取到文件中未写入的0值区域时，这些0值会被缓存到stdio缓冲区中。后续即使writer写入了新数据，fread仍会优先从缓冲区读取旧的0值，而非磁盘上的最新内容，导致始终读取失败。

解决逻辑 #

在reader打开文件后立即调用setvbuf(fp, NULL, _IONBF, 0)禁用stdio缓冲，让fread每次直接从磁盘读取文件最新内容，确保能正确获取writer后续写入的数据。

内容的提问来源于stack exchange，提问作者Danny