在采样perf_event_open文件描述符上使用epoll返回EPOLLHUP问题
我正在编写一个基于Linux Perf API的程序,用来统计目标程序运行时的指令数,并且希望在子进程达到指定指令数时收到溢出通知。Perf官方手册中提到了两种捕获溢出通知的方式:
Overflow handling
Events can be set to notify when a threshold is crossed,
indicating an overflow. Overflow conditions can be captured by
monitoring the event file descriptor with poll(2), select(2), or
epoll(7). Alternatively, the overflow events can be captured via
a signal handler, by enabling I/O signaling on the file
descriptor; see the discussion of the F_SETOWN and F_SETSIG
operations in fcntl(2).
由于程序特性限制,我无法采用启用I/O信号并设置信号处理函数的方式,只能选择用epoll监听perf文件描述符。但实际运行时,epoll_wait持续返回EPOLLHUP(即使还没到触发溢出通知的时机也是如此),始终无法收到预期的EPOLLIN。我查阅了大量关于perf_event_open用法的资料,还是找不到问题原因。
以下是我使用的简化版源代码:
#include <bits/stdc++.h> #include <linux/perf_event.h> #include <sys/mman.h> #include <sys/stat.h> #include <sys/wait.h> #include <sys/ioctl.h> #include <sys/syscall.h> #include <sys/epoll.h> #include <fcntl.h> #include <unistd.h> using namespace std; char* stringToCStr(const string &str) { char* cStr = new char[str.size() + 1]; copy(str.begin(), str.end(), cStr); cStr[str.size()] = '\0'; return cStr; } static long perf_event_open(struct perf_event_attr *hw_event, pid_t pid, int cpu, int group_fd, unsigned long flags) { return syscall(SYS_perf_event_open, hw_event, pid, cpu, group_fd, flags); } long long get_instructions_used(int perf_fd) { long long int instructionsUsed; int size = read(perf_fd, &instructionsUsed, sizeof(long long)); if (size != sizeof(instructionsUsed)) { cout << "read failed"; exit(0); } if (instructionsUsed < 0) { cout << "read negative instructions count"; exit(0); } return static_cast<uint64_t>(instructionsUsed); } int main() { pthread_barrier_t *barrier_ = (pthread_barrier_t*) mmap(nullptr, sizeof(pthread_barrier_t), PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_SHARED, 0, 0); pthread_barrierattr_t attr; pthread_barrierattr_init(&attr); pthread_barrierattr_setpshared(&attr, PTHREAD_PROCESS_SHARED); pthread_barrier_init(barrier_, &attr, 2); pthread_barrierattr_destroy(&attr); int pid = fork(); if(pid == 0) { pthread_barrier_wait(barrier_); munmap(barrier_, sizeof(pthread_barrier_t)); auto programName = stringToCStr("test"); char** programArgv = new char*[2]; programArgv[0] = programName; programArgv[1] = nullptr; execv(programName, programArgv); }; struct perf_event_attr attrs {}; memset(&attrs, 0, sizeof(attrs)); attrs.type = PERF_TYPE_HARDWARE; attrs.size = sizeof(attrs); attrs.config = PERF_COUNT_HW_INSTRUCTIONS; attrs.exclude_user = 0; attrs.exclude_kernel = 1; attrs.exclude_hv = 1; attrs.disabled = 1; attrs.enable_on_exec = 1; attrs.inherit = 1; attrs.sample_period = 500000000LL; attrs.wakeup_events = 1; int perf_fd = perf_event_open(&attrs, pid, -1, -1, PERF_FLAG_FD_NO_GROUP | PERF_FLAG_FD_CLOEXEC); pthread_barrier_wait(barrier_); pthread_barrier_destroy(barrier_); munmap(barrier_, sizeof(pthread_barrier_t)); // Uncomment to enable capturing events via signal handler /*int my_pid = getpid(); fcntl(perf_fd, F_SETOWN, my_pid); int old_flags = fcntl(perf_fd, F_GETFL, 0); fcntl(perf_fd, F_SETFL, old_flags | O_ASYNC);*/ int epoll_fd = epoll_create1(EPOLL_CLOEXEC); struct epoll_event event; event.events = EPOLLIN; event.data.u64 = 1; epoll_ctl(epoll_fd, EPOLL_CTL_ADD, perf_fd, &event); while(true) { struct epoll_event events[1]; epoll_wait(epoll_fd, events, 1, -1); cout << "event: " << events[0].events << ", u64: " << event.data.u64 << ", instruction count: " << get_instructions_used(perf_fd) << "\n"; if(events[0].events == 1) break; } return 0; }
子进程运行的测试可执行文件(对应代码中programName变量的"test")可以是任何消耗大量指令的简单程序,例如以下代码编译后的可执行文件:
#include "bits/stdc++.h" using namespace std; int main() { int n = 1e8, primeCount = 0; vector<bool> sieve(n + 1, true); for(int i = 2; i <= n; i++) { if(sieve[i]) { for(int j = i * 2; j <= n; j += i) sieve[j] = false; primeCount++; } } cout << primeCount << "\n"; return 0; }
这段代码持续输出event: 16, u64: 1, instruction count: <instruction count>,也就是说epoll一直从perf_fd接收到EPOLLHUP。我曾怀疑是因为perf事件初始为禁用状态,仅在子进程执行execv时才启用(由enable_on_exec指定),导致epoll在短时间内监听的是禁用状态的perf_fd,但即使在epoll_ctl前添加短暂的usleep确保perf事件已启用,问题依然存在。
另外需要注意的是,如果取消epoll_create1上方注释的代码(即通过SIGIO信号捕获溢出通知,也就是手册中提到的另一种方式),信号会在正确时机触发并终止进程。这说明溢出通知本身是正常发送到文件描述符的,问题出在epoll的使用上,它无法识别这些通知,反而持续返回EPOLLHUP。
内容的提问来源于stack exchange,提问作者Mikołaj Kołek

