Linux内核tcp_recvmsg参数关系及inet_recvmsg调试异常问询

Let's break this down step by step, starting with your core questions, then unpacking the issue you're seeing with iov_iter.nr_segs.

1. Relationship between tcp_recvmsg Return Value and the msg Parameter #

When tcp_recvmsg returns a positive integer, that value represents the exact number of bytes successfully copied into the user-space buffer pointed to by msg->iov_iter. Your initial understanding here is mostly correct: this return value confirms data has been written to the user's buffer.

However, there's a critical detail you missed: the kernel modifies the iov_iter structure during the copy operation. The iov_iter acts as a cursor tracking how much of the user's buffer has been filled. After tcp_recvmsg completes, the iterator is advanced past the bytes that were just copied—this means it no longer points to the start of the user's buffer, and its metadata (like nr_segs) reflects the remaining unused space, not the original buffer state.

2. Why msg->iov_iter.nr_segs is 0 #

In Linux 4.9.x, the tcp_recvmsg function uses copy_to_iter() (or similar helper functions) to move data from kernel socket buffers to user space. These helpers update the iov_iter in place:

• If the total bytes copied (err=8912 in your case) exactly matches the total available space in the user's iov segments, the iterator will reach the end of all segments. When this happens, nr_segs is set to 0 to indicate there are no remaining unused segments in the iterator.
• Even if you had multiple iov segments, once all of them are fully written to, nr_segs will drop to 0.

Your confusion comes from expecting nr_segs to represent the original number of segments in the user's buffer, but it actually tracks the remaining unused segments after data copy.

3. Fixing Your Custom Operation Timing #

Your goal is to run code right before data is delivered to user space—but you're hooking the wrong point in the call chain. By the time inet_recvmsg reaches your my_function(), the data has already been copied to user space, and the iov_iter is already modified.

Instead, you should inject your logic inside tcp_recvmsg, just before the kernel calls copy_to_iter() to write data to user space. For Linux 4.9, this would be in the section where tcp_recvmsg prepares the data to be copied (look for calls to skb_copy_datagram_iter() or copy_to_iter()).

If you need to keep the hook in inet_recvmsg for some reason, you can save a copy of the original iov_iter state before calling sk->sk_prot->recvmsg(), like this:

// Add this before calling sk->sk_prot->recvmsg()
struct iov_iter original_iter = msg->msg_iter;

err = sk->sk_prot->recvmsg(sk, msg, size, flags & MSG_DONTWAIT, flags & ~MSG_DONTWAIT, &addr_len);

// Now use original_iter in my_function if you need the original buffer state
if (err > 0 && sk->sk_family == AF_INET && sk->sk_protocol == IPPROTO_TCP)
    my_function(sk, &original_iter, err);

Note: You'll need to adjust my_function to accept the saved iterator instead of the modified one.

Key Takeaways #

• tcp_recvmsg's positive return value = bytes copied to user space, and msg->iov_iter is updated to reflect remaining buffer space (not original state).
• nr_segs=0 means all user buffer segments were fully written to during the copy.
• To act on data before it reaches user space, hook earlier in tcp_recvmsg, not after the copy completes in inet_recvmsg.

内容的提问来源于stack exchange，提问作者Fan Xiaoyu