x86-64/Windows下自定义协程库上下文切换崩溃问题求助

Let’s break down the likely issues in your Win64 coroutine implementation—stack alignment, register handling, and context initialization are easy to mess up with this strict ABI, and these are almost certainly the cause of your crash in __cotransfer.

1. Critical _coprepare Bug: Incorrect RIP Initialization #

Your _coprepare routine isn’t properly setting the entry point of the coroutine. Right now you’re loading a random stack address into the new context’s RIP instead of the actual coroutine function (func). On Win64, the 5th function argument lives at [RSP + 32] (since the first 4 are in RCX/RDX/R8/R9, and the caller allocates 32 bytes of shadow space). Fix this by loading func correctly:

; Replace the invalid lea R11 line with this to get the func parameter
mov R11, [RSP + 32]
mov [RDX + OFF_RIP], R11

2. __cotransfer Overwrites Saved RSP (Fatal Stack Corruption) #

In __cotransfer, you first save the old RSP to the context, then immediately overwrite it with a wrong value:

;; save stack frame info
lea R10, [RSP - 8] ; save SP, exclude IP
lea R11, [RSP] ; save IP
mov [RCX + OFF_RSP], R10
mov [RCX + OFF_RIP], R11

;; switch stacks
mov RAX, RSP
mov RSP, [RDX + OFF_RSP]
mov [RCX + OFF_RSP], RAX ; <-- THIS OVERWRITES THE SAVED RSP!

When restoring the old context later, you’ll get a garbage RSP value, leading to an immediate crash. Delete that final line entirely—you already saved the correct old RSP earlier.

3. Incorrect RSP/RIP Saving Logic in __cotransfer #

When you enter __cotransfer, the stack has the return address (pushed by the call instruction) at the top of the stack, with RSP pointing directly to it. Your current code adjusts RSP incorrectly, which breaks the context state. Fix the stack frame saving logic:

;; Fixed stack frame saving
mov R11, [RSP] ; Get the caller's return address (old RIP)
mov R10, RSP ; Save current RSP (points to the return address)
mov [RCX + OFF_RSP], R10
mov [RCX + OFF_RIP], R11

When restoring this context, setting RSP back to R10 and executing ret will correctly jump back to the caller’s code.

4. Win64 Stack Alignment & Shadow Space Violation #

Win64 enforces two non-negotiable stack rules that your code is ignoring:

1. 16-byte stack alignment: RSP must be aligned to 16 bytes before any call instruction (including implicit ret which acts like a call to the saved RIP).
2. 32-byte shadow space: Every function call requires 32 bytes of scratch space on the stack for the callee to use, even if it doesn’t utilize all of it.

Fix your _coprepare RSP initialization to account for these rules:

;; Fixed new context RSP setup
lea R10, [R8 + R9] ; Start at stack top (stack grows downward)
sub R10, 32 ; Reserve 32 bytes of shadow space
and R10, ~0Fh ; Force 16-byte alignment
mov [RDX + OFF_RSP], R10
mov [RDX + OFF_RBP], R10 ; Initialize RBP to aligned RSP (flat stack frame)

5. Coroutine Function Parameter Handling #

Your API states the coroutine function takes two parameters: the coroutine context and a void* argument. On Win64, these need to be in RCX and RDX (the first two argument registers):

• In coenter, set the new context’s RCX to the coroutine* pointer and RDX to enter_arg before calling __cotransfer.
• In coyield, save these registers to the context so they’re preserved across yields.

Debugging Tips #

Since MSVC debugging gets confused across stack switches, use WinDbg for lower-level debugging:

• Insert an int3 instruction at the start of __cotransfer to break into the debugger before the stack switch.
• Check the values of RSP, RBP, and RIP before and after the stack switch—verify they match your expected context values.
• Inspect the stack memory to ensure the new stack is properly aligned and has the expected shadow space reserved.

内容的提问来源于stack exchange，提问作者Jesse Lactin