mfilemon.dll中RDP虚拟通道代码无法执行问题求助

Let’s break down why your RDP virtual channel logic works perfectly in a console EXE but fails silently when moved into mfilemon.dll loaded by the Print Spooler service, plus actionable fixes to get it working.

Key Reasons for the Failure #

Your console app runs in an interactive user session (usually Session 1+ with full user privileges), but the Print Spooler service operates in Session 0 with restricted Local Service permissions. Here’s how this breaks your code:

• Session Isolation: WTSVirtualChannelOpen(NULL, (DWORD)-1, ...) uses (DWORD)-1 to target the current session. In the Spooler service, that’s Session 0, but your RDP virtual channel is registered in the user’s interactive session (Session 1+). The service can’t see or access the channel in the user’s session, so the open call fails silently (your code doesn’t check for NULL handles or errors).
• Restricted Service Permissions: The Print Spooler runs as Local Service, which lacks the interactive session privileges needed to interact with RDP virtual channels. Your console app runs with your user’s full permissions, so it can access the channel without issues.
• Missing Error Handling: Your DLL code doesn’t log or check return values for API calls like WTSVirtualChannelOpen or WTSVirtualChannelWrite. You’re assuming the code isn’t executing, but it’s likely running—just failing at the channel open/write step with no feedback.

Step-by-Step Fixes #

1. Target the Correct User RDP Session #

You need to find the session ID of the user who submitted the print job, then open the virtual channel for that specific session. Here’s how to do it:

// Example: Get the target user's session ID from the print job
DWORD GetUserSessionIdFromPrintJob(HANDLE hPrinter, DWORD jobId) {
    DWORD sessionId = -1;
    JOB_INFO_2* pJob = nullptr;
    DWORD bytesNeeded = 0;

    // Get job details to retrieve the submitting username
    if (!GetJob(hPrinter, jobId, 2, nullptr, 0, &bytesNeeded) && GetLastError() == ERROR_INSUFFICIENT_BUFFER) {
        pJob = (JOB_INFO_2*)malloc(bytesNeeded);
        if (pJob && GetJob(hPrinter, jobId, 2, (LPBYTE)pJob, bytesNeeded, &bytesNeeded)) {
            // Enumerate active sessions to find the matching user
            WTS_SESSION_INFO* pSessions = nullptr;
            DWORD sessionCount = 0;
            if (WTSEnumerateSessions(WTS_CURRENT_SERVER_HANDLE, 0, 1, &pSessions, &sessionCount)) {
                for (DWORD i = 0; i < sessionCount; i++) {
                    if (pSessions[i].State == WTSActive) {
                        LPSTR userName = nullptr;
                        DWORD userNameLen = 0;
                        if (WTSQuerySessionInformation(WTS_CURRENT_SERVER_HANDLE, pSessions[i].SessionId, WTSUserName, &userName, &userNameLen)) {
                            if (_stricmp(userName, pJob->pUserName) == 0) {
                                sessionId = pSessions[i].SessionId;
                                WTSFreeMemory(userName);
                                break;
                            }
                            WTSFreeMemory(userName);
                        }
                    }
                }
                WTSFreeMemory(pSessions);
            }
        }
        free(pJob);
    }
    return sessionId;
}

Then use this session ID to open the channel:

DWORD targetSessionId = GetUserSessionIdFromPrintJob(hPrinter, yourJobId);
if (targetSessionId != -1) {
    HANDLE mHandle = WTSVirtualChannelOpen(WTS_CURRENT_SERVER_HANDLE, targetSessionId, (LPSTR)"PRINTWP");
    if (mHandle != NULL) {
        // Proceed with write logic
        PULONG written = 0;
        bool ret = WTSVirtualChannelWrite(mHandle, (PCHAR)data, 22, written);
        // Log results!
        WTSVirtualChannelClose(mHandle);
    } else {
        LogError("WTSVirtualChannelOpen failed", GetLastError());
    }
}

2. Impersonate the Print Job Submitter #

The Print Spooler can impersonate the user who submitted the job to gain the necessary permissions to access their RDP session’s virtual channel. Add this around your channel logic:

if (ImpersonatePrinterClient(hPrinter)) {
    // Your RDP virtual channel code goes here (open, write, close)
    RevertToSelf(); // Don't forget to revert after!
} else {
    LogError("ImpersonatePrinterClient failed", GetLastError());
}

3. Add Detailed Error Logging #

Silent failures are impossible to debug. Add a simple file logger to track every step:

void LogError(const char* operation, DWORD errorCode) {
    std::ofstream logFile("C:\\temp\\spooler_channel_log.txt", std::ios::app);
    if (logFile.is_open()) {
        SYSTEMTIME st;
        GetLocalTime(&st);
        logFile << "[" << st.wYear << "-" << st.wMonth << "-" << st.wDay << " " 
                << st.wHour << ":" << st.wMinute << ":" << st.wSecond << "] "
                << operation << " failed. Error: " << errorCode << std::endl;
        logFile.close();
    }
}

Call this after every API call to see exactly where things go wrong.

4. Verify Virtual Channel Registration #

Ensure your virtual channel DLL (FrzHostedChannel.dll) is properly registered on the RDP server. The server needs to load it for each user session—check that it’s listed in the registry:

• Path: HKLM\SYSTEM\CurrentControlSet\Control\Terminal Server\Wds\rdpwd\VirtualChannels
• Add a string value named PRINTWP pointing to your DLL path.

Final Notes #

• Session 0 isolation is a critical Windows security feature—you can’t bypass it, but you can work around it by targeting the correct user session and impersonating the user.
• Always check return values for every Win32 API call—silent failures are the #1 cause of "code not executing" misconceptions.

内容的提问来源于stack exchange，提问作者David Bentley