将窄字符串转换为宽字符串时为何用0xFF掩码字符？

Great question! Let’s unpack the logic behind that 0xFF in your fallback conversion code—this is all about avoiding a tricky quirk of how signed characters behave when converted to wider types.

First, a critical detail: on most systems, char is a signed 8-bit type (range: -128 to 127). If your input string includes bytes with values greater than 127 (like extended ASCII symbols, Latin-1 characters, or invalid UTF-8 fragments), treating these signed char values directly would lead to a problem calledsign extensionwhen converting to wchar_t.

What’s sign extension, and why is it a problem here? #

Imagine you have a byte with the value 0xA0 (decimal 160). As a signed char, this gets interpreted as -96. When you assign this to a wchar_t (which is usually 16 or 32 bits wide), the system fills all the higher bits of the wchar_t with the sign bit (the highest bit of the char, which is 1 here). Instead of getting the desired 0x00A0 (for 16-bit wchar_t), you’d end up with 0xFFA0—a negative value in the wide string that doesn’t map to any valid character you intended.

How does input[i] & 0xFF fix this? #

The 0xFF mask is an 8-bit unsigned value. Here’s what happens when you use it:

1. The signed char gets promoted to an int (a wider signed type). For a negative char like -96 (0xA0), this becomes 0xFFFFFFA0 in a 32-bit int.
2. The bitwise AND with 0xFF clears all bits except the lower 8, resulting in 0x000000A0—a positive, unsigned value between 0 and 255, exactly the raw byte value we want.
3. When this value is assigned to wchar_t, there’s no sign extension (since it’s a positive integer), so you get the correct 0x00A0 (for 16-bit wchar_t) or 0x000000A0 (for 32-bit).

Context in your function #

This fallback code runs when the UTF-8 to UTF-16 conversion fails (e.g., invalid UTF-8 sequences). The goal is a "best effort" conversion: take each raw byte from the narrow string and treat it as a single-byte character (like Latin-1) to populate the wide string. Without the 0xFF mask, any byte >127 would get mangled by sign extension, producing invalid or garbage wide characters.

In short, the 0xFF mask ensures we treat each char as an unsigned 8-bit byte, avoiding sign extension bugs that would break the fallback conversion.

内容的提问来源于stack exchange，提问作者MistyD