`windows-rdl` / clang: function-like macro to wide string literal emits wrong rdl

[riverar]

C header:

#define	DebugWide ( L"Dbg" )
#define	Debug ( "Dbg" )

Actual RDL:

#[win32]
mod Test {
    const DebugWide: i32 = 0;
    const Debug: i32 = 0;
}

Expected RDL:

#[win32]
mod Test {
  #[Windows::Win32::Foundation::Metadata::NativeEncoding("utf-16")]
  const DebugWide: String = "Dbg";
  #[Windows::Win32::Foundation::Metadata::NativeEncoding("ansi")]
  const Debug: String = "Dbg";
}

[kennykerr]

Do we need "ansi" in this example? What if its UTF-8?

[riverar]

I don't think we established a default yet (?), so I matched our RDL tests in terms of emission of all native encoding attributes.

It's possible that the unspecified literal in #define Debug ( "Dbg" ) represents a UTF-8 string. But I believe MSVC today, without /utf-8 will interpret that as a narrow string. We could lean more heavily on the source file encoding to resolve this ambiguity, but that's going to surprise everyone with UTF-8+ codebases.

(Also, yes, we can add a u8"Dbg" example too.)

[kennykerr]

I guess I'm not clear on what the downside is of just assuming all literals are UTF-8? Does Windows.Win32.winmd have any "ansi" literals that aren't valid UTF-8? That may be a simple check to perform.

[kennykerr]

We could also use the enum-style repr attribute for a concise encoding in RDL. For example:

#[win32]
mod Test {
  #[repr("utf-16")]
  const DebugWide: String = "Dbg";
  #[repr("ansi")]
  const Debug: String = "Dbg";
}

[riverar]

I guess I'm not clear on what the downside is of just assuming all literals are UTF-8? Does Windows.Win32.winmd have any "ansi" literals that aren't valid UTF-8? That may be a simple check to perform.

These narrow "ansi" string literals were likely written against the Windows-1252 code page, not strict ASCII, so may contain non-ASCII characters that are incompatible with UTF-8. (Consider a header with #define PRICE "€9.99".) We could check the Windows SDK to see if any such literals exist in practice, but these literals may also exist in external headers.

(Because MSVC /utf-8 may or may not be in use, string literal encoding could become a tweakable knob in the future.)

[Alex781937]

Hey, I might be missing some context here, but I have a question about the encoding choice.

If the goal of RDL is to faithfully represent the original Win32/WinRT APIs, wouldn’t it make sense to preserve the original distinction (UTF-16 vs ANSI) rather than normalizing to UTF-8?

From what I understand, a lot of Win32/WinRT APIs fundamentally operate on UTF-16, and even when UTF-8 is used at higher levels (e.g. std in Rust or other languages), it often gets converted to UTF-16 when calling into the OS.

So is the intent here:

• to reflect the original API semantics as closely as possible, or
• to align with a more modern, UTF-8-centric developer experience?

Curious how you’re thinking about that tradeoff.