1812 – DDOC - Unicode identifiers are not correctly marked.

If a D identifier contains Unicode (such as führer), it will not be correctly marked by DDOC, presumably because DDOC's way of finding identifiers was not upgraded when D started allowing Unicode identifiers. Since there is no reasonable workaround, I've given it major severity. The two attachments are the source (in UTF-8) and the documentation produced from "dmd -D test.d".

Created attachment 222 [details] 
UTF-8 source code showing the problem.

Created attachment 223 [details] 
Documentation produced by "dmd -D test.d".

Looking at the attachments, it seems to work. What's the expected output?
BTW, DDoc is (a lot) newer than D's support for Unicode identifiers.

What you should see is that the first fuhrer is underlined, but the second is not, because it's not converted into a DDOC_DECL.
And hm, you're right - DDoc was 0.132, while Unicode identifiers was 0.74. I guess I just noticed them in the opposite order.

I see, I didn't notice the missing underlining at all. Right you are, a bug it is.

Created attachment 304 [details] 
Use Unicode parsing for identifiers.
This patch makes DDoc use Unicode to parse identifiers, which makes it properly match them in the doc comments and etc.
-[Unknown]

Wanted to add that function parameters with unicode characters don't appear in the generated documentation as well.
/**
 * BUG: The parameters are not listed under Params in the generated output
 *
 * Params:
 * ü = first
 * ş = second
 * ğ = third
 *
 */
int foo(int ü, int ş, int ğ)
{
 return ğ;
}

Fixed dmd 1.051 and 2.036