this patch makes the functions which work directly on multibyte characters treat the high bytes as individual abstract code units rather than as multibyte sequences when MB_CUR_MAX is 1. since MB_CUR_MAX is presently defined as a constant 4, all of the new code added is dead code, and optimizing compilers' code generation should not be affected at all. a future commit will activate the new code. as abstract code units, bytes 0x80 to 0xff are represented by wchar_t values 0xdf80 to 0xdfff, at the end of the surrogates range. this ensures that they will never be misinterpreted as Unicode characters, and that all wctype functions return false for these "characters" without needing locale-specific logic. a high range outside of Unicode such as 0x7fffff80 to 0x7fffffff was also considered, but since C11's char16_t also needs to be able to represent conversions of these bytes, the surrogate range was the natural choice.
previously this flag was defined and accepted as a no-op, possibly breaking some software that uses it. given the choice to remove the definition and possibly break applications that were already working, or simply implement the feature, the latter turned out to be easy enough to make the decision easy. in the case where the FNM_PATHNAME flag is also set, this implementation is clean and essentially optimal. otherwise, it's an inefficient "brute force" implementation. at some point, when cleaning up and refactoring this code, I may add a more direct code path for handling FNM_LEADING_DIR in the non-FNM_PATHNAME case, but at this point my main interest is avoiding introducing new bugs in the code that implements the standard fnmatch features specified by POSIX.
the FNM_PATHNAME logic for advancing by /-delimited components was incorrect when the / character was escaped (i.e. \/), and a final \ at the end of pattern was not handled correctly.
a '/' in the pattern could be incorrectly matched against the
terminating null byte in the string causing arbitrarily long
sequence of out-of-bounds access in fnmatch("/","",FNM_PATHNAME)
unlike the old one, this one's algorithm does not suffer from potential stack overflow issues or pathologically bad performance on certain patterns. instead of backtracking, it uses a matching algorithm which I have not seen before (unsure whether I invented or re-invented it) that runs in O(1) space and O(nm) time. it may be possible to improve the time to O(n), but not without significantly greater complexity.
an invalid bracket expression must be treated as if the opening bracket were just a literal character. this is to fix a bug whereby POSIX left the behavior of the "[" shell command undefined due to it being an invalid bracket expression.