5581 – [64-bit] Wrong code with bitwise operations on bools

The following code produces obviously implausible results on 64-bit builds. result ends up being bigger than 0b111, which is the biggest number that correct code could generate here. (The correct value, which 32-bit builds produce, is 3.) This only happens in this test case when -O and -release are disabled, though I think it still happens with -O and -release enabled in the program I reduced this bug from.
import std.stdio;
void screwy(string[] data) {
 immutable size_t mid = data.length / 2;
 immutable uint result = ((cast(uint) (data[0] < data[mid])) << 2) |
 ((cast(uint) (data[0] < data[$ - 1])) << 1) |
 (cast(uint) (data[mid] < data[$ - 1]));
 stderr.writefln("%b", result);
}
void main() {
 auto stuff = ["a", "a", "b"];
 screwy(stuff);
}

I think this is actually a long-standing, hard to reproduce, non-deterministic codegen bug that strikes on 32-bit as well, though due to the non-deterministic nature of it, under different circumstances. Here's some code that segfaults on 32 with stock DMD 2.051, only with -O -inline -release enabled. This is slightly less reduced than the 64 version because anytime I try to reduce it more, the bug goes away.
import std.stdio;
bool comp(long a, long b) { return a < b; }
size_t medianOf3(T)(T[] data) {
 // Removing this print statement changes the result.
 stderr.writeln(data[0], '\t', data[$ / 2], '\t', data[$ - 1]);
 immutable size_t mid = data.length / 2;
 immutable uint result = ((cast(uint) (comp(data[0], data[mid]))) << 2) |
 ((cast(uint) (comp(data[0], data[$ - 1]))) << 1) |
 (cast(uint) (comp(data[mid], data[$ - 1])));
 assert(result != 2 && result != 5 && result < 8); // Cases 2, 5 can't happen.
 switch(result) {
 case 1: // 001
 case 6: // 110
 return data.length - 1;
 case 3: // 011
 case 4: // 100
 return 0;
 case 0: // 000
 case 7: // 111
 return mid;
 default:
 assert(0);
 }
 assert(0);
}
void main() {
 long[] stuff = [4602111750194969114, 4591797823464657051, 4579622497201165564];
 medianOf3(stuff);
}

Forgot to mention: The 32-bit version still only segfaults on Linux.

I did a little bisecting. This issue appeared in 2.050. It's possible that it's been around since before that but requires different magic numbers on earlier releases, though I doubt it because I would have probably noticed the bug if it was around for a long time. I do my day to day work on Windows but use Linux for the occasional computationally intensive project. I haven't done much serious (i.e. Linux-based) computation since 2.050 was released, but I was doing a lot of it earlier on. The function I isolated this test case from was around in its current form back then and was definitely getting called a lot in those projects.

Created attachment 913 [details] 
Diff of 2.049 vs. 2.050 ASM output.
One more hint: Here's a diff of the ASM output for the function body of medianOf3 between 2.049 (test49.asm) and 2.050 (test50.asm). The most notable difference is that in 2.050 the comparison function gets inlined. In 2.049 it doesn't.

Unfortunately this appears to be a Heisenbug. When I put print statements in to figure out what's going on and call the switch statement function manually, it starts working. However, the one hint I did get from mucking around in switch_.d is that replacing the memcmp() call with a manual for loop and recompiling druntime solves the problem. Not sure if this is due to the Heisenbug nature of this thing or if it indicates that the memcmp() call is the culprit.

Ignore the last comment. It was directed to the wrong bug report.

https://github.com/D-Programming-Language/dmd/commit/175bb7540db3ad16655772fe47e4dcbeab7fa380
https://github.com/D-Programming-Language/dmd/commit/cfc773a5bf99b6e636b03644787b71f4acc1c622
This addresses the 64 bit code gen problems, not the 32 bit issue.

Fix for 32 bit code (completely different from 64 bit bug):
https://github.com/D-Programming-Language/dmd/commit/52879f36cb28b05813161002d559fecc11a0cbe6
https://github.com/D-Programming-Language/dmd/commit/525de3d38d8202a810352e6811328afa19166591