Implementation of C Standard Library Function ntohl()

Question

This is an implementation of ntohl() that I wrote as an exercise. ntohl() takes a uint32_t value and returns it unchanged if the host architecture is network-byte-order (big-endian), otherwise the value is converted to host-byte-order.

My version converts to little-endian; is it always the case that host-byte-order is taken to mean little-endian? This appears to be the case, from what I have read, but what if the host architecture is middle-endian? Do real implementations of ntohl() detect other byte-orders, or strictly big- and little-endian?

I am also interested in any comments about the use of a union to detect endianness on the host machine, suggestions and comparison with other methods, and similarly, comments and suggestions relating to the use of bitwise operators to perform the conversion from big-endian to little-endian.

#ifndef _STDINT_H
#include <stdint.h>
#endif
uint32_t my_ntohl(uint32_t netlong)
{
 union {
 uint16_t num;
 uint8_t bytes[2];
 } endian_test = { .bytes = { 0x01, 0x00 }};
 if (endian_test.num == 0x0001) {
 netlong = (netlong << 24) | ((netlong & 0xFF00ul) << 8) |
 ((netlong & 0xFF0000ul) >> 8) | (netlong >> 24);
 }
 return netlong;
}

Answer 1

Unless you wish to optimize the code, with specialized swappers for various hosts orders, you are doing it wrong.

I invite you to check The Byte Order Fallacy by Rob Pike. The punch line: the byte order of the computer you are executing the code on doesn't matter, because the language abstracts it for you.

Thus, only the byte order of the network matters, and the network is big-endian:

#include <stdint.h>
#include <string.h>
uint32_t ntohl(uint32_t const net) {
 uint8_t data[4] = {};
 memcpy(&data, &net, sizeof(data));
 return ((uint32_t) data[3] << 0)
 | ((uint32_t) data[2] << 8)
 | ((uint32_t) data[1] << 16)
 | ((uint32_t) data[0] << 24);
}

This function will work no matter the endianness of the host, even on the crazy middle-endians ones.

Oh, and it optimizes well in general, in case you were wondering:

ntohl(unsigned int):
 mov eax, edi
 bswap eax
 ret

bswap being the native CPU instruction to swap bytes on x86.

• Peter - Reinstate Monica: I'm wondering what a typical compiler emits for a big endian machine.
• Matthieu M.: @PeterA.Schneider: hopefully, it just mov the argument into the return slot.
• ad absurdum: If I am understanding this correctly, this is a clever use of the fact that bitshift operations work on the value, not the underlying representation. But I have one question. It seems that this relies on the integer promotions to convert data[i] to unsigned int. Shouldn't there be a cast here, e.g., ((uint32_t) data[3] << 0), so that a uint32_t value is returned? Also, thanks for the link to Pike's essay.
• Matthieu M.: @DavidBowling: Yes, you understand it correctly. As for the cast, it is necessary for portability; this short form relies on the fact that int is at least 4 bytes because all smaller integers are promoted to int before any arithmetic operation occurs. And since it's int, there might be an issue of shifting into the sign bit.
• Matthieu M.: @PatrickRoberts: Same same. Either you explicitly turn data into a pointer like I did, or you rely on the fact that arrays decay into pointer like you did. I prefer explicit, in general, thus &data.

Answer 2

To properly check the byte order, you must check it using uint32_t, since on the PDP-11 the value 0x01020304 was stored as 02 01 04 03, appropriately called middle-endian. There are 21 other possibilites to arrange the bytes.

You don't need the ul suffix since the operands of binary operators are subject to the usual arithmetic conversions. Furthermore, it is considered bad style to use a lowercase ell as a suffix since it can be confused with a 1.

Don't use an inclusion guard around the #include. Just include the header. As it is now, your code depends on the internals of the <stdint.h> header, which it shouldn't.

• ad absurdum: Thanks for the comments. I am still getting used to inclusion guards, and I guess I did not really need one here. I think that the only place I ever use lowercase ells in identifiers is with hexadecimal literals, and this is to set the suffix off from the digits. But it always makes me wince a little, and this is a style point that I have not settled for myself yet.
• ad absurdum: As I mentioned in the question, I only check for little-endian byte order, which is why I used uint16_t. Does the library version of ntohl() check for PDP-endianness? What about other middle-endian schemes?
• Peter - Reinstate Monica: @DavidBowling The ntohl which comes with your compiler is most likely a compiler built-in or comes with the system library for the specific target CPU designed to squeeze out the last bit of performance, i.e. it will not be a function, and it will not be portable, and it will not be C. Some drivers may rely on that performance. It will specifically be not even a NOP for big endian machines. If you strive for portable code, however, @Matthieu has said it all in his answer.

Answer 3

Consider exploiting the compiler

Your compiler already needs to know the endianness of the target system. There's a good chance that it exposes that information in a way you can access. gcc for example defines macros, so you can do:

#if ((__BYTE_ORDER__) == (__ORDER_LITTLE_ENDIAN__))

and have different code blocks as appropriate.

Don't repeat yourself

Your current approach declares endian_test as a local variable. Depending on how clever your compiler is, it may optimize some of the work out for you. However you should consider giving it hints by declaring the variable as a static (you don't need a new instance for every call) const (you don't need to modify it after its declaration).

• ad absurdum: I was aware that such macros exist, but I have not really looked into them yet since they seem to be implementation-dependent. Thanks for the static const tip. I should have thought of this. In my original code I did not use the designated initializer, so const was not an option, and I missed this when I changed it. But missing static was just a silly oversight!
• edmz: Good tip about macros. You generally will check if well-known implementations(clang, gcc, vscc) define such macros to find the BO; if not, simply fall back to runtime checks.