sizeof tm struct (datetime) different on ESP32 vs linux64x

Question

I am logging binary data to my SD. I log a datetime object of type 'tm', which is 36 bytes on ESP32. When I create a datetime of type 'tm' on linux it is 56 bytes.

ESP32 write tm to SD: [tm datetime = 36 bytes]

time_t t = std::time(nullptr);
tm datetime = *std::localtime(&t);
uint16_t t = file.write((const uint8_t *)&datetime, sizeof(datetime)); // ==36 bytes.

PC [tm datetime = 56 bytes] (compiled using g++ on linux 64)

tm datetime;
std::ifstream fileStream(filename, std::ios::binary);
fileStream.read((char *)&datetime, sizeof(datetime));

tm is defined in time.h:

struct tm
{
 int tm_sec;
 int tm_min;
 int tm_hour;
 int tm_mday;
 int tm_mon;
 int tm_year;
 int tm_wday;
 int tm_yday;
 int tm_isdst;
#ifdef __TM_GMTOFF
 long __TM_GMTOFF;
#endif
#ifdef __TM_ZONE
 const char *__TM_ZONE;
#endif
};

My tm struct doesn't have the 'tm_gmtoff' or 'tm_zone', even if it had it wont add up to 56 bytes.

How can my tm be 56 bytes on my PC? If someone has a better way to log a datetime object, without this issue, let me know.

• PMF: sizeof(int) on arduino (AVR) is 2, on most other platforms its 4.
• Edgar Bonet: Why log in binary? Log as text, in the format of date +'%FT%T%:z'. If you really need binary, log time_t rather than strcut tm.
• washichi: @PMF I'm using ESP32, didn't specify, my bad!
• washichi: @EdgarBonet logging to text is too slow, I log at 100hz using sd.h lib. why is logging time_t better in binary?
• Edgar Bonet: The layout of strcut tm is implementation dependent. In contrast, time_t is usually a simple integer, with very little variations across implementations: either 32 or 64 bits, and maybe a different epoch (which is fixed by simply adding a constant).

Answer 1

You can not trust system-level structs for data storage or transfer between different systems. The main reasons are:

1. Different data type sizes (16 / 32 / 64 bit ints, floats vs doubles, etc)
2. Struct element alignment differences owing to underlying architecture optimizations and limitations

Unless you are working with a system struct that is specifically designed for cross-system communication you must define your own and pay special attention to the layout.

1. Always use explicitly sized variable types (uint8_t, int32_t, etc for example) to ensure both ends always agree on data storage sizes
2. Always "pack" your struct (__attribute__((packed))) to collapse any specially aligned variables into just the room needed to store those variables

Point 2 requires special attention if you're doing much more than filling the struct and saving / transmitting it. It's advisable to never do more than create the struct instance, fill it, save it, then throw it away. If you start doing advanced operations with pointers to the struct you risk corrupting the packing code if the alignment changes. Also packed structs are often slightly slower to access owing to more complex code usually needed to be used to read / write the individual fields within them (especially if there are no byte-wise operations in the underlying architecture).

• washichi: Thank you! multiple people commented solutions, but you gave the reason why. Also thanks for your blogpost, very helpful!