limit length of the generated timestamp from ESP32 with DS3231 to 13

Question

I am using ESP-WROOM-32 (Olimex ESP32-PoE) with DS3231. I have used the a patch from PR on RTClib to avoid the rollover from the DS3231.

Since the DS3231 RTC provides Unix Epoch Timestamps which have a precision of only seconds, I wish to concatenate the millis() to the above mentioned timestamps to provide millisecond precision.

This is critical since I want to store information to InfluxDB from the nodes with ms precision. For that the timestamps should always be 13 digits.

code

#include <RTClib.h>
#include <Wire.h>
// PoE setting for Olimex PoE board
#define ETH_CLK_MODE ETH_CLOCK_GPIO17_OUT
#define ETH_PHY_POWER 12
// Pins for I2C on Olimex PoE board
#define I2C_SDA 13
#define I2C_SCL 16
RTC_DS3231 rtc;
RTC_Millis software_time;
void setup() {
 // put your setup code here, to run once:
 Wire.begin(I2C_SDA, I2C_SCL, 400000);
 Serial.begin(9600);
 if (!rtc.begin()) {
 Serial.println("No RTC Detected");
 }
 // adjust RTC Time
 rtc.adjust(DateTime(F(__DATE__), F(__TIME__)));
 // adjust software timer
 software_time.adjust(rtc.now());
}
void loop() {
 // put your main code here, to run repeatedly:
 DateTime rtc_now_time = rtc.now();
 DateTime software_now_time = software_time.now();
 Serial.println();
 Serial.print("RTC now: "); Serial.print(rtc_now_time.unixtime());
 Serial.println();
 Serial.print("millis: "); Serial.print(millis());
 delay(100);
}

Output

RTC now: 1538502183
millis: 28
RTC now: 1538502183
millis: 128
RTC now: 1538502183
millis: 228
RTC now: 1538502183
millis: 328
RTC now: 1538502183
millis: 428
RTC now: 1538502183
millis: 528
RTC now: 1538502183
millis: 628
RTC now: 1538502183
millis: 728
RTC now: 1538502183
millis: 828
RTC now: 1538502183
millis: 928
RTC now: 1538502184
millis: 1028

I will store the timestamps in String so I was thinking of concatenating the .unixtime() with millis(). However since I want all my timestamps to be of fixed lenght (13) with the increasing millis value seems to be difficult because as soon as the number goes more that 999 the a digit increases making the string length 14.

Side Effect

if the timestamps for ms do not match length of 13, InfluxDB rejects the data passed through

Epoch Time

the output provided above is of 10 digits which is the epoch time since 1970. The precision of the timestamps above is of seconds. If one wants to add milliseconds to it one needs to add 3 more digits behind it. If precision needs to be microseconds it should have 6 digits and so on.. One can check the timestamps above on [https://epochconverter.com]

Any suggestions on overcoming this issue ?

• Jot: Please explain the 13 digits or give a link to the format. Is the format "1234567890123"? As readable text with a decimal number? Should it be the number of milliseconds since 1970? Epoch = 1538498157 which is 10 digits, that means three extra digits for milliseconds? Will it still work if you make the three digits for the milliseconds zero? This number: currentmillis.com ?
• Shan-Desai: Yes. it can work even if the milliseconds are zero. unfortunately the rtc does not provide ms precision hence I am adding the millis which is ms from the beginning of the controller.
• Jot: So making them zero solves all problems? This is a similar question: stackoverflow.com/questions/44644383/… You can do millis() % 1000.
• Juraj: but it can give in one second two timestamps in wrong ordering
• Jot: @Juraj what is the solution? How to get some kind of synchronization? Perhaps set the sqw output to 1Hz and tie it to an interrupt. In the interrupt set a previousMillis. Shan-Desai, can you connect the sqw signal to an interrupt input?

Answer 1

For getting millisecond resolution out of an RTC, I think the only viable solution is to sync with the 1 Hz square wave output, as suggested by Jot in a comment. Note that, since you are only counting seconds and milliseconds, you probably don't care about hours, minutes, days, months, etc. So you won't use the RTC for timekeeping: only for providing the 1 Hz reference signal and, maybe, an initial timestamp at program startup.

Connect the SQW pin of the DS3231 to an interrupt-capable input of your MCU. Add the following to your setup():

pinMode(SQW_PIN, INPUT_PULLUP);
rtc.writeSqwPinMode(DS3231_SquareWave1Hz);
attachInterrupt(digitalPinToInterrupt(SQW_PIN), tick, FALLING);

Use the interrupt routine to both keep track of the total second count (seconds elapsed since the Unix epoch) and record the micros() value of the last tick:

volatile uint32_t seconds_since_epoch;
volatile uint32_t micros_on_tick;
void tick() {
 seconds_since_epoch++;
 micros_on_tick = micros();
}

Now you can combine these two values to get a millisecond-resolved timestamp as follows:

uint64_t millis_since_epoch() {
 noInterrupts();
 uint32_t seconds = seconds_since_epoch;
 uint32_t microseconds = micros_on_tick;
 interrupts();
 uint32_t extra_micros = micros() - microseconds;
 uint16_t extra_millis = extra_micros / 1000;
 if (extra_millis > 999)
 extra_millis = 999;
 return (uint64_t) seconds * 1000 + extra_millis;
}

The value returned from this function is the 13-digit number you want, provided seconds_since_epoch has been sensibly initialized.

• Shan-Desai: won't I need the RTC to provide me the initial epoch time in seconds?
• Edgar Bonet: @Shan-Desai: Yes, that's the easiest way to initialize seconds_since_epoch.
• Shan-Desai: You sir are a Legend! Thank you for this solution. Works great. With a little function to make the uint64_t to String was needed but rest everything works like a charm.

Answer 2

the goal is to generate unique and ordered millis part of timestamps in one epoch second.

unsigned long thisNowMillis;
uint32_t thisNow;

-

 uint32_t now = rtc_now_time.unixtime();
 if (now != thisNow) {
 thisNow = now;
 thisNowMillis = millis();
 }
 Serial.print("millis: "); Serial.print(millis() - thisNowMillis);

not tested and it will not work for counting intervals between timestamps.

• Shan-Desai: will check this and get back to you