ASync Timer2 based OSCCAL value

Question

How to calibrate the internal RC oscillator off of a 32768 Hz crystal serving Timer2?

void OSCCAL_calibrate(void) //This version specific to ATmegaXX8 (tested with ATmega328)
{
 unsigned char calibrate = 0; //FALSE;
 uint16_t temp;
 TIMSK1 = 0; //disable Timer1,2 interrupts
 TIMSK2 = 0;
 ASSR = (1 << AS2); //select asynchronous operation of timer2 (32,768kHz)
 OCR2A = 200; // set timer2 compare value
 TCCR1A = 0;
 TCCR1B = (1 << CS11); // start timer1 with prescaler 8
 TCCR2A = 0;
 TCCR2B = (1 << CS20); // start timer2 with no prescaling (ATmega169 use TCCR2A!)
 while (ASSR & ((1 << TCN2UB) | (1 << TCR2BUB))); //wait for TCN2UB and TCR2BUB to be cleared
 delay(5000); //5 seconds to allow xtal osc to stabilize - does not work from main()
 while (!calibrate)
 {
 cli(); // disable global interrupts
 TIFR1 = 0xFF; // clear TIFR1 flags
 TIFR2 = 0xFF; // clear TIFR2 flags
 TCNT2 = 0; // clear timer2 counter
 while (ASSR & ((1 << TCN2UB) | (1 << TCR2BUB))); //wait for TCN2UB and TCR2BUB to be cleared
 TCNT1 = 0; // clear timer1 counter
 
 while ( !(TIFR2 & (1 << OCF2A)) ); // wait for timer2 compareflag
 TCCR1B = 0; // stop timer1
 sei(); // enable global interrupts
 if ( (TIFR1 & (1 << TOV1)) ) temp = 0xFFFF; //overflow, load max
 else temp = TCNT1;
 
 if (temp > 6150) //expect about (1e6/32768)*201 = 6134 ticks
 {
 OSCCAL--; //RC oscillator runs too fast, decrease OSCCAL
 }
 else if (temp < 6120)
 {
 OSCCAL++; //RC oscillator runs too slow, increase OSCCAL
 }
 else calibrate = 1; //done
 TCCR1B = (1 << CS11); // (re)start timer1
 } //end while(!calibrate)
} //return 
void setup(void) 
{
 EEPROM.put(0x1FD, 0xFFFF);
 OSCCAL_calibrate();
 EEPROM.put(0x1FD, OSCCAL);
}
void loop() { }

I write my value to the EEPROM then use it in another program; it causes all the communications not to work anymore (after I have set all my constants to the expected values for a ~ 8MHz clock). This is not for an Arduino but for an Atmega88PB - could not find anything special about it in the datasheet. FWIW, factory value = 0x96, algorithm value = 0x61.

---

edit Test setup and code:

First Board = signal generator:

ATMEGA328P with 8 MHz, +/- 30 ppm crystal, (room temperature)

Fuses: https://eleccelerator.com/fusecalc/fusecalc.php?chip=atmega328p&LOW=FF&HIGH=DE&EXTENDED=05&LOCKBIT=FF

void setup() {
 // put your setup code here, to run once:
 TCCR2A = TCCR2B = 0;
 
/* Timer Clock = 8 MHz / 1 */
TCCR2B = (0<<CS22)|(0<<CS21)|(1<<CS20); // no prescales, counts 256 times for 32 useconds
/* Clear pending interrupts */
TIFR2 = (1 << TOV2) | (1 << OCF2A) | (1 << OCF2B);
TIMSK2 = (1 << TOIE2); // enable ovf interrupt
 
 DDRB |= (1 << PB1); // pinMode(PIN_CS, OUTPUT);
 PORTB |= (1 << PB1);
 
}
 
 
ISR (TIMER2_OVF_vect) // need it to catch our overflows
{
 PORTB ^= (1 << PB1); //keep the radio chip asleep
}
 
void loop() {
 // put your main code here, to run repeatedly:
 
}

Second Board = signal counter DUT:

ATMEGA168PB with 32768 Hz, +/- 20 ppm crystal, (room temperature)

Fuses: https://eleccelerator.com/fusecalc/fusecalc.php?chip=atmega168p&LOW=D2&HIGH=D6&EXTENDED=05&LOCKBIT=FF

Counting through clk_i/o based Timer2

#include <EEPROM.h>
 
 #undef F_CPU
 #define F_CPU 8000000L
 
 #define PIN_WUP PD2
 
 
 volatile bool fWUP = false;
 volatile bool fOvfT2 = false; 
 volatile unsigned long cntWUP = 0;
 
 ISR (PCINT2_vect) // handle Atmega328P precise signal 
 { 
 if (PIND & (1 << PIN_WUP)) 
 { 
 fWUP = false;
 
 } else {
 
 fWUP = true;
 cntWUP++;
 TIMSK2 = (1 << TOIE2); // enable ovf interrupt > } 
 }
 
 ISR (TIMER2_OVF_vect) // need it to count our overflows
 { fOvfT2 = true; }
 
 
 void setup() {
 EEPROM.put(0x1E6, 0xffff);
 OSCCAL = 0x61;
 
 PCMSK2 |= (1<<PCINT18); // aka PD2, PIN_WUP
 PCICR |= (1<<PCIE2);
 PCIFR |= (1<<PCIF2); 
 
 DDRD &= ~(1 << PIN_WUP);
 PORTD &= ~(1 << PIN_WUP); // disable pullup
 
 TCCR2A = TCCR2B = 0;
 TCNT2 = 0;
 TCCR2B = (0<<CS22)|(1<<CS21)|(1<<CS20); // 1/32 prescales, count at 1/4 MHz
 }
 
 void loop() { // put your main code here, to run repeatedly: 
 static unsigned long cntt2 = 0;
 if (fOvfT2) 
 {
 fOvfT2 = false;
 cntt2++; 
 }
 
 if (100000 == cntWUP) // 6.4 seconds, precise from Atmega328P 
 {
 EEPROM.put(0x1E6, (cntt2 << 8) + TCNT2); // let's see how many ticks there have been
 while (1) { } ;
 } 
 }

The results are consistent across multiple runs:

• running the codes above: eeprom value = 1,249,608; expected ticks = 1.6M => ratio = 0.781005

• running with the OSCAL = 0x61 line commented out: eeprom value = 1,596,554; expected ticks = 1.6M => ratio = 0.99784625

• timemage: 0x61 (hex) is 97 (decimal), nearly 96 decimal ; that may be a coincidence, but confirm that you're using consistent bases.
• timemage: I don't see any reason to think the problem is here and not in the code you haven't shown or even potentially during programming. If you show what is required to replicate everything I would try it.
• KIIV: Side note: Newer AVR Dx series MCUs have this feature. You only need to start 32k xtal and enable Auto-Tuning
• kellogs: @timemage I shall be back with some test code
• timemage: I haven't been able to reliably replicate this in a while, I think because of temperature change here. But, it seemed like the calibration would tuneOSCCAL incorrectly sometimes but if the calibration was repeated immediately it would turn out a valuable that was slightly better than the factory one. At some point I may return to it. About the only thing I have offer now is that ASSR has a flag for synchronizing OCR2A which you aren't using. I don't have specific reason to think that's the cause though.