Turn ESP8266 into an Arduino IDE-like "sketch uploader" (over hardware serial)

Question

I have an ESP8266 module which has downloaded a hex file for an Arduino Mega2560. This hex file (call it mega2560.hex) is sitting in the ESP's SPIFFS file system. What I want to do is basically emulate the behavior of the Arduino IDE when we press the upload button on a sketch (except in my case here, the sketch is already compiled and ready for transfer). What I need is some library or small code example that can steer me in the right direction. On the Mega's end, the processor will be restarted, and the bootloader will wait for the hex file, receive it from the ESP and flash it to application memory.

Here is the basic structure:

// Code for ESP8266
 
SoftwareSerial mega(D1, D2); // since hardware serial is occupied for serial monitor
// Mega will use its hardware serial (Tx1, Rx1)
void setup()
{
 Serial.begin(115200); // use up hardware serial
 mega.begin(115200); // connect to mega
 
 SPIFFS.begin();
 File hexfile = SPIFFS.open("mega2560.hex", "r");
 
 // IDE-like "upload"
 upload(mega, hexfile);
 Serial.println("Done uploading.");
 hexfile.close();
}
// emulate Arduino IDE-like upload behavior
void upload(SoftwareSerial& mega, File& hexfile)
{
 // reset Mega to start bootloader which will first wait to accept new hex file
 reset(mega);
 // send out the hex file for the bootloader to accept and store to Mega's flash
 send(mega, hexfile);
 // reset Mega to start bootloader again, this time so that previously uploaded hex file can be run
 reset(mega);
}

I am fairly new to working with bootloaders and would like to know if the above top-down code is correct in trying to achieve what I have in mind. Can I simply reset the Mega and transfer it as per usual via Serial within the bootloader's "waiting window" ? or are there more considerations that I am missing.

I am trying to avoid using ISP or SPI because that would erase the Mega's bootloader as well and I want to keep the bootloader preserved.

• Fahad: Yes, you are thinking in the right direction. You can simply reset the mega to get the mega bootloader started, but you can not just send the bytes through the serial port. Arduino bootloader uses STK500 commands for reliable handshaking. AVRdude uses the same protocol to talk to an Arduino. Here is a link that might give you some direction: electronicwings.com/arduino/…

Answer 1

You requirement to program the AVR over Serial from esp8266 SPIFFS is very specific. You could use the library called "dfu". I have its Arduino version here. It was made by arduino.org for the first Uno WiFi's esp8266 WiFiLink firmware. Here is the usage example.

With ISP/SPI wiring you can use the ESP8266AVRISP library. It is bundled with the esp8266 Arduino core. It has an example.

Over Serial but without SPIFFS storage, an esp8266 can also be used as direct network proxy for avrdude, where avrdude works over a network port communicating with a simple network to Serial sketch running on the esp8266.

Also an Arduino Mega can do a self update with my ArduinoOTA library and among other options for the transport of the updated binary the ArduinoOTA library can use my WiFiEspAT library which works with an ESP8266 or ESP32 with AT firmware.

Answer 2

I use the esp-link firmware on the ESP8266, when placing the Arduino Uno bootloader on any board with ATmega328, the Arduino IDE can program the Arduino with the 'Arduino Uno Wifi' version (It's usually at the bottom of the list of AVR boards).

The esp-link firmware has the mDNS service already pre-configured for 'arduino', leaving this service enabled, the Arduino IDE can find the IP address. This es-link firmware operates as a router, and needs to be configured to access the router that the Arduino IDE computer is accessing.

The esp-link firmware interface is quite intuitive, and the connection to the microcontroller only requires RX, TX and Reset, as well as a programmer via serial port (or USB-Serial converter such as FTDI232).

The esp-link firmware connects a micro-controller to the internet using an ESP8266 Wifi module

enter image description here

See here: https://github.com/jeelabs/esp-link

• Juraj: there is a problem to make it work for Arduino Mega
• R R: It might be interesting to see the issues in the esp-link repository, instead of downvoting third-party responses.
• Juraj: the downvote is not related with the comment.