Trouble programming custom ATMEGA2560 board

Question

I need some help. I copied the heart of MEGA2560 design into a new project. I have the new PCBs with blank MCUs and now I need to get things programmed. (I have an existing project that runs on a MEGA2560)

I was able to get the MEGA16U2 programmed using the procedure here: https://www.arduino.cc/en/Hacking/DFUProgramming8U2, (I loaded "Arduino-usbserial-atmega16u2-Mega2560-Rev3.hex") and it shows up in the Arduino IDE.

I got the ATMEGA2560 bootloader loaded using the Arduino-as-ISP instructions here: https://www.arduino.cc/en/Tutorial/ArduinoISP

But, now when I try to program my board with the with Arduino IDE (through the USB connection of the MEGA16U2), I just get:

avrdude: stk500v2_ReceiveMessage(): timeout

The full output of the IDE is:

Sketch uses 30056 bytes (11%) of program storage space. Maximum is 253952 bytes.
Global variables use 1280 bytes (15%) of dynamic memory, leaving 6912 bytes for local variables. Maximum is 8192 bytes.
/home/xxxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17/bin/avrdude -C/home/xxxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17/etc/avrdude.conf -v -patmega2560 -cwiring -P/dev/ttyACM0 -b115200 -D -Uflash:w:/tmp/arduino_build_464802/yyyyy.ino.hex:i 
avrdude: Version 6.3-20190619
 Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
 Copyright (c) 2007-2014 Joerg Wunsch
 System wide configuration file is "/home/xxxxx/.arduino15/packages/arduino/tools/avrdude/6.3.0-arduino17/etc/avrdude.conf"
 User configuration file is "/home/xxxxx/.avrduderc"
 User configuration file does not exist or is not a regular file, skipping
 Using Port : /dev/ttyACM0
 Using Programmer : wiring
 Overriding Baud Rate : 115200
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_ReceiveMessage(): timeout
avrdude: stk500v2_getsync(): timeout communicating with programmer
avrdude done. Thank you.
An error occurred while uploading the sketch

I can see the RX led of the MEGA16U2 flashing, and the data at the RX pin of the ATMEGA2560 IC, indicating that data is getting to the MCU from the IDE, but there is no corresponding TX activity.

The IDE and USB cable can program an Elegoo MEGA2560 clone with no problem.

Is this a baud rate problem? A protocol problem? What should be my next debugging step?

Thanks!

---

EDIT: Per Nick Gammon's comment, I have run his chip detector script. Here is the output:

Atmega chip detector.
Written by Nick Gammon.
Version 1.20
Compiled on Sep 5 2020 at 07:17:55 with Arduino IDE 10812.
Attempting to enter ICSP programming mode ...
Entered programming mode OK.
Signature = 0x1E 0x98 0x01 
Processor = ATmega2560
Flash memory size = 262144 bytes.
LFuse = 0xFF 
HFuse = 0xD8 
EFuse = 0xFD 
Lock byte = 0xCF 
Clock calibration = 0x88 
Bootloader in use: Yes
EEPROM preserved through erase: No
Watchdog timer always on: No
Bootloader is 8192 bytes starting at 3E000
Bootloader:
3E000: 0x0D 0x94 0x89 0xF1 0x0D 0x94 0xB2 0xF1 0x0D 0x94 0xB2 0xF1 0x0D 0x94 0xB2 0xF1 
<....snipped for length....>
3FFF0: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
MD5 sum of bootloader = 0xD9 0xE6 0x6B 0x4E 0xD1 0xA6 0x11 0x2C 0x61 0x8F 0x9B 0xD5 0x5D 0x24 0xE2 0x13 
Bootloader MD5 sum not known.
First 256 bytes of program memory:
00: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
10: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
20: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
30: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
40: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
50: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
60: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
70: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
80: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
90: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
A0: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
B0: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
C0: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
D0: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
E0: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
F0: 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 0xFF 
Programming mode off.

• Nick Gammon: I assume you have a few Arduinos lying around. Can you please run my chip detector sketch and report the results? (Amend the question).
• Majenko: Do you have the right fuse settings on the 2560 for your clock?
• GMc: Have you built a 16Mhz external clock (e.g. a crystal oscillator)? When I tried something similar with a ATMega-328P (home design based upon an UNO) , I noticed that when I first uploaded my program it ran super slow (because the default fuses are to run on the internal RC oscillator / 8 = 1 MHz). Reloading the bootloader onto my design resulted in it running at normal speed - because it seems that this activity reset the fuses to use an external clock. Check out this post for reading and writing the fuses instructables.com/id/…
• Juraj: does the auto-reset circuit work? try a manual reset while avrdude attempts the upload
• gearhead: @NickGammon, I ran your program. See updated question. Thanks!

Answer 1

As per the comments your output shows that the chip is working "normally" in that it reacts to the chip detector sketch. Thus the bulk of the design/implementation is OK.

Referring to Engbedded Atmel AVR® Fuse Calculator your fuse settings means the processor is expecting an external crystal oscillator. The bootloader expects (I think) a 16 MHz crystal (with appropriate capacitors). If you have not provided this then the bootloader will be running at the wrong baud rate (so to speak) — or not at all — and not react to attempts to download it by serial programming.

If you have a different frequency crystal then you may be able to work around this by adjusting the baud rate in the boards.txt file by an appropriate scaling factor.

The chip detector sketch worked because that uses ICSP programming, which provides its own clock.

• gearhead: > Thus the bulk of the design/implementation is OK. Great news! I do indeed have a 16 MHz crystal on the board, and it looks like this oscillating correctly. I'll look into the baud rate issue. Thanks again for your help, Nick!
• gearhead: Nick, apologies for the messages. I have played around with various baud rates (via the boards.txt file), and nothing is working. In the output of your program, also says ''' Bootloader MD5 sum not known. ''' Is that anything to be concerned with? I used the procedure outlined here: arduino.cc/en/Tutorial/ArduinoISP
• Nick Gammon: That sketch, written quite a while ago, stored some MD5 sums of known bootloaders (at the time). There may well have been updates released since then. So that is no particular concern.
• Nick Gammon: I suggest you get yourself a cheap logic analyser (say, around 100ドル). Then you can see what is going on when the sketch starts to upload.
• Nick Gammon: I am not doing private consulting at present (paid or unpaid). My support for the Arduino community is through Stack Exchange, plus the (read only) articles on my web site. To problem-solve this issue you might be better off with the Arduino Forum - perhaps the Microcontrollers section. That might be more suitable for the "have you tried this? have you tried that?" style of working on your issue.