EEPROM programmer works with serial monitor but not with pySerial

Question

I'm trying to modify Ben Eater's eeprom programmer to flash a 32K eeprom, since the arduino has 32K of program data total the data wouldn't fit in the program. So I wrote a python program using pySerial to send the data in chunks of 256 bytes over serial. The arduino program works correctly when I manually send the data over the serial monitor, and the python program works correctly when I print the values to a console as a test, but when I actually run the program over serial it seems to only write zeros to the eeprom. Additionally, the Rx led on the arduino seems to be lighting up with the correct data, but like I said only zeros are written.

Here's the relevant arduino code:

void setup() {
 pinMode(SHIFT_DATA, OUTPUT);
 pinMode(SHIFT_CLK, OUTPUT);
 pinMode(SHIFT_LATCH, OUTPUT);
 digitalWrite(WRITE_EN, HIGH);
 pinMode(WRITE_EN, OUTPUT);
 
 Serial.begin(57600);
 dataArrPage = 0;
 
 //delay(1000);
 //Serial.println("Reading EEPROM");
 //printContents();
}
void loop() {
 if (Serial.available() > 0) {
 int len = Serial.readBytes(dataArr, 256);
 for (int i = 0; i < 256; i++) {
 writeEEPROM(dataArrPage + i, dataArr[i]); // writeEEPROM(address, data)
 }
 dataArrPage += 0x100;
 Serial.write('s'); //success, writing is done
 }
}

And the relevant python code:

import time
import serial
ser = serial.Serial('/dev/ttyUSB0', 57600)
time.sleep(4) # wait for arduino to initialize
filename = input("Enter file name: ")
f = open(filename, "rb")
data = f.read()
page = 0
for x in range(128): # 128 pages in 32K eeprom
 for i in range(256):
 #print(hex(data[(page * 256) + i])) # printing prints the correct values
 ser.write(data[(page * 256) + i])
 page += 1
 
 #wait until receive 's' from arduino
 while True:
 if ser.in_waiting > 0:
 suc = ser.read(size=1)
 if suc == b's':
 print("Success")
 break
print("Writing done")
f.close()
ser.close()

I'm a beginner so sorry if it's something obvious.

• Majenko: And if you wait for the Arduino to reboot and pass the bootloader after you open the serial port?
• bobbbob: I had a delay after opening the serial port in the python code but it didn't make a difference, entering the file name takes a bit anyways. I should probably add it back in.
• Majenko: You should consider a two-way protocol, so Python sends something then the Arduino responds once that operation is done with a status. That way you know what is happening from both ends.
• bobbbob: Okay, I'll try that.
• bobbbob: I edited the code to print "Success" when the arduino is done writing the 256 bytes, however it only prints success 3 times then freezes. Would the contents of the file matter at all? It's a 32K file that's mostly zeros after some text at the beginning.

Answer 1

I figured out the problem. The for loop was writing one byte at a time, which probably wasn't fast enough and caused a timeout in the arudino. The code is now:

page = 0
for x in range(128): # 128 pages in 32K eeprom
 datalst = []
 for i in range(256):
 datalst.append(data[(page * 256) + i])
 
 ser.write(datalst)
 #wait until receive 's' from arduino
 while True:
 if ser.in_waiting > 0:
 suc = ser.read(size=1)
 if suc == b's':
 print("Success")
 break
 
 page += 1

I also changed the data types in the arduino sketch to use the ones in stdint.h which solved an unrelated bug. Let me know if I should post the full code for both.

• the busybee: Hm, strange. Since your line runs with 57600 baud, it transfers just some 6000 bytes per second. I would think that the Python loop runs faster. -- However, if the USB-based serial interface connects directly to a USB module inside the AVR, the baudrate is not maintained.