How to get Arduino to read serial from Python?

Question

Specific problem

I am trying to communicate between a Python script and my Arduino Pro Mini using serial with a USB cable. In Python I can read what the Arduino writes but the Arduino can either not read what the Python writes or it is reading it in a form that I can't process. The most obvious way this manifests itself is when I try to write what the Arduino has just read from Python, to Python. Nothing gets displayed in Python.

My Arduino sketch appears to work just fine when I work with it in the Serial Monitor, i.e. it can read and write so I suspect it is something at the Python end. What's causing this?

Details

I want to be able test how quickly I can send 64 byte-long pieces of information to my Arduino. To do this I wrote a Python script that could should be able to periodically send various information to my Arduino. I can change the period and thus test what kind of speed I can get.

However, despite the fact that my Arduino sketch works just fine with Serial Monitor, i.e. I can send it information and read information from it, I can't get it to work with Python.

With my Python script I can open the connection and read what the Arduino writes, but for some reason there is a problem with writing from Python. I have tried the following but none of them have worked:

• Using an Uno instead of a Pro Mini.
• Putting a Serial.println() statement in my get_data() function. Nothing gets printed.
• Sending a string with the format "T(number);" and using sscanf() to extract the number. Using an if statement to examine the number and turn an onboard LED on. Did not work.
• Sending a string beginning with "T" and using an if statement to see if it is stored in raw_data[]. It isn't.
• Writing the data from Python at different times (i.e. using a longer delay).
• Writing from Python using device.write(b"mystring") and device.write("mystring".encode())

I'm at a loss for what to try next...

Python (2.7) code

import serial, time, random, string
device = serial.Serial('/dev/tty.usbserial-A105YOZX', baudrate = 38400)
initial_time = time.time()
current_time = time.time()
counter = 0
control = 0
while counter < 1000:
 current_time = time.time()
 if current_time - initial_time > 0.1:
 initial_time = current_time
 counter += 1
 device.read()
 if current_time - initial_time > 1 and control == 0:
 device.write(b"Message being sent")
 control = 1

Arduino code

#define RAW_LEN 64 //max length of incoming string (buffer size is only 64 bytes)
//variables for incoming data
char raw_data[RAW_LEN];
int raw_data_i = 0; //index
unsigned long my_time;
int control = 0;
int for_print;
int get_data(int i) {
 //fills up raw_data[] with serial data. Clears raw_data if it gets too long
 if (Serial.available()>0) {
 raw_data[i] = Serial.read();
 Serial.println(raw_data[i]);
 //Serial.println(raw_data[i]);
 if (i<RAW_LEN-1){
 i++;
 }
 }
 return i;
}
void clear_data() {
 //clears raw_data and resets raw_data_i
 raw_data_i = 0;
 memset(raw_data,'0円',RAW_LEN-1);
}
//print functions
void raw_print(char data[]) {
 Serial.println("Received data: ");
 //if (raw_data[0] == 'T') {digitalWrite(LED_BUILTIN, HIGH);}
 for (int i=0; i< RAW_LEN;i++) {
 Serial.println(data[i]);
 }
}
void setup() {
 Serial.begin(38400);
 Serial.println("Restart");
 pinMode(LED_BUILTIN, OUTPUT);
 digitalWrite(LED_BUILTIN, LOW);
}
void loop() {
 my_time = millis();
 raw_data_i = get_data(raw_data_i);
 //clear raw_data if it gets full
 if (raw_data_i == RAW_LEN-1) {
 clear_data();
 }
 if (my_time > 5000 && control == 0) {
 Serial.println("Here");
 raw_print(raw_data);
 control = 1;
 }
}

• user2261062: raw_data is a char vector, so when reading from Serial cast it to a char: raw_data[i] = (char)Serial.read()
• chkrs888.: Thank you for the suggestion. I tried it and it still works with Serial Monitor but not with Python...
• gre_gor: What is device.read() supposed to do?
• chkrs888.: I'm printing the data the Arduino has received back to the serial port, so the device.read() should read it and print it. I made a mistake in the code I copied across – it should be print device.read().
• tony gil: flagged for moving to arduino specific community. tks, tg

Answer 1

The problem turned out to be in the Python code. In while loop the first if statement is True after 0.1s, and then executes 9 more times before the second if statement is True.

The problem with this is that device.read() just hangs if it does not receive a character. Because I wasn't sending enough characters from the Arduino the Python code paused at the device.read() and never got to the second if statement.

An annoying mistake that took a while to figure out, hopefully this question and answer will save someone else a lot of debugging time.