Unable to read Arduino+ESP8266 full get Response

Question

Using below code I am able to invoke HTTP Get request, but I am not able to read full HTTP Response. Observing only output like( +IPD,0,141:HTTP/1.7. ). Please let me know how can I read full response? Any Idea what is the issue?

#include <SoftwareSerial.h>
#define RX 2
#define TX 3
String WIFI_SSID = "xyz"; // Your WiFi ssid
String PASSWORD = "xys"; // Password
String HOST = "192.168.0.156";
String PATH = "/test";
String PORT = "8080";
int countTrueCommand;
int countTimeCommand;
boolean found = false;
SoftwareSerial esp8266(RX, TX);
void setup() {
 Serial.begin(9600);
 esp8266.begin(9600);
 esp8266.println("AT");
 Serial.println(esp8266.read());
 sendCommandToESP8266("AT", 5, "OK");
 sendCommandToESP8266("AT+CWMODE=1", 5, "OK");
 sendCommandToESP8266("AT+CWJAP=\"" + WIFI_SSID + "\",\"" + PASSWORD + "\"", 20, "OK");
}
void loop() {
 String postRequest = "GET " + PATH + " HTTP/1.1\r\n" +
 "Host: " + HOST + "\r\n" +
 "Accept: *" + "/" + "*\r\n" +
 "\r\n";
 sendCommandToESP8266("AT+CIPMUX=1", 5, "OK");
 sendCommandToESP8266("AT+CIPSTART=0,\"TCP\",\"" + HOST + "\"," + PORT, 15, "OK");
 String cipSend = "AT+CIPSEND=0," + String(postRequest.length());
 sendCommandToESP8266(cipSend, 4, ">");
 sendData(postRequest);
 readESP8266(8000);
 sendCommandToESP8266("AT+CIPCLOSE=0", 5, "OK");
}
void sendCommandToESP8266(String command, int maxTime, char readReplay[]) {
 Serial.print(countTrueCommand);
 Serial.print(". at command => ");
 Serial.print(command);
 Serial.print(" ");
 while (countTimeCommand < (maxTime * 1))
 {
 esp8266.println(command);
 if (esp8266.find(readReplay))
 {
 found = true;
 break;
 }
 countTimeCommand++;
 }
 if (found == true)
 {
 Serial.println("Success");
 countTrueCommand++;
 countTimeCommand = 0;
 }
 if (found == false)
 {
 Serial.println("Fail");
 countTrueCommand = 0;
 countTimeCommand = 0;
 }
 found = false;
}
void sendData(String postRequest) {
 Serial.println(postRequest);
 esp8266.println(postRequest);
 delay(1500);
 countTrueCommand++;
}
void readESP8266(const int timeout)
{
 String reponse = "";
 long int time = millis();
 while( (time+timeout) > millis())
 {
 while(esp8266.available())
 {
 reponse+=esp8266.readString();
 }
 }
 Serial.print(reponse); 
}

Output:

Success
1. at command => AT+CWMODE=1 Success
2. at command => AT+CWJAP="xyz","xys" Success
3. at command => AT+CIPMUX=1 Success
4. at command => AT+CIPSTART=0,"TCP","192.168.0.156",8080 Success
5. at command => AT+CIPSEND=0,56 Success
GET /test HTTP/1.1
Host: 192.168.0.156
Accept: */*
 
busy s...
Recv 56 bytes
SEND OK
+IPD,0,141:HTTP/1.7. at command => AT+CIPCLOSE=0 Success

• hcheung: +IPD,0,141 means there are 141 bytes data return from the response, your code simply closed too early without read and parsing the data.
• Rajesh: But I have already added delay of 8 sec, shall I add more delay?? readESP8266(8000); If you can provide any code that will be helpful
• hcheung: It is not about the delay (actually 8s is way too long), it is about continue to read the data, sorry I don't have time to write the code for you, but you can read one of my library on reading and parsing the response.
• Rajesh: @hcheung Thanks for your help, I was able to perform HTTP GET request using LiteESP8266Client.h but 1 issue I am facing is after few call probably Arduino memory is becoming full, any idea how we can fix that

Answer 1

Finally the HTTP Get request with Arduino Uno + ESP8266 Wifi shield which is working is below:

// This is a demo example using a public http server for testing both GET and POST requests

#include <Arduino.h>
#include <LiteESP8266Client.h>
#define PACKET_MTU 1500 // Standard network MTU is 1500 bytes
LiteESP8266 radio;
const char ssid[] PROGMEM = "xyz"; //change it to your wifi SSID
const char password[] PROGMEM = "abc"; //change it to your wifi password
const char host[] PROGMEM = "192.168.0.156";
const int port = 8080;
const char http_get_request[] PROGMEM = "GET /test HTTP/1.1\r\n";
const char http_useragent[] PROGMEM = "User-Agent: Arduino-stm32/0.1\r\n";
const char http_content_type_json[] PROGMEM = "Content-Type: application/json\r\n";
const char http_host[] PROGMEM = "Host: 192.168.0.156\r\n";
const char http_close_connection[] PROGMEM = "Connection: close\r\n\r\n";
const char http_content_length_header[] PROGMEM = "Content-Length: ";
const char success[] PROGMEM = "success";
const char failed[] PROGMEM = "failed";
const char CRLF[] PROGMEM = "\r\n";
const char error_data_null[] PROGMEM = "Error: data came back null.";
void setupStationMode() {
 Serial.print("Setup station mode... ");
 if (radio.set_station_mode()) {
 Serial.println("success");
 } else {
 Serial.println("failed");
 }
}
void joinAP() {
 Serial.print("Join AP ");
 Serial.print(ssid);
 Serial.print("... ");
 if (radio.connect_to_ap(ssid, password)) {
 Serial.println("Success");
 } else {
 Serial.println("Failed");
 }
}
void establishTcpConnect() {
 Serial.print("Establish TCP Connection... ");
 if (radio.connect_progmem(host, port)) {
 Serial.println( "Success");
 } else {
 Serial.println( "Failed");
 }
}
void getHttpPacket() {
 char *data;
 while ((data = radio.get_response_packet(PACKET_MTU, 5000))) {
 if (data) {
 Serial.println("Packet Received...");
 Serial.println(data);
 } else {
 Serial.println(error_data_null);
 }
 }
 free(data);
}
void httpGet() {
 Serial.println("Sending GET request... ");
 radio.send_progmem(http_get_request);
 radio.send_progmem(http_useragent);
 radio.send_progmem(http_host);
 radio.send_progmem(http_close_connection);
}
void setup() {
 delay(2000);
 radio.begin(9600,2,3);
 Serial.begin(9600);
 while (!Serial) {};
 setupStationMode();
 joinAP();
}
void loop() {
 establishTcpConnect();
 httpGet();
 getHttpPacket();
 }

Output:

Setup station mode... success
Join AP ⸮... Success
Establish TCP Connection... Success
Sending GET request... 
Packet Received...
HTTP/1.1 200 
Content-Type: text/plain;charset=UTF-8
Content-Length: 27
Date: 2020年8月07日 13:04:03 GMT
Connection: close
Greetings from Spring Boot!

But one issue I am facing is after few call the HTTP invocation is getting stopped probably memory being full. Will check this.

• hcheung: If you run the example on the httpbin.org test site, do you face the same problem?
• hcheung: BTW, don't post your new problem as an "answer", since your code is deviated and totally different from your original question, you should post it as a new question.
• hcheung: The problem could be coming from your server at 192.168.0.156/test, so you should run your sketch on httpbin.org server to make the problem is not caused by this client sketch first.
• Rajesh: Posted new Question: arduino.stackexchange.com/questions/77353/… . Though output of 192.168.0.156/test is quite small
• Rajesh: @hcheung tried with other REST server as suggested and same issue is observed: