HTTP Post error using ESP8266 from Arduino

Question

Using Arduino on an STM32F103C Blue Pill to drive an ESP8266 ESP-01 as shown below, and this tutorial: Arduino Forum

Arduino Forum

I am attempting to do an HTTP POST with the following code:

String ssid = "MySSID";
String password = "MyPassword";
String server = "192.168.1.134"; // IP hosting API (Flask w/ Python)
String uri = "/api/create/current/"; // API Route
String payload = "?outlet_id=1&voltage=2&vrms=3&amps_rms=4"; // Sample sensor payload
void httppost() {
 Serial2.println("AT+CIPSTART=\"TCP\",\"" + server + "\",80"); //start a TCP connection.
 if (Serial2.find("OK")) {
 Serial.println("TCP connection ready");
 }
 delay(1000);
 String postRequest =
 "POST " + uri + " HTTP/1.0\r\n" +
 "Host: " + server + "\r\n" +
 "Accept: *" + "/" + "*\r\n" +
 "Content-Length: " + payload.length() + "\r\n" +
 "Content-Type: application/json\r\n" +
 "\r\n" + payload;
 String sendCmd = "AT+CIPSEND="; //determine the number of characters to be sent.
 Serial2.print(sendCmd);
 Serial2.println(postRequest.length());
 delay(5000);
 if (Serial2.find(">")) {
 Serial.println("Sending..");
 Serial2.print(postRequest);
 if (Serial2.find("SEND OK")) {
 Serial.println("Packet sent");
 while (Serial2.available()) {
 String tmpResp = Serial2.readString();
 Serial.println(tmpResp);
 }
 // close the connection
 Serial2.println("AT+CIPCLOSE");
 }
 }

The post request is failing on the server-side with the following error:

Unexpected Error: malformed JSON in post request, check key/value pair at:

Nothing is else contained the error message or Flask server logs to indicate how/why the JSON is malformed. Using Postman, I'm able to perform a successful post with the following:

192.168.1.134/api/create/current/?outlet_id=1&voltage=2&vrms=3&amps_rms=4

Does anyone see anything obvious? FWIW, using the Blue Pill to power the ESP8266, or alternatively using an external power source produce the same results. It seems to be a syntactic problem but I'm not seeing it. Any help is much appreciated!

• jsotola: post to this site and check to see what is being sent ... beinghttp://leserged.online.fr/phpinfo.php
• Chris Stratton: You shouldn't need to guess. Modify your server to give you a debug message of the actual post body, packet sniff at the server, point it at a copy of netcat in listen mode instead if the server or modify your embedded code to log or snoop with another USB UART or cheapie logic analyzer.

Answer 1

You are mixing up two different ways of sending information to a server.

When you use postman with the URL

192.168.1.134/api/create/current/?outlet_id=1&voltage=2&vrms=3&amps_rms=4

You are not actually sending any useful information in the body, but rather sending all of the data as URL parameters.

In contrast, your embedded code:

String uri = "/api/create/current/"; // API Route
String payload = "?outlet_id=1&voltage=2&vrms=3&amps_rms=4"; // Sample sensor payload

attempts to supply the "payload" in the message body.

But the problem is that your payload is not encoded as json, but rather still encoded as a series of URL parameters.

You should either add that "payload" string onto the end of the URL and not send it in the body.

Or else, if your server supports it, encode the payload as proper json, probably something like:

{"outlet_id": 1, "voltage": 2, "vrms": 3, "amps_rms": 4}

Note that if you want to put this as a constant into a C/C++ program, you'll have to escape the quotes, for (brief) example:

String payload = "{\"outlet_id\": 1}";

Also should you have any json values beyond numbers and true/false/null those would need their own quotes, eg

{"mode": "test"}

Ultimately this is an everyday RESTful API question more suited to Stackoverflow and not an embedded or EE question at all, though needing to be able to work with these kinds of things isn't uncommon in the age of IoT.

• Codo: Fully agree. In addition: Content type should not be set to "application/json" if no JSON is sent.
• rjkunde: Thanks for the input, all. Wasn't sure where best to put this question, but I figured those more versed in STM32 might have better insight. I'm currently attempting option 1) add that "payload" string onto the end of the URL, and if I'm unable to get this to work, I'll try formatting JSON (with escaped characters) inside my embedded code instead.
• Chris Stratton: It's really not about the STM32 in any way at all
• rjkunde: @ChrisStratton, should I remove this post? I found found a solution that others may find useful, but don't want to pollute the EE S.E.
• rjkunde: Since there hasn't been a response, I'm going to go ahead and submit my answer which contains the solution I used. I agree, not EE specific, but does involve a particular combination (STM32 + ESP8266) which has limited documentation / examples on the internet. People may find this useful, and I don't want to cross-post.

Answer 2

Solution: Use ArduinoJson library to properly form JSON within C environment.

//
// ESP8266 HTTP Post Program
//
// Uses an ESP8266 ESP-01, connected to an STM32 Blue Pill
//
// Must have 128k flash variant of the STM32!
//
// Pins
// STM32 pin PA2 Serial 2 (RX) to ESP8266 TX
// Arduino pin PA3 Serial 2 to voltage divider then to ESP8266 RX
// Connect GND from the STM32 to GND on the ESP8266
// Connect 3.3V from the STM32 to VCC on the ESP8266
// Pull ESP8266 CH_PD HIGH via jumper from ESP8266 3.3V line
//
// Original code credit / inspiration:
// https://community.wia.io/d/25-how-to-setup-an-arduino-uno-with-esp8266-and-publish-an-event-to-wia
// Alan - WIA community admin
//
#include <ArduinoJson.h> // Must use library version <= 5.13.4, 6.x.x is incompatible. Handles JSON formatting
String basic_auth = "bmVvOm1hdHJpeA=="; // HTTP basic authentication username/pw string: neo/matrix
String wifi_ssid = "Your_Wireless_SSID"; // Wifi network SSID
String wifi_password = "YOUR_WIRELESS_PASSWORD"; // Wifi password
String host = "YOUR_SERVER_IP"; // Server hostname or IP
String path = "/api/create/current/"; // API Route
String port = "80"; // HTTP Port
int outletId = 1; // Unique, hard-coded ID for each outlet.
int loopDelay = 5000; // 5 second delay between sensor reads
int countTrueCommand; // Used in determining success or failure of serial commands
int countTimeCommand; // Used in determining success or failure of serial commands
boolean found = false; // Used in determining success or failure of serial commands
// Buffer to store JSON object
StaticJsonBuffer<200> jsonBuffer;
JsonObject& root = jsonBuffer.createObject();
void setup() {
 Serial.begin(9600); // Start STM32 Serial at 9600 Baud
 Serial2.begin(9600); // Start ESP8266 Serial at 9600 Baud
 Serial2.println("AT"); // Poke the ESP8266 
 Serial.println(Serial2.read()); // Check that AT firmware on ESP8266 is responding
 sendCommandToSerial2("AT", 5, "OK"); // If status is okay, set radio mode
 sendCommandToSerial2("AT+CWMODE=1", 5, "OK"); // Set radio mode
 sendCommandToSerial2("AT+CWJAP=\"" + wifi_ssid + "\",\"" + wifi_password + "\"", 20, "OK"); // Connect to pre-defined wireless network
}
void loop() {
 // Delays sensor reads as desired
 // delay(loopDelay);
 // JSON Data - using ArduinoJson library object
 // TODO - Stubu in JSON arrays for remaining sensor values
 root["outlet_id"] = 1;
 root["voltage"] = 2;
 root["vrms"] = 3;
 root["amps_rms"] = 4;
 String data;
 root.printTo(data);
 // HTTP post request
 String postRequest = "POST " + path + " HTTP/1.1\r\n" +
 "Host: " + host + "\r\n" +
 "Accept: *" + "/" + "*\r\n" +
 "Content-Length: " + data.length() + "\r\n" +
 "Content-Type: application/json\r\n" +
 "Authorization: Basic " + basic_auth + "\r\n" +
 "\r\n" + data;
 // Send post request using AT Firmware
 sendCommandToSerial2("AT+CIPMUX=1", 5, "OK");
 sendCommandToSerial2("AT+CIPSTART=0,\"TCP\",\"" + host + "\"," + port, 15, "OK");
 String cipSend = "AT+CIPSEND=0," + String(postRequest.length());
 sendCommandToSerial2(cipSend, 4, ">");
 sendData(postRequest);
 sendCommandToSerial2("AT+CIPCLOSE=0", 5, "OK");
}
// Function to determine success / failure of serial commands send to/from ESP8266
void sendCommandToSerial2(String command, int maxTime, char readReplay[]) {
 Serial.print(countTrueCommand);
 Serial.print(". at command => ");
 Serial.print(command);
 Serial.print(" ");
 while (countTimeCommand < (maxTime * 1))
 {
 Serial2.println(command);
 if (Serial2.find(readReplay))
 {
 found = true;
 break;
 }
 countTimeCommand++;
 }
 if (found == true)
 {
 Serial.println("Success");
 countTrueCommand++;
 countTimeCommand = 0;
 }
 if (found == false)
 {
 Serial.println("Fail");
 countTrueCommand = 0;
 countTimeCommand = 0;
 }
 found = false;
}
// Send post request to Arduino and ESP8266
void sendData(String postRequest) {
 Serial.println(postRequest);
 Serial2.println(postRequest);
 delay(1500);
 countTrueCommand++;
}

Python/Flask API: app.py

#!/usr/bin/env python3
from flask import Flask, jsonify, abort, make_response, request, url_for
from flask_basicauth import BasicAuth
import time
import sys
import pandas as pd
from pandas.io import sql
import mysql.connector as mariadb
import pymysql
import logging
app = Flask(__name__)
# Setup Logging
logger = logging.getLogger('werkzeug')
handler = logging.FileHandler('api.log')
logger.addHandler(handler)
## Flask-BasicAuth
## As header for programmatic access use: "Authorization: Basic bmVvOm1hdHJpeA=="
app.config['BASIC_AUTH_USERNAME'] = 'neo'
app.config['BASIC_AUTH_PASSWORD'] = 'matrix'
basic_auth = BasicAuth(app)
## API Routes Type Description
##
## /api/alarms GET Returns all records from alarm table
## /api/alarms/id GET Returns single record from alarms table matching integer id
## /api/alarm_log GET Returns all records from alarm_log table
## /api/create/current/{JSON} POST Inserts one record (comprised of JSON body) into sensor_acc712_current table
## 404 handler
@app.errorhandler(404)
def not_found(error):
 return make_response(jsonify({'error': 'Not found'}), 404)
## API route to query alarms table, and return all results
@app.route('/api/alarms', methods=['GET'])
@basic_auth.required
def get_alarm_table():
 mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
 query_result = pd.read_sql('select * from alarms;',mariadb_connection)
 mariadb_connection.close()
 return query_result.to_json(date_format='iso')
## API route to query alarms table by given id, and return all results
@app.route('/api/alarms/<int:alarm_id>', methods=['GET'])
@basic_auth.required
def get_alarm_by_id(alarm_id):
 id = []
 id.append(alarm_id)
 mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
 query_result = pd.read_sql('select * from alarms where alarm_id = %s;',mariadb_connection, params=(id))
 mariadb_connection.close()
 return query_result.to_json(date_format='iso')
## API route to query alarm_log table, and return all results
@app.route('/api/alarm_log', methods=['GET'])
@basic_auth.required
def get_alarm_log_table():
 mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
 query_result = pd.read_sql('select * from alarm_log;',mariadb_connection)
 mariadb_connection.close()
 return query_result.to_json(date_format='iso')
## API route to query alarm_log table by given id, and return all results
@app.route('/api/alarm_log/<int:alarm_log_id>', methods=['GET'])
@basic_auth.required
def get_alarm_log_by_id(alarm_log_id):
 id = []
 id.append(alarm_log_id)
 mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
 query_result = pd.read_sql('select * from alarm_log where alarm_log_id = %s;',mariadb_connection, params=(id))
 mariadb_connection.close()
 return query_result.to_json(date_format='iso')
## API route to load current sensor data into sensor_acc712_current table
@app.route('/api/create/current/', methods=['POST'])
@basic_auth.required
def create_current_reading():
 #print(request.headers)
 try:
 mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
 mycursor = mariadb_connection.cursor()
 sql = "INSERT INTO sensor_acc712_current (outlet_id, voltage, vrms, amps_rms, timestamp) VALUES (%s, %s, %s, %s, NOW())"
 values = (request.json['outlet_id'], request.json['voltage'], request.json['vrms'], request.json['amps_rms'])
 mycursor.execute(sql, values)
 mariadb_connection.commit()
 mariadb_connection.close()
 except mariadb.Error as error:
 logger.error("Database error has occured, possible cause: ")
 logger.error(error)
 return jsonify({'status': 'failed DB'}), 400
 except IOError as e:
 logger.error("IO error has occured, possible cause: ")
 logger.error(e)
 return jsonify({'status': 'failed IO'}), 400
 except Exception as ue:
 logger.error("Unexpected Error: malformed JSON in POST request, check key/value pair at: ")
 logger.error(ue)
 return jsonify({'status': 'failed JSON'}), 400
 # Return 201 on success
 return jsonify({'status': 'succeeded'}), 201