2
\$\begingroup\$

Using Arduino on an STM32F103C Blue Pill to drive an ESP8266 ESP-01 as shown below, and this tutorial: Arduino Forum

Arduino Forum

I am attempting to do an HTTP POST with the following code:

String ssid = "MySSID";
String password = "MyPassword";
String server = "192.168.1.134"; // IP hosting API (Flask w/ Python)
String uri = "/api/create/current/"; // API Route
String payload = "?outlet_id=1&voltage=2&vrms=3&amps_rms=4"; // Sample sensor payload

void httppost() {
  Serial2.println("AT+CIPSTART=\"TCP\",\"" + server + "\",80"); //start a TCP connection.
  if (Serial2.find("OK")) {
    Serial.println("TCP connection ready");
  }
  delay(1000);
  String postRequest =
    "POST " + uri + " HTTP/1.0\r\n" +
    "Host: " + server + "\r\n" +
    "Accept: *" + "/" + "*\r\n" +
    "Content-Length: " + payload.length() + "\r\n" +
    "Content-Type: application/json\r\n" +
    "\r\n" + payload;
  String sendCmd = "AT+CIPSEND="; //determine the number of characters to be sent.
  Serial2.print(sendCmd);
  Serial2.println(postRequest.length());
  delay(5000);

  if (Serial2.find(">")) {
    Serial.println("Sending..");
    Serial2.print(postRequest);

    if (Serial2.find("SEND OK")) {
      Serial.println("Packet sent");

      while (Serial2.available()) {
        String tmpResp = Serial2.readString();
        Serial.println(tmpResp);
      }
      // close the connection
      Serial2.println("AT+CIPCLOSE");
    }
  }

The post request is failing on the server-side with the following error:

Unexpected Error: malformed JSON in post request, check key/value pair at:

Nothing is else contained the error message or Flask server logs to indicate how/why the JSON is malformed. Using Postman, I'm able to perform a successful post with the following:

192.168.1.134/api/create/current/?outlet_id=1&voltage=2&vrms=3&amps_rms=4

Does anyone see anything obvious? FWIW, using the Blue Pill to power the ESP8266, or alternatively using an external power source produce the same results. It seems to be a syntactic problem but I'm not seeing it. Any help is much appreciated!

\$\endgroup\$
2
  • \$\begingroup\$ post to this site and check to see what is being sent ... beinghttp://leserged.online.fr/phpinfo.php \$\endgroup\$
    – jsotola
    Dec 21, 2019 at 20:07
  • \$\begingroup\$ You shouldn't need to guess. Modify your server to give you a debug message of the actual post body, packet sniff at the server, point it at a copy of netcat in listen mode instead if the server or modify your embedded code to log or snoop with another USB UART or cheapie logic analyzer. \$\endgroup\$ Dec 21, 2019 at 20:08

2 Answers 2

3
\$\begingroup\$

You are mixing up two different ways of sending information to a server.

When you use postman with the URL

192.168.1.134/api/create/current/?outlet_id=1&voltage=2&vrms=3&amps_rms=4

You are not actually sending any useful information in the body, but rather sending all of the data as URL parameters.

In contrast, your embedded code:

String uri = "/api/create/current/"; // API Route
String payload = "?outlet_id=1&voltage=2&vrms=3&amps_rms=4"; // Sample sensor payload

attempts to supply the "payload" in the message body.

But the problem is that your payload is not encoded as json, but rather still encoded as a series of URL parameters.

You should either add that "payload" string onto the end of the URL and not send it in the body.

Or else, if your server supports it, encode the payload as proper json, probably something like:

{"outlet_id": 1, "voltage": 2, "vrms": 3, "amps_rms": 4}

Note that if you want to put this as a constant into a C/C++ program, you'll have to escape the quotes, for (brief) example:

String payload = "{\"outlet_id\": 1}";

Also should you have any json values beyond numbers and true/false/null those would need their own quotes, eg

{"mode": "test"}

Ultimately this is an everyday RESTful API question more suited to Stackoverflow and not an embedded or EE question at all, though needing to be able to work with these kinds of things isn't uncommon in the age of IoT.

\$\endgroup\$
5
  • \$\begingroup\$ Fully agree. In addition: Content type should not be set to "application/json" if no JSON is sent. \$\endgroup\$
    – Codo
    Dec 21, 2019 at 20:24
  • \$\begingroup\$ Thanks for the input, all. Wasn't sure where best to put this question, but I figured those more versed in STM32 might have better insight. I'm currently attempting option 1) add that "payload" string onto the end of the URL, and if I'm unable to get this to work, I'll try formatting JSON (with escaped characters) inside my embedded code instead. \$\endgroup\$
    – rjkunde
    Dec 22, 2019 at 22:36
  • \$\begingroup\$ It's really not about the STM32 in any way at all \$\endgroup\$ Dec 22, 2019 at 23:40
  • \$\begingroup\$ @ChrisStratton, should I remove this post? I found found a solution that others may find useful, but don't want to pollute the EE S.E. \$\endgroup\$
    – rjkunde
    Dec 23, 2019 at 0:52
  • 1
    \$\begingroup\$ Since there hasn't been a response, I'm going to go ahead and submit my answer which contains the solution I used. I agree, not EE specific, but does involve a particular combination (STM32 + ESP8266) which has limited documentation / examples on the internet. People may find this useful, and I don't want to cross-post. \$\endgroup\$
    – rjkunde
    Dec 24, 2019 at 21:45
1
\$\begingroup\$

Solution: Use ArduinoJson library to properly form JSON within C environment.

//
// ESP8266 HTTP Post Program
//
// Uses an ESP8266 ESP-01, connected to an STM32 Blue Pill
//
// Must have 128k flash variant of the STM32!
//
//  Pins
//  STM32 pin PA2 Serial 2 (RX) to ESP8266 TX
//  Arduino pin PA3 Serial 2 to voltage divider then to ESP8266 RX
//  Connect GND from the STM32 to GND on the ESP8266
//  Connect 3.3V from the STM32 to VCC on the ESP8266
//  Pull ESP8266 CH_PD HIGH via jumper from ESP8266 3.3V line
//
// Original code credit / inspiration:
// https://community.wia.io/d/25-how-to-setup-an-arduino-uno-with-esp8266-and-publish-an-event-to-wia
// Alan - WIA community admin
//

#include <ArduinoJson.h> // Must use library version <= 5.13.4, 6.x.x is incompatible. Handles JSON formatting

String basic_auth  = "bmVvOm1hdHJpeA==";                  // HTTP basic authentication username/pw string: neo/matrix
String wifi_ssid = "Your_Wireless_SSID";                  // Wifi network SSID
String wifi_password = "YOUR_WIRELESS_PASSWORD";          // Wifi password
String host = "YOUR_SERVER_IP";                           // Server hostname or IP
String path = "/api/create/current/";                     // API Route
String port = "80";                                       // HTTP Port

int outletId = 1;           // Unique, hard-coded ID for each outlet.
int loopDelay = 5000;       // 5 second delay between sensor reads
int countTrueCommand;       // Used in determining success or failure of serial commands
int countTimeCommand;       // Used in determining success or failure of serial commands
boolean found = false;      // Used in determining success or failure of serial commands

// Buffer to store JSON object
StaticJsonBuffer<200> jsonBuffer;
JsonObject& root = jsonBuffer.createObject();

void setup() {
  Serial.begin(9600);                                                                           // Start STM32 Serial at 9600 Baud
  Serial2.begin(9600);                                                                          // Start ESP8266 Serial at 9600 Baud
  Serial2.println("AT");                                                                        // Poke the ESP8266             
  Serial.println(Serial2.read());                                                               // Check that AT firmware on ESP8266 is responding
  sendCommandToSerial2("AT", 5, "OK");                                                          // If status is okay, set radio mode
  sendCommandToSerial2("AT+CWMODE=1", 5, "OK");                                                 // Set radio mode
  sendCommandToSerial2("AT+CWJAP=\"" + wifi_ssid + "\",\"" + wifi_password + "\"", 20, "OK");   // Connect to pre-defined wireless network
}

void loop() {

  // Delays sensor reads as desired
  // delay(loopDelay);

  // JSON Data - using ArduinoJson library object
  // TODO - Stubu in JSON arrays for remaining sensor values
  root["outlet_id"] = 1;
  root["voltage"] =  2;
  root["vrms"] =  3;
  root["amps_rms"] =  4;
  String data;
  root.printTo(data);

  // HTTP post request
  String postRequest = "POST " + path  + " HTTP/1.1\r\n" +
                       "Host: " + host + "\r\n" +
                       "Accept: *" + "/" + "*\r\n" +
                       "Content-Length: " + data.length() + "\r\n" +
                       "Content-Type: application/json\r\n" +
                       "Authorization: Basic " + basic_auth + "\r\n" +
                       "\r\n" + data;

  // Send post request using AT Firmware
  sendCommandToSerial2("AT+CIPMUX=1", 5, "OK");
  sendCommandToSerial2("AT+CIPSTART=0,\"TCP\",\"" + host + "\"," + port, 15, "OK");
  String cipSend = "AT+CIPSEND=0," + String(postRequest.length());
  sendCommandToSerial2(cipSend, 4, ">");
  sendData(postRequest);
  sendCommandToSerial2("AT+CIPCLOSE=0", 5, "OK");
}

// Function to determine success / failure of serial commands send to/from ESP8266
void sendCommandToSerial2(String command, int maxTime, char readReplay[]) {
  Serial.print(countTrueCommand);
  Serial.print(". at command => ");
  Serial.print(command);
  Serial.print(" ");
  while (countTimeCommand < (maxTime * 1))
  {
    Serial2.println(command);
    if (Serial2.find(readReplay))
    {
      found = true;
      break;
    }

    countTimeCommand++;
  }

  if (found == true)
  {
    Serial.println("Success");
    countTrueCommand++;
    countTimeCommand = 0;
  }

  if (found == false)
  {
    Serial.println("Fail");
    countTrueCommand = 0;
    countTimeCommand = 0;
  }

  found = false;
}

// Send post request to Arduino and ESP8266
void sendData(String postRequest) {
  Serial.println(postRequest);
  Serial2.println(postRequest);
  delay(1500);
  countTrueCommand++;
}

Python/Flask API: app.py

#!/usr/bin/env python3
from flask import Flask, jsonify, abort, make_response, request, url_for
from flask_basicauth import BasicAuth
import time
import sys
import pandas as pd
from pandas.io import sql
import mysql.connector as mariadb
import pymysql
import logging

app = Flask(__name__)

# Setup Logging
logger = logging.getLogger('werkzeug')
handler = logging.FileHandler('api.log')
logger.addHandler(handler)

## Flask-BasicAuth
## As header for programmatic access use: "Authorization: Basic bmVvOm1hdHJpeA=="
app.config['BASIC_AUTH_USERNAME'] = 'neo'
app.config['BASIC_AUTH_PASSWORD'] = 'matrix'
basic_auth = BasicAuth(app)

## API Routes                   Type        Description
##
## /api/alarms                  GET         Returns all records from alarm table
## /api/alarms/id               GET         Returns single record from alarms table matching integer id
## /api/alarm_log               GET         Returns all records from alarm_log table
## /api/create/current/{JSON}   POST        Inserts one record (comprised of JSON body) into sensor_acc712_current table

## 404 handler
@app.errorhandler(404)
def not_found(error):
    return make_response(jsonify({'error': 'Not found'}), 404)

## API route to query alarms table, and return all results
@app.route('/api/alarms', methods=['GET'])
@basic_auth.required
def get_alarm_table():
    mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
    query_result = pd.read_sql('select * from alarms;',mariadb_connection)
    mariadb_connection.close()
    return query_result.to_json(date_format='iso')

## API route to query alarms table by given id, and return all results
@app.route('/api/alarms/<int:alarm_id>', methods=['GET'])
@basic_auth.required
def get_alarm_by_id(alarm_id):
    id = []
    id.append(alarm_id)
    mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
    query_result = pd.read_sql('select * from alarms where alarm_id = %s;',mariadb_connection, params=(id))
    mariadb_connection.close()
    return query_result.to_json(date_format='iso')

## API route to query alarm_log table, and return all results
@app.route('/api/alarm_log', methods=['GET'])
@basic_auth.required
def get_alarm_log_table():
    mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
    query_result = pd.read_sql('select * from alarm_log;',mariadb_connection)
    mariadb_connection.close()
    return query_result.to_json(date_format='iso')

## API route to query alarm_log table by given id, and return all results
@app.route('/api/alarm_log/<int:alarm_log_id>', methods=['GET'])
@basic_auth.required
def get_alarm_log_by_id(alarm_log_id):
    id = []
    id.append(alarm_log_id)
    mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
    query_result = pd.read_sql('select * from alarm_log where alarm_log_id = %s;',mariadb_connection, params=(id))
    mariadb_connection.close()
    return query_result.to_json(date_format='iso')

## API route to load current sensor data into sensor_acc712_current table
@app.route('/api/create/current/', methods=['POST'])
@basic_auth.required
def create_current_reading():
    #print(request.headers)

    try:
        mariadb_connection = mariadb.connect(user='DB_USER', password='DB_PASSWORD', database='development_database')
        mycursor = mariadb_connection.cursor()
        sql = "INSERT INTO sensor_acc712_current (outlet_id, voltage, vrms, amps_rms, timestamp) VALUES (%s, %s, %s, %s, NOW())"
        values = (request.json['outlet_id'], request.json['voltage'], request.json['vrms'], request.json['amps_rms'])
        mycursor.execute(sql, values)
        mariadb_connection.commit()
        mariadb_connection.close()

    except mariadb.Error as error:
        logger.error("Database error has occured, possible cause: ")
        logger.error(error)
        return jsonify({'status': 'failed DB'}), 400

    except IOError as e:
        logger.error("IO error has occured, possible cause: ")
        logger.error(e)
        return jsonify({'status': 'failed IO'}), 400

    except Exception as ue:
        logger.error("Unexpected Error: malformed JSON in POST request, check key/value pair at: ")
        logger.error(ue)
        return jsonify({'status': 'failed JSON'}), 400

    # Return 201 on success
    return jsonify({'status': 'succeeded'}), 201
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.