Losing sensitive data when trying to send them via GSM

Question

I am working on a system which collects data from a set of ultrasonic sensors. Those sensors work at 40Hz frequency, and each data they send are valuable to me. I need to send those data to a mqtt broker using mqtt protocol. I use TCP/UDP connection of SIM900 in order to send data. There are two scenarios to make my system work:

1- I send the data as soon as I read it from sensors. It means I should connect to a server, send data and disconnect it in less than 1/40 seconds. I dont know, but using AT commands with SIM900; it seems pretty impossible to me.

2- I collect the data for a while, (lets say for a minute) then I send the data to server as a bulk. But there another problem arises: I am facing losing sensitive data I collect from sensor while I am making TCP/UDP connection.

My hardware consists of SIM900, an arduino and bunch of ultrasonic sensors work at 40Hz (HR-SC04)

Is there a way to save every bit of data I read from sensors and send them to Mqtt server, losslessly?

Answer 1

There is probably a way of doing this using "cooperative multitasking" -- a main loop that calls non-blocking functions (possibly using state machines). Blink Without Delay shows the first steps in this direction.

Alas, lots of code has been written with blocking functions, so rather than re-write that code to be non-blocking, often people:

• do the most time-sensitive stuff in an interrupt routine (the "foreground task").
• handle the less time-sensitive stuff in the main loop() (the "background task").

Often the code has 2 separate buffers so it can do double-buffering: the writer process slowly fills one buffer. When it is full, the reader process processes the data in that buffer -- meanwhile, the writer process slowly fills up the other buffer. Hopefully the reader process will be done with that buffer long before the other buffer is completely full, although we expect the writer to write something to that other buffer before the reader process is done. In your case, while you are making the TCP connection and sending out the data in one buffer, the processor can be occasionally interrupted to let the interrupt routine add another data item to the other buffer.