0
\$\begingroup\$

I am making an application that has multiple i2c sensors controlled by an Arduino Uno. I want to get all of the raw data from the sensors and transfer them to a PC as fast as possible, on which I will perform calculation with the raw data using python. My plan is to write the arduino data onto a text file and then have python read that data and perform some calculations immediately after. I want to transfer the data as quickly as possible so it comes as close to Real Time results as possible. Should I use Ethernet, USB, or something else for comms? Should I use a raspberry Pi3 for this instead? Am I approaching this problem incorrectly? Any help would be appreciated! Thank you.

\$\endgroup\$
  • 2
    \$\begingroup\$ How many sensors? How much data? \$\endgroup\$ – filo May 11 '18 at 4:54
  • 1
    \$\begingroup\$ You say fast, and you also say real time. Do you want to emphasise throughput, or latency? How much data? This will affect whether a USB/COM port is sufficient, or whether you need to fake an e-SATA interface. Starting with a Pi would probably be better, easier, as then you can ethernet the data to the PC, and that's a lot of known hardware and software options, sockets, UDP/TCP, ZeroMQ etc. 'Fastest' without context is meaningless. What's the fastest way to get from A to B? Sports car, or UPS truck? What if its a ton of parcels? Is a bike adequate, or do you need a jet? \$\endgroup\$ – Neil_UK May 11 '18 at 5:23
  • \$\begingroup\$ think thruput or latency not speed without considering overhead \$\endgroup\$ – Tony Stewart Sunnyskyguy EE75 May 11 '18 at 5:59
  • \$\begingroup\$ both throughput and latency are important since I don't want a delay in between sensing and sending data. The system will contain 10 i2c sensors. Each sensor will account for sending 20 bytes of data per sensor from the arduino to the PC so total 200 bytes per transfer. Is the Arduino ethernet shield a bad option vs Rpi? \$\endgroup\$ – Bob Smith May 11 '18 at 6:38
  • 1
    \$\begingroup\$ "I don't want a delay" is not a specification. How much delay is acceptable? 1s, 1ms, 1us, 1ns, 1ps, ... ? And how often do you want to read your 10 sensors? Note that there will be a delay between reading the first sensor and the last one, and you don't seem to be bothered by that delay. \$\endgroup\$ – Wouter van Ooijen May 11 '18 at 8:04
5
\$\begingroup\$

FTDI FT232RL serial-usb chips can run at 2Mbd , i.e. 200kbytes/sec. This is much faster than the I2C rate.

Say you read I2C at max rate of 400kHz (44kByte/s), you only need 887kBd to send the data back as hex-ascii (2 chars/byte).

Of course that is the sustained I2C read rate once addressed. In practice you have start/stop,addressing, control register writes, arduino code, etc etc. So your actual ability to get data to send back is probably only half that. You are likely to find that 230kbd or 460kbd are quite adequate.

So you are good with the standard usb-serial chip, just change baud rate. However, the arduinos serial routines might not be up to it.(not an arduino user myself).

Your starting place is to get the serial running a test. https://arduino.stackexchange.com/questions/296/how-high-of-a-baud-rate-can-i-go-without-errors

Realterm is able to set these (any) baud rates, and capture the data to file, timestamp, run postprocessing scripts etc. It also has a capability to convert and display hexcsv data (specifically from an I2C device)

Note that at high baud rates the Arduino might need to you to space chars you send to it. You might also find that to get max send rate, you need to handle the TX uart directly to bypass any software fifos, which can be pretty time consuming. The good news is that you don't have to buffer, when the uart sends so fast.

What you want to do will work fine.

You are correct to capture to file then process. It is much easier to get going, and to fault find later. If you have a choice, ASCII chars are easier than binary. A hexcsv format is easy for what you want to do.

A good tip is you can zip these data files if you want to keep them, they always compress massively.

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ How would one go about to 'handle the TX uart to bypass the FIFOs'? Is it a bad idea to send the data from the arduino to the PC as a string and then record the data using the python serial listener? Also, I was under the impression that Ethernet would send the data faster. Is this not the case? \$\endgroup\$ – Bob Smith May 11 '18 at 6:42
  • 1
    \$\begingroup\$ Your arduino hardware can already send the data faster through the uart, than you can get it from the I2C. Yes ethernet, firewire, and SpaceX Falcon9 are all faster. So? \$\endgroup\$ – Henry Crun May 11 '18 at 7:26
  • \$\begingroup\$ Your starting place is to test how fast you can make serial work, not worry about making faster until you know it isn't good enough. See link above \$\endgroup\$ – Henry Crun May 11 '18 at 7:31
0
\$\begingroup\$

The uno doesn't support this, but newer arduino models do away with the external USB to serial chip and instead directly connect the USB port to the microcontroller. I think by default it emulates a serial port over USB, but since it's not actually a serial port it's not limited to serial port baud rates and as a result will transfer data much faster. Might be worth looking at as you should be able to get much better performance.

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Might get much worse performance also. Some of these on-chip usbs have abysmal performance, as they weren't developed beyond "seems to go ok", don't have drivers setup properly etc etc. I have had a few "why is my XXX/atmel/pic/stm usb not working properly/not able to open/not listed in ports/really slow" queries. \$\endgroup\$ – Henry Crun May 11 '18 at 5:53
0
\$\begingroup\$

Would you like 5.3Mbit/sec (5333333 Baud) ? Here is how to:

http://peter.lorenzen.us/embedded/dprint

http://peter.lorenzen.us/3d-printer/stress-testing-5-3mbit-sec-serial-debug-stream-from-arduino

Maybe you will run to problem to have ready so much data at this rate :)

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Nice to know that PL2303 is faster than FT232 for this job. The main problem is that every single PL2303 thing I have is no longer recognised by the driver on W10 - apparently all non-prolific parts. \$\endgroup\$ – Henry Crun May 12 '18 at 3:39
  • \$\begingroup\$ The question was about Arduino and computer with python with high priority on speed. PL2303 works well on Linux. If the main matter is speed and reliability, I would not use W10 anyway. (No other aspects was mentioned by author, just speed, speed and speed.) \$\endgroup\$ – gilhad May 12 '18 at 23:30
  • \$\begingroup\$ You did not missed it, he did not state OS he is using, neither Linux or W10. He also did not say, that he is to process all that data on desktop. He just repeated, that he need it really fast and that he 1ms latency is "acceptable" - not hard to suppose, that he would use dedicated computer for that reason and so he would install system according to application needs - that usually is linux these days (be it for web needs, computational needs, or many other reasons). Even Microsoft offers Linux based solution for bussiness. It is just about using the right tool the for job at hand. \$\endgroup\$ – gilhad May 13 '18 at 9:01
  • \$\begingroup\$ Basically the strong need to refresh full dataset more that 10x faster than is typical screen refresh with latency under 1ms does not evoke typical desktop use at all, sounds more like something dedicated to contrlol some machine in real time - and it is not typical "Window 10 desktop" area at all. And talking about 2018, when I go shopping, the music and ads in market all are goes from python running on PC with linux for more than 10 years anywhere i go. None from windows machine. Windows are used by the shop staff to access web pages on linux server to arrange order of the music and adds. \$\endgroup\$ – gilhad May 13 '18 at 9:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.