0
\$\begingroup\$

My project is to control a Nintendo 3DS hand held video game system from my Arduino Mega. I have all the basic controls wired up and working well.

I need to be able to recognize the state of a video game being played by it's known audio cues.

Specifically, the Super Smash Bros fighting game. At the end of the match there will be a winner. The winner is announced with a known sound clip.

An example is: When Mario wins, there will be a known sound clip that plays with the announcer saying "The winner is Mario!".

To my advantage the "match winner" sound clip that plays are all known and will be EXACTLY the same each time it is played.

I need to recognize this specific sound clip and then have the Arduino press the "Start" button on the Nintendo 3DS to move the game to the next screen.

I have done numerous searches on FFT, FHT, Matched Filters, Speech Recognition, etc and have not come up with a solid way to do this.

I don't think speech recognition techniques work well since the audio is not a regular human voice, but a video game announcer "cartoon" voice with lots of background sound affects.

One theoretical approach I have is to somehow "train" the Arduino with pre-recorded sound clips and have it react when it hears them. Similar to the EasyVR Shield 3.0.

Any thoughts on how to do this?

Video of the sound clips: https://www.youtube.com/watch?v=VzC6psWEGM8

I am a software engineer and I am very comfortable around coding! Any help is appreciated.

\$\endgroup\$
4
  • \$\begingroup\$ This is going to be very difficult, especially with only the processing power of the mega. You may have an easier time doing a memory dump/analysis on the fly and looking for specific values. \$\endgroup\$ Jul 29 '15 at 18:26
  • \$\begingroup\$ the Mega is not suited to this sort of audio analysis. Get a more modern ARM Cortex M4 with built-in DSP (digital signal processing) instructions, which WILL be able to analyze the audio signal on the fly, and do what you want. \$\endgroup\$
    – KyranF
    Jul 29 '15 at 18:31
  • \$\begingroup\$ Any idea how this is being done with the EasyVR Shield 3.0? It doesn't appear to have any separate processor. sparkfun.com/products/13316 \$\endgroup\$ Jul 29 '15 at 19:18
  • \$\begingroup\$ The sheild itself has another IC which handles it. If you look at the images, you can see a black blob of epoxy which is covering some die or IC. That is most likely what handles all the voice recognition stuff \$\endgroup\$
    – efox29
    Jul 29 '15 at 19:57
2
\$\begingroup\$

The device in question appears to be based on a COB (chip-on-board) version of Sensor Inc.'s neural network voice recognition chips.

That's mounted on a module and "shield" (barf) made by VeeaR, apparently an Italian company.

There are open-source voice recognition programs (maybe this one, for example), so this might be appropriate for a reasonably capable processor running Linux- perhaps Beaglebone Black or Raspberry Pi.

\$\endgroup\$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.