Creating a large motion sensing plane

I want to try and create a "motion sensing plane" where users can have contact points (their hands) detected when it passes through an invisible plane. Think of it like a giant tablet touch screen, except you can pass through it and there's no actual screen. The important thing is that I am able to locate not just that motion is detected, but WHERE on the invisible plane.

I've been trying to figure this one out, and was thinking about using a pair of IR sensors for the task, but I'm not totally sure that it will work. I've attached a rough sketch of what I'm considering, but I'm open to suggestions. I have a roughly $200-$300 budget for this.

Sketch:

Here are my requirements:

• The plane is roughly 8ft tall by 10ft wide, and stationary while in use
• The plane does not need accurate motion sensing at the edges of the plane (roughly 1 ft from edges of plane is not important)
• The information needs to be able to be passed to a PC for processing, ideally via USB but other methods are acceptable.
• Ideally I want to be able to recognize "push"/"pull" motions. I was considering a second layer of IR sensors, so you'd have a "front" and "back" set that would deliver rudimentary depth sensing. I don't need something more accurate than that really, just a "shallow" and "deep" touch sense.
• I also want to allow anybody to simply come up and use the plane without preparation. For instance, another alternative I was considering was a Wii Remote setup with reflective gloves or something much like the Johnny Lee samples from years ago, but if possible I'd like to avoid needing equipment on the user side.

Thanks everyone! I appreciate any and all ideas on this topic.

• A couple of additional questions that might help drive answers that help: What precision do you need for the position? How many simultaneous penetrations do you need to be able to detect? Can you tolerate a requirement for illumination along with sensors, and possibly a requirement for a specific treatment (color, reflectivity) of the boundary of the plane? – RBerteig May 6 '14 at 20:47
• See also: the Kinect, its smaller cousin the Leap Motion, and the FlatFrog optical touchscreen. – pjc50 May 6 '14 at 21:26
• @RBerteig The kind of precision I'm looking for is pretty rough, like resolution of inches or something. As far as simultaneous penetrations, 2 is necessary, while more is always useful but not necessary. What kind of illumination are you talking about? A color/reflectivity on the physical structure surrounding the plane is perfectly fine. – bearcano May 7 '14 at 19:11
• @pjc50 Yeah, I've tinkered with the Kinect, I was just wondering if there was something that could be a little more specialized and didn't require me to analyze full camera data. It still might be the best option though. – bearcano May 7 '14 at 19:12
• @bearcano Having light sources as part of the overall design can be useful. There is an ancient (1980s or so) touch screen technology based on aligned pairs of LED and photodiodes, forming a grid over a screen. Touch is observed when a finger interrupts both a horizontal and vertical beam, and works because fingers are opaque. This requires mechanical alignment, some optics at both ends of each beam, and will likely be fussy. But it can detect multiple penetrations, and depth could be handled with multiple planes, stacked. – RBerteig May 8 '14 at 0:54

Expanding my earlier comment into an answer...

A mechanically reliable technique for detecting the position within a 2D plane of an opaque object is to pair light sources with detectors such that shadows cast by the opaque object can be identified and measured. This technique was used in the 70s and 80s to provide for touch activation on top of classic dumb terminals. I can't remember the maker, but I do recall a 3rd party replacement bezel sold for the ADM-3A that held the necessary parts, for example. A quick Google for "photodiode touch screen" turned up a surprising amount of photos, designs, and even products.

Classic IR LED Beams

A straightforward way to achieve this is with a grid of light beams. Along each pair of edges you put LEDs on one and phototransistors on the other. The LED/receiver pairs are aligned and optics arranged so that each receiver primarily sees a single LED. Something like this crude ASCII-art sketch:

  vvvvvvvvvvvvvvv
|||||||||||||||
>-+++++++++++++++-> 1
>-++++X|||||||||| > 0
>-++++-++++++++++-> 1
>-++++-++++++++++-> 1
>-++++-++++++++++-> 1
|||| ||||||||||
vvvvvvvvvvvvvvv
111101111111111


To improve physical resolution, the LEDs can be lit in turn, and information from more than one receiver combined to estimate locations between the beams.

With two sets of edges covering a plane, you can identify a single penetration's location with 100% confidence. A second penetration can be identified, but will have phantom locations as well so some additional tracking heuristics would likely be required to verify which of two possible solutions matches reality:

  vvvvvvvvvvvvvvv
|||||||||||||||
>-+++++++++++++++-> 1
>-++++X|||||O|||| > 0
>-++++-++++++++++-> 1
>-++++O|||||X|||| > 0
>-++++-+++++-++++-> 1
|||| ||||| ||||
vvvvvvvvvvvvvvv
111101111101111


From the shadows alone in a single snapshot, it isn't possible to tell whether the finger tips are on the two X or two O locations. However, if the X in the upper left were seen first, then adding the second contact would hint that it is most likely that the UL didn't move, and that the fingers are on the X marks.

With scanning and wider angles of view for both the LEDs and receivers, you likely can use the far off normal LEDs to see around the shadows cast by the real fingers and rule out the virtual fingers. Taken to a logical extreme, you are re-inventing CT scanning and the math used there might actually be worth examining. You could cover a suitable diameter hoop with alternating receivers and LEDs, and the really apply the CT scanner algorithms at low resolution.

To get depth of the penetration, you would use more than one 2D array.

The biggest downsides to this approach that come to mind as I write are the logistical implications of the large number of discrete components mounted precisely, including optics. Even optics as simple as a tube to reduce field of view on each sensor still have to be fabricated and installed. And then there is all the analog signals to condition, sense, and relay back to a CPU. But solve these mechanical and logistical issues and you have an approach to consider.

Retrorflectors?

It occurs to me that one way that this might be improved to make it easier to build and alight would be to place the LEDs and photo-transistors in pairs in close proximity on one wall and a retroreflector on the far wall. Each beam would be twice as long when unbroken, but optical alignment would be a lot less critical due to mounting the emitter and receiver on the same board along with a retroreflector to return the light from one emitter mostly to just its mating receiver.

You could make a small circuit board for each pair along with a small CPU, and collect all the data (and time the beam probing in the array as a whole) with an I2C or similar bus along each edge. This would contain all of the interesting analog bits neatly, and greatly reduce the interfacing requirement to the central controller. It would also make the design modular to the point where the basic beam sensor can be built and fully tested in small quantities before you have committed to building out the whole array.

In the spirit of the earlier ASCII art, here's a sketch showing a single point detection:

     1 0 1 1 1 1 1
v^v^v^v^v^v^v^
||| ||||||||||
>-+++-++++++++++-\
1 <-+++-++++++++++-/
>-++X |||||||||| \
0 >-||  |||||||||| /
>-++--++++++++++-\
1 <-++--++++++++++-/
>-++--++++++++++-\
1 <-++--++++++++++-/
||  ||||||||||
\/\/\/\/\/\/\/

• Thanks for this, I think this is the exact kind of thing I was hoping would come from this question. Re: logistics - ideally I'd put the LEDs and receivers on "strips" that could be affixed/ removed from the sides of the panel as needed but obviously great care would need to be taken during reassembly to be sure that the measurements stayed the same. I'm more concerned with converting the analog signals to digital and getting them to the CPU. Lot to consider, but this starts me off on a path. If you have anything to add please let me know, especially wrt getting info to the PC. – bearcano May 9 '14 at 1:15
• @bearcano See my update for a further thought about co-locating the emitter and detector pair to make alignment and wiring easier. – RBerteig May 12 '14 at 22:52

It looks, in principle, like a pair of quite cheap video cameras will do the job. With fairly wide-angle lenses to give you 90 degree field of view, you'll also get close enough focusing that you can cover close enough to the cameras to meet your 1 foot requirement. Obviously, you can get USB video cameras to send the data to your PC. Analyzing the resulting images is beyond my competence. Certainly you ought to get good enough resolution to identify multiple hand-size objects.

There do exist line cameras which will give you better resolution in a single plane, but these tend to be more expensive than you are willing to spend.

• Normal 2D cameras will easily give you the Z axis information as well. – RBerteig May 6 '14 at 20:50
• If you go, for instance, to Newegg.com. and search for webcams, there is quite an assortment. Most don't seem to list FOV, but one did as 68.5 degrees, which may be a bit low. However, there was also a wideangle videoconference camera with an FOV of 100 degrees. Since most of these are listed as HD cameras, they give more than adequate resolution for detecting hands. – WhatRoughBeast May 7 '14 at 0:07
• I had considered this and have done some early experimentation using a Kinect. I was mostly wondering if there was a non-camera/Kinect-like solution that wouldn't require me to analyze camera images, since I only really need distance values of objects (such as hands) breaking the plane. – bearcano May 7 '14 at 19:13