6
\$\begingroup\$

Background

Johnny Lee demonstrated various interesting ideas (detailed, e.g., in this video as well as this page) that take advantage of the Infrared camera from a Wii-Remote. The IR camera has 1024X768 resolution, and is designed to position-track the 4 brightest infrared-lit points in its view, at 100 Hz. Each of these 4 "points" could be a moving marker in the form of, e.g., an infrared LED -- the LED's emission is detected by the camera, which in turn outputs at 100 Hz, the position data of the IR "blob" observed. Which allows a fast and inexpensive DIY position-tracking system.

Problem

In the above setup, if each IR LED is not just powered on but also somehow made uniquely IDENTIFIABLE, it would give rise to many interesting possibilities. For example, this would allow continuously position-tracking each marker in space uniquely (i.e., with knowledge of which blob is which). In addition, having each IR LED marker being unique also means the setup could be extended to any number of points (say 50 markers) instead of just the 4 brightest points.

The question is: Assuming you start by connecting each IR LED marker to a microcontroller, what would be the most effective way to extend the above setup so that each IR LED marker is uniquely IDENTIFIED? I roughly describe one approach below -- is there some more versatile or simpler approach than it, or perhaps can improvements be made to it?

First, a method that is NOT promising: Since each infrared MarkerLED is connected to a microcontroller, you could have each MarkerLED blink in a unique pattern. But the IR camera has only 100 Hz refresh rate so if there were 50 LEDs, it would be difficult to fit in a unique pattern for each, without the camera's effective position tracking of the points becoming really slow.

Below is a rough idea I'm currently considering (tracks & identifies 50 IR LED markers):

  • Start by tagging on a simple IR-Detector next to the IR-Camera, both of whose ouputs are read/tracked in sync by a common microcontroller or computer.

  • Now, let's say there are 50 Markers. For each MarkerLED/Microcontroller circuit, you also add on a second IR LED, called the IdentifierLED, thus there is a pair of IR LEDs for each marker, both controlled by the microcontroller.

  • For a given marker, so that its position can be tracked, the MarkerLED is only turned ON for exactly a specific 20-millisecond-window out of each second (each marker has its own 20-msec window). During that same particular 20-msec-window, the corresponding IdentifierLED is pulsed in a specific manner by the microcontroller at a high frequency (e.g., 38 kHz), setting up a unique pattern/ID for that specific marker.

  • And the same for the remaining 49 Markers consecutively, each with its own different 20-millisecond-window and its own high-frequency identifying-pattern. That takes care of the markers' side of things.

  • Now, on the sensing side of things: For each consecutive 20-millisecond-window during a second, the IR-Camera detects the position of one specific marker (whose 20-millisecond-window it is) via the corresponding MarkerLed. At the same time, the IR-Detector identifies WHICH marker it is, from the detected pattern of the corresponding IdentifierLED.

  • And this position- and identification-tracking continues for all fifty of the 20-millisecond-windows within each second.

  • Thus, all 50 markers are tracked, with the tracking-side microcontroller able to update each marker's data once per second.

\$\endgroup\$
  • \$\begingroup\$ Do you try 2D or 3D tracking? If 2D: What kind of surface? \$\endgroup\$ – suha Aug 25 '12 at 20:04
  • \$\begingroup\$ Using one such camera, it's possible to do 2D tracking in air (or any surface). Note that the data is in fact pretty much pseudo-3D, because the camera output data also includes the intensity of each of the 4 brightest blobs, and the depth/3rd-axis could be inferred from the intensity). Of course, just adding a 2nd camera and using trilateration, the tracking could be made true 3D. \$\endgroup\$ – boardbite Aug 25 '12 at 20:14
  • \$\begingroup\$ From how you described the problem, the hardware limits you to tracking only 4 points.I'm not sure it's changeable. If you're idea is to have access to the full data of an IR camera, if you can share data about your sensor , it would be helpful. \$\endgroup\$ – hulkingtickets Sep 3 '12 at 21:08
  • \$\begingroup\$ @yaniv: The camera/sensor merely outputs the X and Y coordinates of the FOUR brightest infrared points it sees. I can't change the camera/sensor; Hence, the question is about how I can add to my whole setup to make it possible to identify and track multiple LEDs, even if it costs me some of the speed/update rate. \$\endgroup\$ – boardbite Sep 5 '12 at 11:44
1
+50
\$\begingroup\$

Rather than transmit the marker ID code from the marker to a central receiver, perhaps it would be simpler to transmit the marker ID code from a central transmitter to the marker.

The central transmitter (perhaps a 38 kHz IR transmitter or some wireless transmitter) would send, in effect, "Marker number 22, please turn on for the next 20 ms on my mark: NOW". (Ideally, while that marker is glowing for those 20 ms, the central transmitter is sending out the ID of the next marker to turn on).

Since that one central transmitter is controlling the timing, you won't have to deal with markers getting out-of-sync and accidentally transmitting at the same time.

Hopefully you can place that central transmitter close enough to the position tracker, so that if any marker can't see the commands sent by the central transmitter, that marker wouldn't be in the visual field of the position tracker anyway.

That also gives you the flexibility of using the data you get back from your position tracker to dynamically adapt which markers you select:

  • If some markers seem to be motionless or extremely slow-moving, perhaps you only need to check on them once every 3 seconds or so.
  • Perhaps you can check up on the latest positions of more than one marker at a time; something like "OK, marker number 22 and 23, please turn on for the next 20 ms on my mark: NOW".
  • If some markers are not visible from this vantage point, perhaps you only need to check if any of them have re-entered the visual field once every 3 seconds or so.
  • The time slots you free up with the above techniques could be used to track the remaining markers at a somewhat faster update rate than you could if you simply cycle through every marker in a fixed pattern.
  • If, say, marker #22 is so close to marker #23 that receiver of #23 is blinded when marker #22 is active, you could shuffle the order you turn on the markers so that the "#23 please turn on" message happens a few slots before the "#22 please turn on" message.
\$\endgroup\$
  • \$\begingroup\$ I'm seeing lots of interesting routes with the dynamic adaptation idea; appreciate the thought you put into this! I'll likely update this page with my own Answer based on an offshoot of this combined with my original method. \$\endgroup\$ – boardbite Sep 13 '12 at 19:09
3
\$\begingroup\$

You forgot some sort of initialization/broadcast heartbeat, otherwise the marker would not know when to light up. Depending how good your sensors and leds are, you may be able to use different slices of the IR spectrum. This way you could distinguish an LED with a 300 µm wavelength from one with 200 µm. If you have multiple cameras, you could use different optics/filters (is feasible). Otherwise invest in a camera with a higher frame rate and let each marker blink in an unique pattern. The pattern must not only be on/off, but can leverage frequency modulation too.

\$\endgroup\$
  • \$\begingroup\$ (1) Init broadcast: I planned to "sync" the LEDs then provide each marker an offset with a multiple of 20 msec (all in the code), so that they autonomously take turns by virtue of time, but I think your method is more robust. The greeting from sensor-side could be a 38 kHz pattern. (2) What is the strategy you are suggesting with using diff parts of the spectrum? (assuming LEDs were indeed chosen so as to emit in slightly diff wavelengths) (3) A better camera -- if by better you mean even higher frame rate - would have to be MUCH faster to allow the markers to blink in a unique pattern, no? \$\endgroup\$ – boardbite Aug 25 '12 at 20:10
  • 1
    \$\begingroup\$ @boardbite if you sync, you have to guarantee that the LEDs stay in sync. Your strategy sounds reasonable so far. For the rest: i updated my post. \$\endgroup\$ – suha Aug 25 '12 at 20:23
  • \$\begingroup\$ Hmm, regarding the freq modulation with just one LED: I'm trying to imagine how it might go. For guaranteed position detection of the blob, at a minimum, the marker has to be CONTINUALLY on/visible to the IR camera for at least 20 msec. But since the LED has to be CONTINUALLY on during that time, how could a freq-modulated pattern by the same LED be made to occur during this same time? \$\endgroup\$ – boardbite Aug 25 '12 at 20:30
2
\$\begingroup\$

I assume there is some kind of powerful computer which is processing every frame of video, and can do things like measure (approximately) the brightness of each IR LED.

Simply modulate the brightness of each IR LED at a different frequency, and let the computer recognise the frequency of each one.

Take a simple case first: LED1 would vary its brightness in a 10Hz sinewave, from 50% to 100% brightness. The computer can now track the brightness of the LED, run it through a low pass filter, and use zero-crossing to measure its frequency. LED2 would be varying at, say, 15Hz, and the PC could easily distinguish between them.

It could take up to a second before the PC got a good frequency lock on both of them.

OK, but this isn't going to work for 50 LEDs. It's hard to have that many distinguishable frequencies that can be sampled by a 100Hz camera in a shortish space of time. The solution is to use DTMF!

DTMF is a method used on oldy worldy telephones to send data using tones. 8 tones are defined, and the transmitter would send two different tones at the same time, and the receiver would look up the pair of tones in a grid to choose one of 16 results.

DTMF

Now, you could easily use a 7x7 grid, to allow you to have 49 different IR LEDs. The computer should be able to distinguish between 14 frequencies if it can see the LEDs for about 1 second each. You would use much lower frequencies than the DTMF ones, say, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32 Hz.

alternatively, use just 8 frequencies, and select any two of the 8 to give 56 (8x7) combinations.

\$\endgroup\$
  • \$\begingroup\$ This is absolutely fascinating, thank you. I'm now researching this idea further to see how well it might fit the identification scheme. \$\endgroup\$ – boardbite Sep 4 '12 at 14:21
  • \$\begingroup\$ I don't think this works where the camera only reports the four brightest LEDs. \$\endgroup\$ – Ben Voigt Sep 6 '12 at 22:23
  • \$\begingroup\$ No, I imagine it wouldn't. He's need a different system to make this work. \$\endgroup\$ – Rocketmagnet Sep 6 '12 at 22:41
  • \$\begingroup\$ @BenVoigt and Rocketmagnet: Well, the four-brightest points is an integral constraint of the camera; I'm taking it as a given in the problem. But the main challenge I'm facing is the identification, not the quantity. I.e., I would be fine working with a smaller update rate for more leds (e.g., 100 Hz distributed among 50 LEDs with 4 points/second implies all fifty could be tracked at 8 Hz). That said, one reason this brightness scheme appears to NOT solve the identification is b/c the IR Leds will also move in Z-axis (near-far); their intensity will change in uncontrolled manner. Any thoughts? \$\endgroup\$ – boardbite Sep 7 '12 at 3:29
  • \$\begingroup\$ @BenVoigt and Rocketmagnet: By the way, note that (as stated in the Question) I'm OK with adding a second LED to each marker (thus a pair). So while the main LED in each pair just acts as stable position marker, the 2nd LED can be varied as necessary for any identification scheme. (E.g., I proposed kHz-pattern-pulsing in the Ques.) And likewise, note that, at the fixed point at the camera, I'm OK with having a second detection scheme of choice. The problem to solve, even in that case, is how to correlate the two detections for each marker in a way that allows both identification and position. \$\endgroup\$ – boardbite Sep 7 '12 at 3:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.