# Tracking pen within whiteboard surface

(Question in response to the comments.)

I have a fixed flat white rectangular whiteboard of dimensions 1 meter by 0.5 meter and pen. I want to track the position of the tip of the pen within the whiteboard, in real time.

I am allowed to have as many different sensors and processors on the sides of the whiteboard as I want, but there are three main restrictions:

1. The pen should operate freely; no wires or strings between the pen and the sensors are allowed.

2. The pen should be low power, ideally powerless.

3. The position of the tip of the pen should be determined with an accuracy on the order of 1mm.

What electronic setup would allow me to compute the position of the tip of the pen within the whiteboard with the above restrictions?

• Make sure you understand the difference between triangulation and trilateration. If you're measuring distances, you're talking about the latter, not the former. The point is, the error distributions of the two methods vary quite a bit (and differently) over a rectangular field. With either method, the region that lies between the two reference points is going to have poor resolution in one axis or the other when converted to rectangular coordinates. Nov 13, 2012 at 12:40
• possible duplicate of Ideas to measure 2D position of an object constrained to X-Y plane Nov 13, 2012 at 12:45
• @DaveTweed Triangulation also measures distance. The inputs from the observation points are angles, not distances themselves, if that's what you mean. Nov 13, 2012 at 16:01
• @Samuel: The OP is talking about observations that are distances, not angles. Nov 13, 2012 at 16:28
• @DaveTweed: Thanks for the comments. The end goal is that the position of the point object be determined. Whether I use distance measuring sensors or angle measuring sensors is not important. Whether I use triangulation or trilateration is not important either. I have rewritten the question to make this clearer. Nov 13, 2012 at 21:52

The technology used by Microsoft Kinect is possibly the most cost-effective solution for the stated requirements.

The approach consists of a digital video camera / webcam at the sensing position, with a field of view covering the area of interest. In order to consistently locate the target object to a 1 cm precision, the camera must be able to capture a minimum resolution of 1 pixel for every 0.5 cm. This translates to 200 pixel resolution for the long edge, and 100 pixels for the short edge, at the bare minimum.

This is comfortably covered by using a 640 x 480 VGA resolution webcam, providing sufficient oversampling for enhanced precision if needed.

Depending on the processing power available to you, sampling precision and position update rates can be changed, to meet your unspecified "real-time update" rate requirement.

Alternatively, the Kinect SDK released by Microsoft (and a Kinect device) would allow a rapid prototyping of the requirement in a matter of hours. Then, depending on performance and price considerations, the final approach can be worked out.

• Thank you for this idea. The problem is that the point object ("the pen" in the updated question) should be very lightweight (very low power). The sensing should be done by sensors on the periphery of the surface. Nov 13, 2012 at 21:54
• @Randomblue That changes the question entirely, doesn't it? Not quite an "updated" question but a new one. I'll post a fresh answer in a bit. Nov 14, 2012 at 4:14
• Yes, sorry about that. I'm now thinking that four ultrasonic receivers around the whiteboard, and an ultrasonic transmitter in the pen could do the job. Nov 14, 2012 at 4:21

A 1mm accuracy is going to be difficult to achieve. I noticed this is 10-20 times more accurate than what you originally asked for, but let's go with it and see where we get.

There are at least two approaches that warrant further research and testing:

1. Surface area sensing-
You haven't explicitly said you can't do it, but why not put a capacitive sensor grid behind the board? You could fairly easily build a 10cm x 10cm board section that has ~1mm accuracy. Building multiple modules would allow the system to be expandable. I asnswer a previous question on capacitive sensing here, that should help you get started on research for this method. I have a feeling you're not telling us the whole story though, and this option won't work for you because maybe you're trying to do what this guy was doing and write on any surface.

2. Large array of edge sensors -
It's good that you've removed the limitation of just two sensors, I think it would have been nigh impossible to make the system you want with only two sensors. As discussed elsewhere here if you're using sensors on the boards edge you'll either need to measure angle from sensor to target and use angulation, or you'll need to measure distance and use lateration. Distance measurement is going to be the easiest to measure and the easiest to array.
So, here we are, measuring distance from an array of sensors on the edges of the board. You likely can't fire all the sensors at once if you're using optical distance sensors, so you'll need to pulse through each one, maybe in succession, maybe not, it depends on how many there are and how quickly you can sample from them. In any case you'll have all these sensors measuring the distance from themselves to whatever, sometimes the other side of the board, which is not likely a useful measurement. You'll have to process the data coming in from all these sensors, filter it slightly and decide what is a measurement of empty space and what is a measurement to something between the sensor and the edge of the board. So let's say you've figured that out, you have this set of distance measurements from the sensor to the object of interest. What are you going to do with it?
Ideally this sensor array goes around the entire circumference of the board, the density of this array will alter the ultimate accuracy of the system. Accuracy is really what this is all about right? The problem with any sensor, of course, is it's inherent inaccuracy. So although the mathematics for lateration are quite simple in the trilateration case where three perfect distance measurements can only point to one location, you're not going to get perfect measurements. Also, you're almost certainly going to need more that three measurements to achieve the accuracy you're after. What you need here is some n-sensor error tolerant lateration equations. I happened to find an excellent paper discussing (nearly) exactly that. Read it to learn more.

It's hard to say what you were expecting from this project but it will be quite an undertaking, especially with the constraints you've given. Good luck.

This problem can be easily and cheaply solved by tracking IR pen dot with IR camera built in Wiimote. Here is a project link.

• I see several problems with this which would be solved by having the sensors on the edges of the whiteboard: 1) The wiimote should be (re)positioned to always "see" the pen. 2) Four-point touch calibration is required. 3) The resolution is not great. Nov 14, 2012 at 18:29
• If semi transparent whiteboard is allowed, then you can use a laser pen on one side and put a high resolution camera on the other side to track the laser dot. Alternatively you can put several side cameras to track laser dot and use math to determine 2D position of the dot with accuracy depending on camera resolution.
– avra
Nov 15, 2012 at 8:30
• I forgot to mention that alternative proposal should have cameras on the same side of the pen, and in this case you can also use IR instead of laser if you use appropriate cameras.
– avra
Nov 15, 2012 at 8:39

I would probably go with two IR cameras one on the long edge and one on the short edge, and a pen that is composed of an IR LED as the tip, I would also make this pressure sensitive so it only turns on when pressed against the board. Depending on the lighting conditions of where this would need to work you may be able to get away with using a visible light led and filtering for only that colour.