I saw that the Surface table (now called PixelSense) detect objects. The strange thing is that it's basically an IR screen with LCD on bottom. How does it "scan" them then?

Here's a video http://www.youtube.com/watch?v=I0QBivma6j8



A step-by-step look at how PixelSense works:

  1. A contact (finger/blob/tag/object) is placed on the display
  2. IR back light unit provides light (though the optical sheets, LCD and protection glass) that hits the contact.
  3. Light reflected back from the contact is seen by the integrated sensors
  4. Sensors convert the light signal into an electrical signal/value
  5. Values reported from all of the sensors are used to create a picture of what is on the display
  6. The picture is analyzed using image processing techniques
  7. The output is sent to the PC. It includes the corrected sensor image and various contact types (fingers/blobs/tags)

Source: Microsoft Pixelsense

Also, take a look at a similar conceptual implementation (i.e., IR-camera-based) by Johnny Lee (who in fact is now in the Microsoft team AFAIK): Tracking Fingers with the Wii Remote


The surface is a vision based system that UI you see is actually projected from below and it's using infrared cameras below the glass to detect touch, objects and those scanned documents you see.


  • \$\begingroup\$ Hmm.... but how's it getting the colors correct on the scanned documents? \$\endgroup\$ – Dave Tweed Oct 8 '12 at 21:16
  • \$\begingroup\$ Yes, what technology does it use? \$\endgroup\$ – blez Oct 8 '12 at 21:52
  • \$\begingroup\$ I think it's just using object recognition to pull up a color image of that document. From what I've seen you can get pretty good black and white images but not color. Maybe the new one has color cameras as well. Here's an old video about it that's interesting at the end youtube.com/watch?v=wiraxqHVdBk \$\endgroup\$ – Some Hardware Guy Oct 8 '12 at 23:54
  • \$\begingroup\$ can't you put some links to patents ? \$\endgroup\$ – Standard Sandun Oct 9 '12 at 2:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.