# How are objects rendered with a MEMS mirror for AR?

This question is an intersection of both hardware and software. I am trying to understand the theory of AR technology, but hitting some roadblocks and cannot find any good online resources that talk about this in detail. Please correct me if my understanding is incorrect.

2D MEMS mirrors will reflect a pulse of light from a laser at a very high frequency so it can achieve rendering x pixels/sec (each pixel is a single pulse from the laser). I am using a rendering software, OpenGL to design the actual object I want to render. What I do not understand is the interaction between the rendered object, the laser, and the MEMS mirror.

I think I have an idea. The rendered image in OpenGL can be converted to a PPM file where every pixel is defined with the RGB value. These values can be fed into the laser driver so it can generate each pixel for each pulse. Now the MEMS mirror has to be angled correctly so it can be displayed at a certain position in the coordinate system. In order to actually angle the MEMS mirror, a certain voltage must be passed into the x, y inputs of the mirror at certain times.

By steps:

1. Render object
2. Convert to PPM
3. Generate a “angle” file which maps voltage to specific angles for the MEMS mirror

The coordinate system is a totally different question, but it looks like extrinsic and intrinsic matrices are the right direction.

• sounds about right? Jun 25, 2020 at 20:04
• MEMS mirrors move very slowly (many milliseconds to reach to a specific position) - to get an adequate frame rate you will probably need to sort the list of angles and pull the laser on the fly - or just arrange to have the mirror do a raster scan. Jun 25, 2020 at 20:18
• When looking into the holo lens, I did see they had 2 1D MEMS. 1 for horizontal and 1 for vertical. The horizontal was fast scanning and the vertical was slow scanning. I’m still unsure what that means, but it seems like that solves the issue of MEMS taking a couple of milliseconds to reach a specific position. Jun 25, 2020 at 20:35
• After sitting and thinking I was able to understand the reasoning for a fast and slow scanning mirrors. If it takes so long (relative) to move the MEMS, how are such high resolution images able to be created in the new HoloLens 2? And I would assume the angles will already be sorted so you consecutively display each pixel in order. Jun 26, 2020 at 3:24

MEMS systems can have resonant frequencies measured in kilohertz, so as long as you don't mind having the mirror continuously moving you scan fairly fast (accelerating/deaccelerating to hit specific point would be much slower or even impossible depending on the scanning mechanism).

The other option is to use an array of MEMS mirrors as in a DLP chip. These are very small, but typically have a frequency of about 32khz. The array then works like a 32000 frame per second, 1 bit per pixel display. PWM is used to encode gray levels.

If it takes so long (relative) to move the MEMS, how are such high resolution images able to be created in the new HoloLens 2?

If the 54 kHz you quoted is accurate for the fast axis rate, the scanner draws one line of pixels every 18.5 microseconds, and an individual pixel probably every few tens of nanoseconds. It is therefore moving quite fast. Fortunately, 54/2 kHz = 27 KHz (presuming both the forward and backward direction is used to reach that speed) is too high a frequency to hear or it would be quite annoying to be around.

And I would assume the angles will already be sorted so you consecutively display each pixel in order.

Yes, pixels are drawn out in sequence with a MEMS scanner, but it is more complex than that. The fast axis scanner is at resonance, meaning it traces out a sinusoidal trajectory. It thus moves very fast in the center of the field and very slowly at the edges, so you must pre-warp your pattern to compensate for the non-uniform rate at which pixels are drawn.

• I’ve looked into DLP chips as an alternative, but my biggest concern is that the size of the chip determines caps the resolution. Looking at AR glasses for example, that could be a pretty big disadvantage. I see your point on the scanning mechanism issue. It would seem like “painting” an object onto a scene with color fills and not just outlines, would be quite tough (especially assuming its ) Jun 26, 2020 at 1:06
• @joethemow The size of the mirror in a scanning system limits resolution in exactly the same way as the width of the DLP chip limits it, since diffraction only cares about your focal length and your aperture diameter, not what the mechanism generating the image is. The advantage of a scanning system however is that you can choose to reduce resolution in order to improve frame rate, whereas for a DLP chip you can reduce grayscale levels if you want to increase frame rate. Jun 26, 2020 at 2:01
• Hm but isn’t MEMS just limited by its resonant frequency? Like if you wanted to increase the resolution of an image you would have to increase the size of the DLP chip, but with MEMS you’d have to figure out a way to increase the resonance frequency which may increase the size of the mirror, but doesn’t seem like its a 1:1 physical size increase when it comes to DLP vs MEMS. microvision.blogspot.com/2019/05/…. 2nd to last question they mention that the HoloLens has a mirror cycle time of 54,000 times/sec. Jun 26, 2020 at 3:17
• @joethemow The fast axis will be resonant, but the slow axis is not since it moves at only a few Hz. If you get 10fps at 1000x1000, you could get 20 fps at 500x500 (or 1000x500 if you can tolerate anisotropic pixels). Increasing resolution will require you to make both solutions bigger, or to make the focal length of the lens shorter (increase ray angle) due to diffraction. Jun 26, 2020 at 3:24
• Ok that makes sense, just to be sure I am understanding this correctly - the fast axis will “paint” the major axis (assume it would make more sense to have fast axis paint the axis with greater number of pixels) while the slow axis will move the angle along the minor axis. Jun 26, 2020 at 3:30