Some years ago I tried to play with 3D and head tracking. The idea was popularized by Johnny Chung Lee and is explained in http://www.youtube.com/watch?v=Jd3-eiid-Uw. But since I didn't have a wiimote, I used my macook webcam and two red LEDs for tracking. Results can be seen in this year 2007 video.

But what surprised me was that there was no portal illusion when watching with the naked eye, or even with only one eye. The brain just refused to believe that the world visible on the screen is part of the real world. I didn't know whether it was a problem with head tracking accuracy (camera position and angle), delay (~100ms at the time), jitter (camera resolution, sub-pixel accuracy for LED position calculations), lack of stereo (separate picture for each eye) or the lack of focal depth. Still don't know for sure, but John Carmack's this years QakeCon keynote suggests that the main problem is the delay.

But now, in the era of github and 3D TV-s, it's time to upgrade the experiment. This time I'm using a WiiRemote for LED tracking (to get rid of the 100ms delay) and a 3D TV to render a separate picture for each eye.

(The sad thing is that nearly every TV introduces its own 100ms of lag. Oh well, can't have everything..)

Link to the evolving project in github.