OMFG - It's Stereoscopic, Baby!

After about 8 months part-time development, after many VHDL-related headache, after lots of failures, after few successful moments, after anger, after surpises and finally with a big portion of luck I am today very proud to annouce the first working proof-of-concept for live stereoscopic vision using two CMOS camera modules, the Zeiss Cinemizer Plus video eyewear and of cause with my DIY side-by-side encoder.

What is the side-by-side encoder about? Well simply speaking it does nothing more than synchronising two video streams from the camera modules, horizontially shrinking them to 50% original width and rearranging them into a new video stream, where the images of both cameras appear on the left and right side of output video signal. This 3D-format is called side-by-side and can be watched e.g. with the Zeiss Cinemizer Plus or other suitable gear. If your have full control over your eye muscles your may even be able to enjoy 3D-vision with the crossed-eye method. If your want you can transmit such a video signal via standard RF-modules form FatShark or ImmersionRC. You don't need two video links for this purpose, one is enough as all the stereoscopic information is stored in a standard video frame. The video goggles takes this signal, splits it up in two halfs, descales it to original hoizontal size and displays it in the separate lcd or oled screens inside the goggles.

For a short demonstration I have mounted the two camera modules on my small robot like this. As I still do not possess a real workshop I prefer to work with balsa wood.

The small box on the left side encloses the SBS-encoder, which the robot has to carry on its back. Wireless video transmission is not yet done, but the RF-modules are already in my shelf and the wiring of all components is envisioned to be like this:

On the left the Zeiss Cinemizer video goggles with a FatShark receiver, on the right the camera box, the encoder and the video transmitter from ImmersionRC. The next photo shows a close-up of the encoder itself. With the exeption of the XuLA-200 daughter board it's all DIY.

Finally some remarks on the creation phase of this converter:

1. In the begining of this project I have chosen the cmos camera modules rather arbitrarily, so I ended up with modules that output analog baseband video signals (CVBS). That's nice for a quick look on your TV set but a rather bad idea for what I have done here. Just imagine: the cmos camera module works in the digital domain and creates analog CVBS and the first thing I do is converting it back to digital domain BT.656 video stream. That's not very smart, as I recently leant that there are indeed ccd or cmos camera modules on the market with a digital BT.656 conforming output. Using two of such modules the encoder PCB you see above would shrink to half of the present size because all the components on the right hand side of the XuLA-200 FPGA are devoted to CVBS-to-BT.656 conversion (with two video channels).
2. Don't underestimate the synchronisation issue of both video streams. This is really essential. You will need some kind of adaptive delay line in order to compensate for the (a priori unknown) delay between both streams. In my case I even had to use the external SDRAM resources of the XuLA-board since the internal SRAM of the FPGA is way too small. This was really a had fight to get the adaptive delay up and running.
3. I was not able to use two camera modules out of the box, since the local XTAL jitter between both modules undermines any attempt to synchronize the video signals. See my other blog posts on this problem. I would estimate that a camera module with dedicated CLK input and digital video data output would not have this problem.

There is a video attached with some lousy live footage. Before you ask - I just grabbed some toys from my kids. If you have  3D-video glasses check it out or try the crossed-eye-trick. I think youtube is smart enough to offer more 3D-formats than just the side-by-side material I have uploaded.

So far for this moment. More is yet to come. Any comments are welcome.

Cheers, A.