Pasky’s Log

I spent large part of the weekend resurrecting my multitouch setup attempts again. I’m trying to use rear diffused illumination, but without much success yet. I have purchased two 1×1 meter 5mm thick plexiglass squares, but they are too impractical for the prototype yet, so I’m just using an A3-sized piece of glass with tracing paper underneath.

My elite IR light source:

Yesterday, I’ve used a setup with projector, camera and IR light source horizontally shining into a mirror reflecting everything at the surface. However, due to short cables ;-) this wasn’t very practical and everything had to be much too far from the mirror, plus there were some (although not so bad) double-reflections. So today I’ve rearranged the setup to a much simpler vertical one.

At the bottom is an Acer 1024×768 DLP projector, then on the white box is my IR light source illuminating the surface from an angle, then on the chair is sticked (literally) an IR webcam taking in a subset of the surface – a piece of glass with tracing paper sticked to the bottom side.

Unfortunately, so far I’m unable to get reasonably even IR illumination; it seems six leds are way too few, and the light is not diffused enough. I need to get multiple IR sources, and some translucent surface to diffuse the light a bit. Another problem is that the projector also shows the image in the IR spectrum, thus it somewhat interferes with the reflections; however, having strong enough IR light source should marginalize that problem.

The last problem is that my webcam is too crappy, on Linux it gives me only one to two FPS on 320×240. I’m planning to buy the absurdly cheap MSI StarCam Mini+ which should give me 30fps on 640×480 and has already night vision support so it should be enough to block visible light; I’m not feeling yet like shelling out huge money for Xbox or PS3 webcam.

Now I mainly wish I knew where to buy IR light sources in Prague…

4am is approaching and I’m very frustrated. I’m building a Microsoft Surface style multitouch screen, and maybe after all the software part will be more challenging than hardware… (more on the hardware some other day)

Quick intro to multitouch: You have a sheet of (plexi)glass on top of a box, and behind it an IR camera and a projector (or just IR camera if you aren’t going to show anything on the surface yet). Usually, you also have a bunch of IR leds shining in certain way (setups differ here), and your fingers touching the surface create light blobs in the IR camera input, and a software processes these blobs and converts them to useful input.

Now, unless you have fancy hardware setup, this input is very low quality and you want some good software to make sense of the blobs. CCV (Community Core Vision, formerly “tbeta”) is quite a fancy piece of software that should make your hardware prototypes easy to setup and experiment with, giving somewhat useful results even for extremely rudimentary setups. The problem is getting it talk to your IR cam on Linux, of course. Enter: hell.

tbeta-1.1 will refuse to select the correct pixel format for my camera, and there is about nothing I can do about it since it is closed-source. Now, the awesome folks of NUIGroup released the source as CCV-1.2, and I really appreciate that even though I sound frustrated from the build problems. The binary version will see no video devices at all – nada, none. Now, CCV is build around OpenFrameworks, which uses about 4357 other libraries. Some of them unpackaged, yay. And the only way to build it is using a Code::Blocks IDE (never heard of it before, either).

I will try to sum up the required changes to build everything on Debian on the NUIGroup forums and put a link here. It was very frustrating journey, also because I had to dust off my anyway-nonexistant Debian packaging skills and met utter user-unfriendliness here – I wanted to use dh_make and dpatch to do all the hard work, and it appeared to do so. Back-stabbing me with two mysterious weirdnesses (if it at least wouldn’t look so friendly, I would read the docs more carefully or look on): dpatch-edit-patch requires -0 argument to do anything actually useful, and dh-make will prepare the install rule with #dh_install commented out, making you scratch your head on mysteriously empty generated packages.

So, in the end I managed to so-so package the oscpack library, after heavily patching it to even make it compile(!)… CCV has so many library dependencies that it ships with many of the libraries included in binary form – but only 32bit, and I’m compiling on 64bit. So far before going to bed I ended up at:

../../../libs/FreeImage/libfreeimage.a(BitmapAccess.o): In function `.L309':
BitmapAccess.cpp:(.text+0xc10): undefined reference to `operator new(unsigned int)'

I’m ending up just heavily editing the build project and wondering whether it would be best to just write some makefiles. I really do wonder if I manage to build this devious thing eventually, but I’m growing determined. I kind of look forward to contribute patches to clean this stuff up.