Archive for the ‘Programming’ Category

Pantomime 2.0 in c’t 10/2012

Monday, April 23rd, 2012

The six page article about gesture recognition has finally come out in c’t issue 10/2012. The magazine is seen as one of the most prestigious technology journals in the German-speaking world with a range of over 1 million readers (Source: Wikipedia).

The article is written in German, here a brief abstract:

“In der von Nintendo, Sony und Microsoft beherrschten Konsolen-welt gehört die Steuerung per Körperbewegung bereits zum Alltag: Von der Pflege des virtuellen Haustiers bis zur Gymnastikstunde im Wohnzimmer gibt es reichlich Einsatzmöglichkeiten. Bei Mobilgeräten wie dem iPad bleibt dieses Potenzial bislang weitgehend ungenutzt. Dabei fällt mit den passenden Frameworks die Programmierung passender Anwendungen gar nicht schwer.”


Links to additional content:

Video: Visuelle Gestenerkennung @ Macoun 2011

Tuesday, January 3rd, 2012

The video of the talk about Visual Gesture Recognition (Visuelle Gestenerkennung) is now online and available from and on iTunesMacoun is the largest iOS and Mac OS X developers conference in the german-speaking world and takes place on an annual basis in Frankfurt, Germany. This session was held on October 2nd, 2011 and is in German.

Macoun Video

The game demonstrated in the end has been further developed and is available now: Meteora+ on the App Store.

Meteora+ hits the App Store

Sunday, November 13th, 2011

The first gesture-controlled 3D airplane game has finally landed in the App Store. The objective is simple: Board the plane and collect the portals in the magical forest!

You can tilt your device like a steering wheel or, on devices with a front-facing camera (such as the iPhone 4/4S or iPad 2), even use gestures to manoeuvre the plane.

Gesture control works by tracking two objects and their respective position. Object recognition is based on color, so ensure the objects tracked have a distinct color from everything else in the picture (face, background, etc.) and you are in an evenly lit environment. For this purpose, colored gloves seem to work particularly well.

Gesture Control in Meteora+

The game, originated from two University projects, brings camera-driven interaction to mobile devices in the form of an addictive arcade game for people of all ages. An early preview of this still very young technology was shown in Frankfurt am Main at Macoun 2011, the biggest iOS and Mac developers’ conference in the German-speaking world.

Meteora+ ($3.99) is now available on the iTunes App Store.


That was Macoun 2011

Tuesday, October 4th, 2011

Macoun 2011 is over and I just arrived back in London. It was my first time and I was impressed with the quality of speakers and talks, people’s incredible openness and how brilliantly it was all organised. Also, thanks everyone for their lovely feedback on Twitter. I tried my best to retweet the most relevant tweets regarding gesture recognition, many more can be found using the #macoun hashtag.

The whole session was also recorded on video and will be posted soon. Meanwhile, here are some first pictures and the slides of my talk to download:

Norbert M. Doerner, @cdfinder Norbert M. Doerner, @cdfinder

photo by @toastedtoast

photo by @kopf_marcus

photo by @kopf_marcus

photo by @futuretap

Last but not least, the video recorded on stage. Thanks Chris, Thomas et al. for this wonderful thing you created!

Gesture Library at Macoun 2011

Monday, September 26th, 2011

Less than a week left till the biggest iOS and OS X developer conference in the german-speaking world starts. Macoun 2011 will take place on the first weekend of October in Frankfurt am Main, Germany. I’m proud to be one of the speakers in such an amazing line-up. My session is going to be on Sunday, 11 am (Grosser Saal). So for those developers who still want to sign up and attend, today is the last day!


The talk is going to be about the Gesture Library on the iPad 2 and is going to focus on the concepts and technology behind it. We will also look at some code samples. Here an abstract of the talk (Source:

Visuelle Gestenerkennung

Gestengesteuerte Interaktion mit einer virtuellen Umgebung — ganz im Stile von Minority Report — ist dank Front-Kamera auch auf dem iPad 2 möglich. Anhand einer Beispiel-App werden die spannenden Möglichkeiten solcher Anwendungen gezeigt und einige dahinterliegende Technologien und Konzepte erläutert.

I still have a lot to prepare, only so much: People who were waiting for Meteora might really like it :)

Procedural modelling in Houdini based on Function Representation

Tuesday, August 16th, 2011

Some of you might have followed the progress of my master’s project in the devlog. The project is finally done and it has been a great learning experience.

“In modern computer graphics, objects are mostly represented by boundary representation models like polygonal meshes. Such models only store information about an object’s boundary and are relatively easy to render and often highly scalable. While this is sufficient for a variety of applications like many types of computer animation and games, other uses require information about volume rather than just surface. Function representation (FRep) allows defining objects as a set of geometric primitives with certain operations and relations. Since objects are defined as mathematical functions as opposed to a list of points, models are resolution independent and can be polygonised at any desired level of detail.

Building upon the current library developed at the NCCA and its Maya plugin, FRep mod- elling functionality has been integrated into Houdini and its node-based environment. The library is developed in C++ using the Houdini Development Kit (HDK) and comes as a set of custom nodes.” (Abstract taken from the thesis)

Basic FRep setup in Houdini using built-in primitive types

Bounded blending

Penguin FRep model

Metamorphosis between Penguin and Screw

Download Thesis (PDF, 4.7 MB)

Master’s Project — Dev Log

Tuesday, June 7th, 2011

The first three terms of my Master’s course are done and only the final dissertation is left to do. My topic is about the creation of an FRep Modeller in Houdini using the Houdini Development Kit (HDK). This post will be be used as a personal log, documenting key decisions, progress and solutions to problems encountered on the way to completion.

I officially started work on the June, 7th. Read the whole article to find out more.


Rendering a Chocolate Ball using RenderMan

Sunday, May 8th, 2011

For the RenderMan part of the CGI Tools assignment, a simple real world object had to be re-created and rendered. The tasks were to analyse the object and apply the correct techniques in terms of displacement, shading, lighting, etc. The report documents these steps and the methods used. In order to produce two final images, a Ferrero Rocher chocolate ball has been created from scratch and placed in a scene.

The renderer used is 3delight and can be installed on Mac, Linux and Windows for free. Also see my TextMate bundle for rib files and 3delight.

These are the final renders:

Rendered Scene A

Rendered Scene B

Download Report

Download Source

RenderMan TextMate bundle

Saturday, April 23rd, 2011

Doing my first steps with Pixar’s RenderMan I was installing 3delight on my Mac and started playing around with shaders and rib files. To simplify development I created a little bundle for my text editor of choice, TextMate. 3delight comes with the commands shaderdl and renderdl to compile shaders and render rib files.

Using the bundle, all you have to do is press Cmd+R when editing your rib file. The file automatically gets saved and a window pops up showing the status of the renderdl command. Also, the rendered image is displayed. The bundle also supports a slightly better syntax highlighting (based on the C bundle).

Renderman TextMate Bundle


  1. Download the bundle and double-click on the .tmbundle to install
  2. Set the variable DELIGHT In TextMate’s preferences => Advanced => Shell Variables to the path of your 3delight installation, e.g. /Applications/Graphics/3Delight-9.0.84 (You can find this out by typing echo $DELIGHT into Terminal)

Feel free modify, improve, tweak it to your taste. Feedback welcome! N.B.: Your rendered image has to have the extension .tif and has to be named like the the rib file for the preview to work. If you want to use other formats, you can change it directly within the bundle editor.

Download: TextMate RIB bundle

Gesture Library + Head Tracking prototype on iPad 2 [Update]

Monday, April 18th, 2011

I’ve spent a couple of days porting my gesture library to iOS to do some tests with the iPad 2’s front-facing camera. I also added head tracking with OpenCV for Head-Coupled Perspective (HCP) and a sort of 3D effect. It’s an early prototype for an ongoing project at the NCCA, still a lot to do, but I thought I’d give you a short preview. Stay tuned! The system is implemented in C++ and some Objective-C (for iOS glue code), powered by openFrameworks.

Update: New video showing an updated version with some new functions like the feature in the end that could be used for some sort of AR shooter game.

Update 2: Thanks to Renato Mitra for featuring this video on his site, one of the most popular blogs in Switzerland. Salü Äpflblog-Bsüecher :-)