Visualchemy lab notes

The trailer from latest project – dance theatre performance ‘Transkrypcje – Hommage pour Chopin” by Sopocki Teatr Tanca. Probably the most complex project I’ve done in Quartz Composer so far. On the outside we have visuals, that react to live cello and dancers’ movements, use IR camera and are controlled live by MIDI to fit timely to live music and dance. And did I mention that all the elements of stage are mapped using 3d model of the stage that can be calibrated to almost any projector and stage dimensions? All that done from one projector and five years old Macbook Pro, using custom build Quartz Composer system.

Photo by Imagination

It’s official now – the installation I’ve done for Imagination is up and running in Vancouver’s Samsung Area. Here’s a bit from Samsung’s official press release:

“Wirelessly transfer photos taken from Samsung’s Omnia II phone to the large Share Wall (approximately eight square metres). Visitors can capture and share their memorable WOW moments and post it on the photo mosaic frame which will be constantly updated with new images.”

Share Wall and the whole Samsung space concept, art directing and construction were done by Imagination – an incredible company, I had pleasure working with during my last few weeks in London. My job was to build an application in Quartz Composer that would pick up the freshly transfered images from server and display it on the big screen as a mozaic of pictures. In Vancouver the installation was running in realtime at 1080p (Full HD) on Mac Pro, but was designed and tested on iMac which was handling 720p pretty well (steady 30fps). The programming was done in basic Quartz Composer 3.0 with severe use of JavaScript and Timelines. Optimisation done with a help of Kineme’s PerformanceInspektor.

Some experiments with optical flow in Quartz – no interface this time.

More fun possible thanks to Vade’s optical flow ports to Quartz Composer.

Feedbacks, difference modes plus some OF controlled, bouncing sparks….

Reaching the performance and complexity limits within Quartz and seeing things possible in alternate frameworks, made me look out for the additional tools to master. It seems that in the area of relatively simple image manipulation there are two main projects out there worth considering: Processing and openFrameworks. Both used by acclaimed visual artists and both with active communities around them. Both require to master some programming skills and effort. Both are free and work on all the platforms (OSX, Windows, Linux)

Let’s have a look at differences (some of the being just mine better or worse informed conclusions):

Processing:

+ based on Java
+ works the same way on all platforms
+ able to compile to desktop applications or web applets
+ easier to learn
- slower
- more limited

openFrameworks:

+ based on C++
- there are differences between OSX and Windows libraries
- able to compile only to desktop applications
- more difficult
+ as fast as it gets
+ able to bridge with Quartz environment (very promising steps were made by another hero in community Vade are described here)
+ not much of limits here

Although Processing seems to have more pros, there’s not a straightforward, general answer to ‘which one is better’ here. I guess it all comes down to question: What do I want to do with it? I’m generally thinking about employing it to enhance my current Quartz projects and be able to do new stuff with particles and video analysis. This directs me more toward openFrameworks solution. So I guess its OF then..

In November 2009, I’ve been commisioned to develop a video installation for Samsung. It will be a part of the Olympic Rendezvous @ Samsung – “Official Meeting Place of Athlethes and Their Families” at Vancouver 2010 Winter Olympics. The space has been designed by Imagination which does art directing for the whole project. The installation is now going through final tweaks and I’ll publish more on the topic as soon as its up and running.

This exciting project has been commisioned by Matt Black (ColdCut). Using Quartz Composer, I’m developing a new, real time video synthesizer, which Matt will be able to incorporate into his audio visual shows. More info and videos will follow soon..

Uprojected CV is the latest version of Uprojected video installation – this time fully controlled by Computer Vision algorithms, analysing user’s movements and creating an additional control layer. Themed specificaly for the ‘Artful Badger’ event, it features pawn shaped buttons and scratch pad sliders In terms of hardware it was based on IR-enabled camera and IR light (visible above the screen).

As soon as I got my hands on Vade’s Optical Flow plugin for quartz, I ported it to VDMX. This is the second version of a plugin with improved interface and additional features added to enable performance/quality tweaking.

To install the VDMX plugin, you need to download and install Vade’s Quartz plugin first from here:

http://002.vade.info/?page_id=28

After that just download the VDMX plugin below, unzip and copy the qtz file it into qcfx subfolder.

va OFdisplace2

The outcome of the private view was this short video showcasing the installation and out friends’ reaction to it. Video was edited from recorded footage by Patrycja Grimm

I’ve had the optical flow interface in mind since I first learned about this method. Unfortunately, the algorithms provided in Quartz’ Examples folder where too slow and innacurate to work. To be fair, back then my knowledge of QC and programming was quite lacking too. The breakthrough came with toneburst porting Andrew Benson’s simplified, shader based OF calculations to Quartz. Another reason to push the OF interface further, was the invitation from Artful Badger collective to bring my video installation ‘uProjected’ to their tent at Secret Garden Party. Bored with using the wiimote as a controller, I’ve decided to base the installation fully on optical flow. First I planned to keep the installation within the VDMX  and write a Quartz plugin that would bring an interface overlay and send out the signals as MIDI to VDMX. The performance hit turned out to be to high, so I had to create the project fully inside Quartz.