NodeBeat was created by Seth Sandler and Justin Windle earlier this year and released in April for the iOS platform. It is an experimental node-based audio sequencer and generative music application. By combining music and playful exploration, NodeBeat allows anyone to create an exciting variety of rhythmic sequences and ambient melodies in a fun and intuitive fashion.

How Does it Work?

Octaves and Notes make up the two types of nodes. Octaves pulse and send messages to Notes within proximity. Each Octave is assigned a random octave and each Note, a random note; therefore, a Note will play in several octaves depending on the Octave it’s connected to. Pause nodes to create your own beats or let them roam free to have them generate their own.

Cross platform development

Because NodeBeat was developed using the C++ based open source framework openFrameworks, I did not expected a lot of trouble getting the core to work on Android. However, since the Android port of openFarmworks is still pretty new (we’re using the development branch) and officially only supported on the Mac and Linux platforms, I decided to put some effort into making it work on Windows as well. I’m a Windows user and developer, so if I can avoid dualbooting, I will .

Native Development Kit

As soon as you want to use C or C++ in your Android projects, you will have to install the Native Development Kit (NDK). It basically allows you to compile your code into a library which you can access using JNI calls. While in general it is recommend to code using the SDK in Java for your Android projects (the Dalvik VM with JIT show really good performance), lazy coders (like me ) are always trying to find ways to reuse existing code. Instead of having a native codebase for iOS (in Objective-C) and Android (in Java), it is nicer to have a shared core in C++ with a thin layer (Obj-C or Java) to interface with it. Sure the NDK might sound intimidating at first sight and Google doesnt recommend it unless you know what you’re doing, but honestly I don’t think its rocket science either. After downloading the NDK, you will need to setup a unix like environment such as MinGW or Cygwin.

For my previous projects I already had MinGW installed (you could use Cygwin, but in general I don’t like their approach). I did a fresh checkout from Github and started to mess around with the Makefiles to see if it would compile.

It turned out that all I had to do is replacing a few IF statements (the ones that are checking the build platform) and make them point to the NDK location on my Windows computer. I’ve created a tutorial that explains the steps if you want to try it out yourself. However, if you want to use openFrameworks for your own Android applications, I would highly recommend just using my openFrameworks fork instead (until they accept my pull request). It includes all the patches from the tutorial and it should be compatibly with the latest NDK. The tutorial actually also explains how to run one of the examples so be sure to check that out.

Porting the GUI

For the UI I wanted to stay as close to the iOS version as possible. As I don’t own an iOS device, Seth gave me some screenshots of NodeBeat running on iOS which I used as a reference.

Since the iOS and Android framework are quite distinct, there are cases where I had to do an alternative implementation. For example, on Android most devices have the following buttons:

• Back
• Menu
• Home
• Search

1
2
3
4
5
6
7
8
9
10

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
    <item android:id="@+id/node"
          android:icon="@drawable/node_icon"
          android:title="Node" />
    <item android:id="@+id/audio"
          android:icon="@drawable/audio_icon"
          android:title="Audio" />
          ...
</menu>

Example: Audio menu

Example: Recording dialog

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

AlertDialog.Builder builder = new AlertDialog.Builder(this);
builder.setMessage("You can stop recording by pressing the record button in the menu again")
       .setCancelable(false)
       .setPositiveButton("Yes", new DialogInterface.OnClickListener() {
           public void onClick(DialogInterface dialog, int id) {
                // Start recording
           }
       })
       .setNegativeButton("No", new DialogInterface.OnClickListener() {
           public void onClick(DialogInterface dialog, int id) {
                dialog.cancel(); // Cancel dialog
           }
       });
AlertDialog alert = builder.create();
alert.setTitle("Start recording");

Using JNI Callbacks

After we’ve ported the UI, we still need to pass on our settings to the core application. Fortunately we can use JNI callbacks to get and set NodeBeat’s properties. It is good to know that you should minimize the number of JNI callbacks for performance reasons (so don’t go mental and call tons of JNI methods each time you render a frame).

Let say we would like to pass a value from one of the sliders from the Audio context menu to our NodeBeat core. In such case we first create a new static method in our Java source file:

OFActivity.java

1

public static native void sliderChanged(float v);

It is important to use the static and native keyword when you define your method. This is all we need to do in our java code and this method can be used anywhere in our java class.

Now the tricky part is how to implement the function on the C++ side of your application. It’s not exactly complex, but you will have to pay attention to a few details. Three things are important here:

1. The namespace (in Java)
2. The class name
3. The method name

If we assume we’re implementing this callback in one of the openFrameworks examples, this means:

1. The namespace (in Java): cc.openframeworks.androidEmptyExample
2. The class name: OFActivity
3. The method name: sliderChanged

main.cpp

Here is the code you implement in C++

1
2
3
4
5

#include <jni.h>
extern "C"{
void Java_cc_openframeworks_androidEmptyExample_OFActivity_sliderChanged (JNIEnv*  env, jclass  thiz, jfloat value){
// Do smth here
}

As ugly as this method looks like, take a brief look at how it is constructed. It starts with the return type (which is void just like how we specified it in Java). Next it starts with Java_ and is succeeded by the namespace, classname , method name. All dots from the namespace are replaced by underscores and between each element we place an underscore as well.

In the arguments list, the “JNIEnv*  env, jclass  thiz” part is mandatory (so if you have a something like this: public static native void methodname(), it would be void methodname(JNIEnv*  env, jclass  thiz)). For our method we want to pass a float as argument. For some reasons you can’t just pass a float, you will need to use JNI mapping types. The float becomes a jfloat.
Note: for booleans you need to compare the value to JNI_TRUE or JNI_FALSE and not to true or false.

Honeycomb Tablets

Unlike the iOS devices from Apple, Android devices run at so many different configurations and API levels, it can be a bit tricky to support all of them. For NodeBeat we decided to create two versions: A phone version and tablet version. We basically distinguish between phones who are running Eclair or Gingerbread (2.2+) and tablets running Honeycomb (3.0). On the Android Market it seems like we only provide one version, but depending on what device you’re using to download the app, it will give you a certain version.

In order to maximize the use of the display, I’m using this line in the AndroidManifest.xml file:

1

android:theme="@android:style/Theme.NoTitleBar.Fullscreen"

On phones it sets the application to fullscreen. While this works for anything running Android 2.2+, it is a problem on Honeycomb devices. Honeycomb tablets don’t have any physical buttons and when the application is running in fullscreen mode, there is no shortcut to the menu button. This shortcut is normally placed in the top right corner.

This means that for the tablet version, we run the application in regular display mode.
Other than that, NodeBeat provides a rich user experience on tablet devices such as the XOOM or Asus transfomer.

Earbleeding masterpiece created by Sharath Patali (professional coder, horrific musician)

Android Market

Publishing the app to the Android market is no hassle at all (we were just a bit unfortunate and had our app pulled down by accident). There is no annual fee (just a one time 25$) and apps are approved instantly. The dev guide provides a comprehensive overview on how to build your project in release mode and how to sign it.

Go get it!

NodeBeat is available on the Android market:
https://market.android.com/details?id=com.AffinityBlue.NodeBeat

Try it out! It’s just a dollar

Website: http://nodebeat.com/
Twitter: http://twitter.com/NodeBeat

Of course, there is always the debate whether it is legal to build an emulator. Console manufacturers will always forbid any type of emulation of their systems. Although its a rather grey area as most documents on the chip design are publicly available (e.g. MIPS / ARM instruction sets).

Since I got excited about emulation in general, I started to do some research on how to build one myself. As I didn’t want to spend days or even weeks to get some results, I looked for a simple system. A friend introduced me to a small system called the Chip 8.
The Chip 8 itself didn’t existed as a hardware system, but was implemented in software. Due to the small instruction set (~30 opcodes), it was a very good candidate to work on as a first emulation project. The instruction set defines the functionality of the CPU. It for example contains instructions that allow the CPU to load or store data, but also perform mathematical tasks such as multiplications.

Having worked on a couple of emulation projects (and emulator plugins), I started to write a guide that will hopefully help out aspiring emulator authors and inspire emulation enthusiast. The guide explains how emulators work and provides a detailed overview on how to write a Chip 8 interpreter from scratch.

Writing a Chip 8 emulator shouldn’t be too time consuming. On average people tend to finish such project in one or two days. It’s a fun project to test out your programming skills and of course quite educational (and it isn’t just for CS majors )

If you’re ready to take up the challenge, click the link below! Have fun!

• How to write an emulator (CHIP-8 interpreter)

Summary:
“Earth Friends is a free Facebook application to visualize your social network on Google Earth. Locate your friends by using the Google Earth Webplugin or download your personal datafile for use with the Desktop version of Google Earth.”

Rewriting…

While my code from last week was running fine, there was a lot of room for improvement. Basically three major parts changed:

1. The database structure
2. Using a template engine
3. File compression

MySQL

The first thing I worked on was the database itself. The local database is primarily used for converting a location (defined by a city, state and country name) to a latitude and longitude.
Each time we need a location, we query the database, simple as that.

So let’s look at the following example:
First assume you have about 100 friends on Facebook. Probably most of them share their current location, but some of them might not want to share their location with Facebook apps or simply did not filled out the location at all.
If 75 friends filled this field with their current location, the old code would query the database 76 times (75 times for your friends + 1 time for your location).

While this looks like a lot of queries, the time for a query to complete also depends on the design of the database. As I was using MaxMind cities database, I initially imported all the data into one MySQL table. Just because it was convient to work with. However, the dataset has about 2.699.356 entries (cities).
Doing a search 75 times within this table was not going to be fast…

Besides the users’ patience, I am also limited by PHP’s execution time limit. I think that scripts are often allowed to run for 30 or 60 seconds, although I can’t stand a website that is taking longer than 10 seconds to show up.

So what to do next? Can we find a way to split the database into multiple small tables? Just limiting the number of entries to a fixed number (e.g. 100.000) wasn’t going to work as I would need some kind of lookup system in order to figure out where I could find my city.

Improvement one
The easiest solution I found for it (not saying its perfect) was by splitting the database by country. In my case I have 231 countries in the database, thus 231 tables are created.

Taking a look at the top 3 countries in our database reveals the following:

• Russia – 176.934 cities
• USA – 141.989 cities
• China – 117.508 cities

The average number per country is around the 20.000 cities. While the top 3 shows a significant higher quantity than the average, look ups are performed much faster than before.

Improvement two
We could’ve stopped here with improving the database part as the results were reasonable. However, it is a small effort to tweak it a bit more.

In our example, we query 76 locations from the database. But wait… what did we query? The location of friends! And what do a lot of friends have in common? Right, they share the same location. So by first creating a list of cities we need to look up, we can reduce the number of queries required to collect our data.

Smarty Template Engine

While PHP provides a nice environment for rapid prototyping, it can also become a mess easily. Using print or echo statements is fine for testing purposes but it is better to keep code separated from HTML.

This is why you need to use template engines. When using a template engine, you first collect data from your database and then parse your variables and arrays to the template engine. In your templates you specify where this data need to be placed. (Summary: PHP is used for data collection and prepared into arrays, Template engines use these arrays and are used for the actual design of the website).
Fortunately, there are free open source template engines such as Smarty.

Take a look at the following code:

1
2
3
4
5
6
7
8
9
10

{* Add placemark for friends *}
{foreach $friendlist as $friend}
  <Placemark>
    <name>{$friend.name}</name>
    <styleUrl>#{$friend.uid}</styleUrl>
    <Point>	
      <coordinates>{$friend.lon},{$friend.lat},0</coordinates>
    </Point>
  <Placemark>
{/foreach}

This is an actual code snippet from the template responsible for generating the KML. This particular section is used to display your friends icon on Google Earth.

Within the template I can specify a block (in the example its a Placemark) that will be looped. After setting up Smarty and collecting data from the database, I pass the result ($friendlist) to Smarty. In the template engine, it will now perform loop through this foreach loop and place the variables in the correct location.

A few more benefits of using a template engine is that you can store the results in a cache. By caching the results we can skip ‘expensive’ MySQL queries if we know the page hasn’t changed. By specifying the cache lifetime (for example 30 minutes), we can make sure that Smarty will regenerate the page if the cached page is older than 30 minutes.

ZIP

In the previous version of Earth Friends, I embedded the KML file into the header of the website (in Javascript). While this method works fine for small data sets, it has a large impact on page loading and render times when data sets are growing.

KML files are plain text files formatted in XML. Besides KML files, Google Earth also accept compressed KML files which have the extension KMZ. KMZ files are basically KML files compressed with ZIP.
Tests show noticeable differences in loading times when using KMZ. For example, my test data set in KML was about 693 KB. After compressing this file using zip (max. compression), the size was reduced to 92 KB. That’s around 13% of the original file size! As a result, loading times were reduced significantly.

Where can I find the application?

Ready to try out this application on facebook? Just click the following link to open up Earth Friends: Earth Friends application on Facebook.

After authorizing Earth Friends to access your profile data, it will reload the page and launch the Google Earth browser plugin. If the plugin is not installed, please follow the instructions that are displayed instead. The plugin should work in Windows and Mac OS X.

Important: Make sure you set your own location (with the correct privacy settings) or the curves will not appear!

More information can be found on Earth Friends profile page on Facebook.

How to use this?

To help you get started with Earth Friends, I have created two screencasts which demonstrate you how to add Earth Friends to your Facebook account and use the application.

Tutorial 1: How to use Earth Friends (View in 720p HD)

Tutorial 2: How to download the KMZ file for Google Earth Desktop (View in 720p HD)

Because I’m using Facebook to connect with my friends, I decided to dig into the documentation of the facebook APIs. Apparently there are multiple ways to get hold of your and your friends information. The most commonly used APIs are: Graph API / FQL . The first one lets you retrieve information about a friend or page by loading a specific URL, the second one lets you actually send a SQL query to retrieve the  information.

Since I want to make this a hassle free experience, I decided to make a Facebook application which would use the Google Earth Web plugin. This way, users only need to download the plugin, but everything works just in the browser.

How does it work?

Basically a Facebook app is just a website running on some server. In my case, I’m hosting my application on the same domain as my blog. Since the application is embedded into the Facebook website, normal users won’t notice. The app itself can be written in all kinds of languages but for the sake of simplicity I used PHP.

First we need to connect to Facebook using an API/SDK. This allows us to authenticate and securely connect to the Facebook servers. After enstablishing a connection, we use FQL to query: Our friendslist and the location of our friends. Unfortunately the friendslist only contains the name of the location and not the geospatial coordinates.

Therefor I had to create a lookup database that would translate a City/State/Country name into a geospatial coordinate (latitude and longitude). This was done by downloading a free database from Maxmind.com.

Now we should have all data available to create our KML file on the fly. For now I embed the KML result into the javascript header which seems to work fine for ~200 friends. I still need to do some benchmarks to see how well this scales. A demonstration video of the result can be found below:

Video

Can I try?

Since this project is still WIP, it is not available yet in the Facebook Application Directory. I’m planning to release this application for free soon.

While it is interesting to see the spread of a certain day, it is even more interesting to see how the spread is evolving over time. By keeping track of updates of the mirror page on Wikileaks, I was able to collect enough data for an animated version of the spread. My current dataset contains a 7 day period covering December 5th to December 12th.

As some of the commenters pointed out, the edges (curves) in the previous dataset didn’t always followed the shortest path. This was due to the (simple & stupid) algorithm I was using to draw the path between two points (basically just the mathematical shortest distance). In the latest dataset (download link is at the bottom) this is corrected. In particular this website: http://www.movable-type.co.uk was very useful for figuring out the the correct path.

* update 21-12-2010 11pm CET *
- Got featured on ReadWriteWeb, Thanks!

* update 23-12-2010 10pm CET *
- Embedded Google Earth web plugin in demo section

Mirror spread

The result of plotting the spread in a line chart:

Top 10 locations

An overview of the spread based on country:

1. Germany: 498
2. United States: 394
3. France: 194
4. Netherlands: 152
5. United Kingdom: 72
6. Sweden: 67
7. Canada: 49
8. Spain: 47
9. Switzerland: 36
10. Russian Federation: 32

Screencast / Video

Short screencast (Watch it in HD 720p, in fullscreen mode)

Online Demo

To view this demo you will need to install the Google Earth Browser Plug-in

Note: You need to move the range marker all the way to the left to make the timeline work:

• Move the time slider all the way to the right
  
• Move the range marker (the small attachment on the left of the time slider) all the way to the left
  
• Now you can move the time slider as you want
  

KML source

If the online demo doesn’t work for you, you can also try it in Google Earth!

• Make sure you’ve installed Google Earth.
• Next download and unzip the file following file:
• Wikileaks Google Earth KML files (Animated) (222)
• Open up the KMZ file.
• Enjoy!

Whether Julian Assange should be considered as a Hero (by publishing information) or Terrorist is an open question. Because of threats from governments, companies in the USA started to ban payment, hosting and dns services to Wikileaks. Soon Wikileaks moved from servers based in the USA to Sweden/Switzerland.

For now Wikileaks seem to be safe, however, they also started a call for mirrors.

Interested to see which people in the world are supporting Wikileaks, I had the idea to visualize the Wikileaks mirrors on Google Earth.

* update 09-12-2010 8pm CET *
- Added Google Earth web plugin links

* update 10-12-2010 10am CET *
- Got featured on ReadWriteWeb / NY Times, thanks guys!
- Stay tuned for more updates!

* update 23-12-2010 10pm CET *
- Embedded Google Earth web plugin in demo section

Data mining

In order to get the data, I went to the official Wikileaks website which had all mirrors listed on this page: mirrors. I wrote a small PHP script that would open up the mirror page and scanned the document for URLs. If a mirror was found, it would be stored in a file. At this moment there are about 1334 mirrors on the website.

Data manipulation

At this point I only have the URLs of the mirrors, but how do I know where these servers are located?

In order to know where the server is located, I used GeoLite City which is a service from MaxMind. GeoLite City allows you to resolve most IPs to a geospatial location. Of course it doesn’t gives the exact location, but usually it is able to show in which City the server is located, which is good enough for my purpose.

After obtaining the GeoLite City database (there is a free version!) I used PHP to write a script that would first resolve the URL to an IP address (PHP function: gethostbyname() ) and then use this IP address to look up the longtitude and latitude.

The last step was converting the data into the KML format (and adding some artificial altitude information) for use with Google Earth.

Data visualization

Below are some of the early results.

Currently the main server (http://46.59.1.2) seem to be located in Sweden, when we view the data in Google Earth we can see that a lot of mirrors are actually located in Europe.

Europe
Red pin: Wikileaks server
Yellow pins: Wikileaks mirrors
Greenlines: Connections from the Wikileaks server to a Wikileaks mirror

USA
Wikileaks mirrors in the USA

Sweden
Since some of the servers are at the same location (probably sharing the same data center), we can click on a pin and it will expand to show all mirrors located at this data center.

Screencast / Video

Short screencast (Watch it in HD 720p, in fullscreen mode)

Online Demo

To view this demo you will need to install the Google Earth Browser Plug-in

KML source

Want to try it out locally on your computer?

No problem! Here is how:

• Make sure you’ve installed Google Earth.
• Next download and unzip the file following file
  Wikileaks Google Earth KML files (2942)
• Open the KML files one by one.
• Enjoy!

Recently I started to organize the footage I made and collected over the years and decided to put some of it online. To maintain the overview, a new section has been added to the site: Research Projects. It includes most of the projects I have been working on at the University of Amsterdam (UvA) and Harvard University.

Each project includes a short description, pictures and a video. Enjoy!

(Click the project title for more information)

Interactive Networks

This project introduces the Interactive Network concept and describes the design and implementation of the first prototype.

Twilight

Twilight is an interactive graph exploration tool for multi-touch systems. Twilight provides a flexible environment that can be used to visualize and analyse graphs and networks found in the computational science.

Treepanorama

This projects involves the visualization of large phylogenetic tree structures such as the ones found in the Tree of Life. By combining high performance computer graphics with multi-touch input interaction methods, his project will create an interactive exploration environment that allows us to view the data interactively and in different representations. This research will lead into a better understanding of the evolutionary tree.

Involv

INVOLV is a research project that combines cutting-edge interactive technology with emerging information visualization techniques to create innovative explorations for large hierarchical data sets

Phylogenie

This application is designed to be a collaborative activity to teach undergraduate students about phylogeny and to prevent misconceptions about evolution. The system guides the students through a set of steps required to construct a phylogenetic tree based on morphological and DNA sequence data.

Since this project is still active, more media content will be released in future!

* update: 27 November 2010 *

Old footage from personal projects: Touch tracer and Real time fluid dynamics running on the UvA-MTT

Touch Tracer v0.3

Real time fluid dynamics

So what exactly is the Kinect? Is it similar to the PS2 eyetoy?

Actually it is much more advanced than the PS2 eyetoy. Unlike the eyetoy, the Kinect contains 2 cameras. One RGB camera which is used for ‘normal’ images and a depth camera is used to figure out the position of objects in its view. A nice explaination (and an overview of the components) can be found at ifixit.

Because Adafruit required the contest winner to open source his source code, this means that others can now enjoy hacking their Kinect as well! Currently the code is available at github and renamed to libfreenect (irc: #openkinect @ freenode): https://github.com/OpenKinect/libfreenect/

Getting the code to compile might be a bit tricky and it involves CMake to create the project files. Running it on Linux is very trivial (just make sure you’ve installed all dependencies), on Windows and Max OS X there are some extra steps involved to compile the library and demo application.

For the windows version you will need libusb, glut and pthreads. Also, don’t forget to select the win32 branch when you do a checkout from git. After creating the visual studio project files you will need to manually fix the path to the include and lib directories (The current CMake file is broken).

If you can’t be bothered, I have compiled a windows binary (vs2k8): Kinect-VC9-Release.zip

Before running the binary, make sure you’ve installed the drivers from github (First the XBOX motor, then camera and audio). To control the motor in the Kinect, you can use this code: NUI_Motor.cpp

My experiments

Last Friday I actually bought myself a Kinect. In the video below you can see me running the Kinect on my machine. For now it is just retrieving both camera streams and putting it on display.
Basically the library (libfreenect) is providing you with two images through a callback. One depth image and one RGB image. The depth image actually maps a depth to a certain color. In this video for example, the color red/white means something is really close to the camera and green/blue is further away.

Hopefully I will have some nice apps later this month .

Kinect hacks by others

3D mapping by Oliver Kreylos

ofxKinect 3D draw by Memo

Multitouch hack by Florian Echtler

Kinect Point Cloud by cc laan

Contest

P.s.: Before I forget, Matt Cutts (Google) started another contest. Check it out at: http://www.mattcutts.com/blog/open-kinect-contest/

Watch it in HD on Youtube or Vimeo!

Changelog for v3:

• Fixed compatibility issue with Adobe AIR 1.5 (Fullscreen bug)
• This version should autodetect your screensize to support any screen resolution.
• Added a Google Maps object
• Added support for adding pictures on the fly (for example if you want send a picture from your cellphone with bluetooth, read the readme.txt for the instructions)
• Added a exit-button, reset view-button, canvaslock and displaylock buttons.
• Temporary removed DCIM support due to slowdowns

Want to try it out yourself?
Download it now: Multitouch Media Application Pro v3 (12186) - 2.25 MB

Please read the install instructions in readme.txt (and download the latest Adobe AIR 1.5.x).

note: If you want to try out the bluetooth feature, I recommend installing BlueSoleil to handle the pairing of the devices and file transfers.

p.s. Sorry for the ending it wasn’t me!

A few weeks ago I made some adjustments to make it compatible with the latest touchlib and this is the result:

tDesk v2 allows the user to move and resize multiple windows at the same time by using multi-touch gestures. Because Windows XP/Vista was not designed for multi-touch there are some issues with this type of input. In multi-touch mode it is not possible to do a mouse click, therefor I added an option to switch between multi-touch and mouse simulation mode.

Please keep in mind that when the mouse simulation mode is activated only single touch input is allowed. Do not use this software on unstable multi-touch systems.

Finally, use this software at own risk .

Install instructions are in: readme.txt !

Downloads:
tDesk v2 (2596) - 13.32 kB
Touchlib (speedfix) (3162) - 785.69 kB – rev393

Any tips or problems can be posted in the comment section.

* update *
In some cases tDesk does not seem to work properly with touchlib. Try downloading rev393 and overwrite your current touchlib dll files.

A few weeks ago I was busy tweaking the Touchlib sourcecode and noticed a bug/feature in the code which caused the image processing pipeline to stall for 32 milliseconds. This might not sound like a big issue however on our system it caused the system to be less responsive.

I have compiled a version without this ‘sleep’ function. This version might not be safe for single core cpu’s (I could not test it out myself, hopefully someone could check it out). Because the sleep function is removed, the videoplayback (using cvcapture + *.avi) is not working properly.

Check out this version of touchlib and report any problems in the comment section !

Download:
Touchlib (speedfix) (3162) - 785.69 kB compiled libraries, revision 393 speedfix (21 apr 2008)

MMA Pro v2d

Changelist:
- Added smoothing, scaled images should look a lot better
- Added an View Reset and Exit button
- Increased the size of the canvas
- Compatible with Adobe Air v1.0

Install instructions are in: readme_pro.txt !

Download:
Multitouch Media Application Pro v2d (3155) - 2.22 MB
Adobe AIR – required

Example video showing how to load photos dynamicly into MMA Pro

Wong!

Wong is a game based on pong which uses the multitouch input to create dynamicly sized paddles.

A game of Wong on our multitouch table:

The game is made by Seth Sandler and is part of the audiotouch application suite: http://code.google.com/p/audiotouch/

One of the demo applications which comes with Touchlib is Smoke. The technique which is used to create ‘smoke’ is based on the algorithm presented by Jos Stam in his paper Real-Time Fluid Dynamics for Games.

Often multi-touch enthusiast use this application to demonstrate their multi-touch displays.

Some time ago I modified the Smoke application which allows users to draw objects into the scene. The application is extended to have a new drawing mode which is allow (multiple) users to draw objects/boundaries into the scene. This can be done while the simulation is running. If sources are placed in the scene this means you can view in real-time how flow will behave on a particular shaped object.

In the demonstration video I have added 3 different coloured sources and the logo of my university is used as an object to perform the simulation on.

A higher quality wmv can be downloaded here:
RTFD.wmv

Multi-touch video puzzle game

I had this idea for a while and even the basic puzzle game framework (using photo’s) was already finished. After messing around with FFmpeg I managed to export the video frames from an mpeg2 (.vob) file and convert them to an OpenGL texture. The only problems left were the synchronization and the audio stream. Because FFmpeg is really horrible coded (you need special hacks to get it even to compile under windows), I never managed to get those working. For those who are interested, this site: dranger.com has helped me a lot.

After searching around for an alternative I decided to drop the cross platform support for this application and use MS DirectShow. Usually I do not like the DirectX due its ‘clean’ coding style. But after giving DirectShow a try I was surprised how easy it was to get a basic demo working which played a mpeg2 file without any problems. The audio and video streams where synchronized automatically.
With some help of a friend I merged the video player code into the puzzle application.

This is the result (recorded with Fraps):

In the video I used the mouse as input device, but the application works with the multi-touch table as well.

And as usual: Don’t forget to read the readme_pro.txt !

Download:
Multitouch Media Application Pro v2c
Adobe AIR – required

Last saturday MMA Pro was presented on national television (NED3, VPRO) in the show called “In de ban van ‘t ding”. The documentary created by Dahl TV and Bright Magazine is about the (new) technologies which have an impact on our daily lives.

In this weeks episode Erwin visits SOCO Amsterdam. SOCO (Social Contemporary Center for the Arts) is a new exhibition/project space in Amsterdam. Peter Distol and Ralph Das are giving a demonstration of the table we have build together at the SOCO. The table uses front illumination and touchlib for touch detection and processing.

A part of the episode can be viewed here (it is in Dutch):

The whole episode can be viewed here:
Uitzending gemist

The applications which have been demonstrated:
- Smoke (basic demo from touchlib)
- Newswell (Ralph Das)
- Multitouch Media Application (showing Jimi Hendrix!) (Laurence Muller)

Currently there are two multi-touch tables available to the general public.

Multitouch Media Application Pro Update

The previous release of MMA Pro only worked with Adobe AIR beta1. A few users have contacted me about the troubles when installing MMA Pro v2a with Adobe AIR beta3 installed.

I have updated the application to make it compatible with Adobe AIR beta3 (Thank You Adobe! for renaming zillion functions).

The application requires Adobe AIR beta3 runtimes, so do not forget to install them first. Besides the fixes, there is nothing new.
Ofcourse this version also supports adding images on-fly. Just connect your digital camera (or memorycard) and all your images will be downloaded to the desktop. Your camera should be using the DCIM standard).

Don’t forget to read the readme_pro.txt !

Download:
Multitouch Media Application Pro v2b
Adobe AIR – required