NodeBeat was created by Seth Sandler and Justin Windle earlier this year and released in April for the iOS platform. It is an experimental node-based audio sequencer and generative music application. By combining music and playful exploration, NodeBeat allows anyone to create an exciting variety of rhythmic sequences and ambient melodies in a fun and intuitive fashion.

How Does it Work?

Octaves and Notes make up the two types of nodes. Octaves pulse and send messages to Notes within proximity. Each Octave is assigned a random octave and each Note, a random note; therefore, a Note will play in several octaves depending on the Octave it’s connected to. Pause nodes to create your own beats or let them roam free to have them generate their own.

Cross platform development

Because NodeBeat was developed using the C++ based open source framework openFrameworks, I did not expected a lot of trouble getting the core to work on Android. However, since the Android port of openFarmworks is still pretty new (we’re using the development branch) and officially only supported on the Mac and Linux platforms, I decided to put some effort into making it work on Windows as well. I’m a Windows user and developer, so if I can avoid dualbooting, I will .

Native Development Kit

As soon as you want to use C or C++ in your Android projects, you will have to install the Native Development Kit (NDK). It basically allows you to compile your code into a library which you can access using JNI calls. While in general it is recommend to code using the SDK in Java for your Android projects (the Dalvik VM with JIT show really good performance), lazy coders (like me ) are always trying to find ways to reuse existing code. Instead of having a native codebase for iOS (in Objective-C) and Android (in Java), it is nicer to have a shared core in C++ with a thin layer (Obj-C or Java) to interface with it. Sure the NDK might sound intimidating at first sight and Google doesnt recommend it unless you know what you’re doing, but honestly I don’t think its rocket science either. After downloading the NDK, you will need to setup a unix like environment such as MinGW or Cygwin.

For my previous projects I already had MinGW installed (you could use Cygwin, but in general I don’t like their approach). I did a fresh checkout from Github and started to mess around with the Makefiles to see if it would compile.

It turned out that all I had to do is replacing a few IF statements (the ones that are checking the build platform) and make them point to the NDK location on my Windows computer. I’ve created a tutorial that explains the steps if you want to try it out yourself. However, if you want to use openFrameworks for your own Android applications, I would highly recommend just using my openFrameworks fork instead (until they accept my pull request). It includes all the patches from the tutorial and it should be compatibly with the latest NDK. The tutorial actually also explains how to run one of the examples so be sure to check that out.

Porting the GUI

For the UI I wanted to stay as close to the iOS version as possible. As I don’t own an iOS device, Seth gave me some screenshots of NodeBeat running on iOS which I used as a reference.

Since the iOS and Android framework are quite distinct, there are cases where I had to do an alternative implementation. For example, on Android most devices have the following buttons:

• Back
• Menu
• Home
• Search

1
2
3
4
5
6
7
8
9
10

<?xml version="1.0" encoding="utf-8"?>
<menu xmlns:android="http://schemas.android.com/apk/res/android">
    <item android:id="@+id/node"
          android:icon="@drawable/node_icon"
          android:title="Node" />
    <item android:id="@+id/audio"
          android:icon="@drawable/audio_icon"
          android:title="Audio" />
          ...
</menu>

Example: Audio menu

Example: Recording dialog

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

AlertDialog.Builder builder = new AlertDialog.Builder(this);
builder.setMessage("You can stop recording by pressing the record button in the menu again")
       .setCancelable(false)
       .setPositiveButton("Yes", new DialogInterface.OnClickListener() {
           public void onClick(DialogInterface dialog, int id) {
                // Start recording
           }
       })
       .setNegativeButton("No", new DialogInterface.OnClickListener() {
           public void onClick(DialogInterface dialog, int id) {
                dialog.cancel(); // Cancel dialog
           }
       });
AlertDialog alert = builder.create();
alert.setTitle("Start recording");

Using JNI Callbacks

After we’ve ported the UI, we still need to pass on our settings to the core application. Fortunately we can use JNI callbacks to get and set NodeBeat’s properties. It is good to know that you should minimize the number of JNI callbacks for performance reasons (so don’t go mental and call tons of JNI methods each time you render a frame).

Let say we would like to pass a value from one of the sliders from the Audio context menu to our NodeBeat core. In such case we first create a new static method in our Java source file:

OFActivity.java

1

public static native void sliderChanged(float v);

It is important to use the static and native keyword when you define your method. This is all we need to do in our java code and this method can be used anywhere in our java class.

Now the tricky part is how to implement the function on the C++ side of your application. It’s not exactly complex, but you will have to pay attention to a few details. Three things are important here:

1. The namespace (in Java)
2. The class name
3. The method name

If we assume we’re implementing this callback in one of the openFrameworks examples, this means:

1. The namespace (in Java): cc.openframeworks.androidEmptyExample
2. The class name: OFActivity
3. The method name: sliderChanged

main.cpp

Here is the code you implement in C++

1
2
3
4
5

#include <jni.h>
extern "C"{
void Java_cc_openframeworks_androidEmptyExample_OFActivity_sliderChanged (JNIEnv*  env, jclass  thiz, jfloat value){
// Do smth here
}

As ugly as this method looks like, take a brief look at how it is constructed. It starts with the return type (which is void just like how we specified it in Java). Next it starts with Java_ and is succeeded by the namespace, classname , method name. All dots from the namespace are replaced by underscores and between each element we place an underscore as well.

In the arguments list, the “JNIEnv*  env, jclass  thiz” part is mandatory (so if you have a something like this: public static native void methodname(), it would be void methodname(JNIEnv*  env, jclass  thiz)). For our method we want to pass a float as argument. For some reasons you can’t just pass a float, you will need to use JNI mapping types. The float becomes a jfloat.
Note: for booleans you need to compare the value to JNI_TRUE or JNI_FALSE and not to true or false.

Honeycomb Tablets

Unlike the iOS devices from Apple, Android devices run at so many different configurations and API levels, it can be a bit tricky to support all of them. For NodeBeat we decided to create two versions: A phone version and tablet version. We basically distinguish between phones who are running Eclair or Gingerbread (2.2+) and tablets running Honeycomb (3.0). On the Android Market it seems like we only provide one version, but depending on what device you’re using to download the app, it will give you a certain version.

In order to maximize the use of the display, I’m using this line in the AndroidManifest.xml file:

1

android:theme="@android:style/Theme.NoTitleBar.Fullscreen"

On phones it sets the application to fullscreen. While this works for anything running Android 2.2+, it is a problem on Honeycomb devices. Honeycomb tablets don’t have any physical buttons and when the application is running in fullscreen mode, there is no shortcut to the menu button. This shortcut is normally placed in the top right corner.

This means that for the tablet version, we run the application in regular display mode.
Other than that, NodeBeat provides a rich user experience on tablet devices such as the XOOM or Asus transfomer.

Earbleeding masterpiece created by Sharath Patali (professional coder, horrific musician)

Android Market

Publishing the app to the Android market is no hassle at all (we were just a bit unfortunate and had our app pulled down by accident). There is no annual fee (just a one time 25$) and apps are approved instantly. The dev guide provides a comprehensive overview on how to build your project in release mode and how to sign it.

Go get it!

NodeBeat is available on the Android market:
https://market.android.com/details?id=com.AffinityBlue.NodeBeat

Try it out! It’s just a dollar

Website: http://nodebeat.com/
Twitter: http://twitter.com/NodeBeat

Of course, there is always the debate whether it is legal to build an emulator. Console manufacturers will always forbid any type of emulation of their systems. Although its a rather grey area as most documents on the chip design are publicly available (e.g. MIPS / ARM instruction sets).

Since I got excited about emulation in general, I started to do some research on how to build one myself. As I didn’t want to spend days or even weeks to get some results, I looked for a simple system. A friend introduced me to a small system called the Chip 8.
The Chip 8 itself didn’t existed as a hardware system, but was implemented in software. Due to the small instruction set (~30 opcodes), it was a very good candidate to work on as a first emulation project. The instruction set defines the functionality of the CPU. It for example contains instructions that allow the CPU to load or store data, but also perform mathematical tasks such as multiplications.

Having worked on a couple of emulation projects (and emulator plugins), I started to write a guide that will hopefully help out aspiring emulator authors and inspire emulation enthusiast. The guide explains how emulators work and provides a detailed overview on how to write a Chip 8 interpreter from scratch.

Writing a Chip 8 emulator shouldn’t be too time consuming. On average people tend to finish such project in one or two days. It’s a fun project to test out your programming skills and of course quite educational (and it isn’t just for CS majors )

If you’re ready to take up the challenge, click the link below! Have fun!

• How to write an emulator (CHIP-8 interpreter)

Summary:
“Earth Friends is a free Facebook application to visualize your social network on Google Earth. Locate your friends by using the Google Earth Webplugin or download your personal datafile for use with the Desktop version of Google Earth.”

Rewriting…

While my code from last week was running fine, there was a lot of room for improvement. Basically three major parts changed:

1. The database structure
2. Using a template engine
3. File compression

MySQL

The first thing I worked on was the database itself. The local database is primarily used for converting a location (defined by a city, state and country name) to a latitude and longitude.
Each time we need a location, we query the database, simple as that.

So let’s look at the following example:
First assume you have about 100 friends on Facebook. Probably most of them share their current location, but some of them might not want to share their location with Facebook apps or simply did not filled out the location at all.
If 75 friends filled this field with their current location, the old code would query the database 76 times (75 times for your friends + 1 time for your location).

While this looks like a lot of queries, the time for a query to complete also depends on the design of the database. As I was using MaxMind cities database, I initially imported all the data into one MySQL table. Just because it was convient to work with. However, the dataset has about 2.699.356 entries (cities).
Doing a search 75 times within this table was not going to be fast…

Besides the users’ patience, I am also limited by PHP’s execution time limit. I think that scripts are often allowed to run for 30 or 60 seconds, although I can’t stand a website that is taking longer than 10 seconds to show up.

So what to do next? Can we find a way to split the database into multiple small tables? Just limiting the number of entries to a fixed number (e.g. 100.000) wasn’t going to work as I would need some kind of lookup system in order to figure out where I could find my city.

Improvement one
The easiest solution I found for it (not saying its perfect) was by splitting the database by country. In my case I have 231 countries in the database, thus 231 tables are created.

Taking a look at the top 3 countries in our database reveals the following:

• Russia – 176.934 cities
• USA – 141.989 cities
• China – 117.508 cities

The average number per country is around the 20.000 cities. While the top 3 shows a significant higher quantity than the average, look ups are performed much faster than before.

Improvement two
We could’ve stopped here with improving the database part as the results were reasonable. However, it is a small effort to tweak it a bit more.

In our example, we query 76 locations from the database. But wait… what did we query? The location of friends! And what do a lot of friends have in common? Right, they share the same location. So by first creating a list of cities we need to look up, we can reduce the number of queries required to collect our data.

Smarty Template Engine

While PHP provides a nice environment for rapid prototyping, it can also become a mess easily. Using print or echo statements is fine for testing purposes but it is better to keep code separated from HTML.

This is why you need to use template engines. When using a template engine, you first collect data from your database and then parse your variables and arrays to the template engine. In your templates you specify where this data need to be placed. (Summary: PHP is used for data collection and prepared into arrays, Template engines use these arrays and are used for the actual design of the website).
Fortunately, there are free open source template engines such as Smarty.

Take a look at the following code:

1
2
3
4
5
6
7
8
9
10

{* Add placemark for friends *}
{foreach $friendlist as $friend}
  <Placemark>
    <name>{$friend.name}</name>
    <styleUrl>#{$friend.uid}</styleUrl>
    <Point>	
      <coordinates>{$friend.lon},{$friend.lat},0</coordinates>
    </Point>
  <Placemark>
{/foreach}

This is an actual code snippet from the template responsible for generating the KML. This particular section is used to display your friends icon on Google Earth.

Within the template I can specify a block (in the example its a Placemark) that will be looped. After setting up Smarty and collecting data from the database, I pass the result ($friendlist) to Smarty. In the template engine, it will now perform loop through this foreach loop and place the variables in the correct location.

A few more benefits of using a template engine is that you can store the results in a cache. By caching the results we can skip ‘expensive’ MySQL queries if we know the page hasn’t changed. By specifying the cache lifetime (for example 30 minutes), we can make sure that Smarty will regenerate the page if the cached page is older than 30 minutes.

ZIP

In the previous version of Earth Friends, I embedded the KML file into the header of the website (in Javascript). While this method works fine for small data sets, it has a large impact on page loading and render times when data sets are growing.

KML files are plain text files formatted in XML. Besides KML files, Google Earth also accept compressed KML files which have the extension KMZ. KMZ files are basically KML files compressed with ZIP.
Tests show noticeable differences in loading times when using KMZ. For example, my test data set in KML was about 693 KB. After compressing this file using zip (max. compression), the size was reduced to 92 KB. That’s around 13% of the original file size! As a result, loading times were reduced significantly.

Where can I find the application?

Ready to try out this application on facebook? Just click the following link to open up Earth Friends: Earth Friends application on Facebook.

After authorizing Earth Friends to access your profile data, it will reload the page and launch the Google Earth browser plugin. If the plugin is not installed, please follow the instructions that are displayed instead. The plugin should work in Windows and Mac OS X.

Important: Make sure you set your own location (with the correct privacy settings) or the curves will not appear!

More information can be found on Earth Friends profile page on Facebook.

How to use this?

To help you get started with Earth Friends, I have created two screencasts which demonstrate you how to add Earth Friends to your Facebook account and use the application.

Tutorial 1: How to use Earth Friends (View in 720p HD)

Tutorial 2: How to download the KMZ file for Google Earth Desktop (View in 720p HD)

Because I’m using Facebook to connect with my friends, I decided to dig into the documentation of the facebook APIs. Apparently there are multiple ways to get hold of your and your friends information. The most commonly used APIs are: Graph API / FQL . The first one lets you retrieve information about a friend or page by loading a specific URL, the second one lets you actually send a SQL query to retrieve the  information.

Since I want to make this a hassle free experience, I decided to make a Facebook application which would use the Google Earth Web plugin. This way, users only need to download the plugin, but everything works just in the browser.

How does it work?

Basically a Facebook app is just a website running on some server. In my case, I’m hosting my application on the same domain as my blog. Since the application is embedded into the Facebook website, normal users won’t notice. The app itself can be written in all kinds of languages but for the sake of simplicity I used PHP.

First we need to connect to Facebook using an API/SDK. This allows us to authenticate and securely connect to the Facebook servers. After enstablishing a connection, we use FQL to query: Our friendslist and the location of our friends. Unfortunately the friendslist only contains the name of the location and not the geospatial coordinates.

Therefor I had to create a lookup database that would translate a City/State/Country name into a geospatial coordinate (latitude and longitude). This was done by downloading a free database from Maxmind.com.

Now we should have all data available to create our KML file on the fly. For now I embed the KML result into the javascript header which seems to work fine for ~200 friends. I still need to do some benchmarks to see how well this scales. A demonstration video of the result can be found below:

Video

Can I try?

Since this project is still WIP, it is not available yet in the Facebook Application Directory. I’m planning to release this application for free soon.

While it is interesting to see the spread of a certain day, it is even more interesting to see how the spread is evolving over time. By keeping track of updates of the mirror page on Wikileaks, I was able to collect enough data for an animated version of the spread. My current dataset contains a 7 day period covering December 5th to December 12th.

As some of the commenters pointed out, the edges (curves) in the previous dataset didn’t always followed the shortest path. This was due to the (simple & stupid) algorithm I was using to draw the path between two points (basically just the mathematical shortest distance). In the latest dataset (download link is at the bottom) this is corrected. In particular this website: http://www.movable-type.co.uk was very useful for figuring out the the correct path.

* update 21-12-2010 11pm CET *
- Got featured on ReadWriteWeb, Thanks!

* update 23-12-2010 10pm CET *
- Embedded Google Earth web plugin in demo section

Mirror spread

The result of plotting the spread in a line chart:

Top 10 locations

An overview of the spread based on country:

1. Germany: 498
2. United States: 394
3. France: 194
4. Netherlands: 152
5. United Kingdom: 72
6. Sweden: 67
7. Canada: 49
8. Spain: 47
9. Switzerland: 36
10. Russian Federation: 32

Screencast / Video

Short screencast (Watch it in HD 720p, in fullscreen mode)

Online Demo

To view this demo you will need to install the Google Earth Browser Plug-in

Note: You need to move the range marker all the way to the left to make the timeline work:

• Move the time slider all the way to the right
  
• Move the range marker (the small attachment on the left of the time slider) all the way to the left
  
• Now you can move the time slider as you want
  

KML source

If the online demo doesn’t work for you, you can also try it in Google Earth!

• Make sure you’ve installed Google Earth.
• Next download and unzip the file following file:
• Wikileaks Google Earth KML files (Animated) (222)
• Open up the KMZ file.
• Enjoy!

Whether Julian Assange should be considered as a Hero (by publishing information) or Terrorist is an open question. Because of threats from governments, companies in the USA started to ban payment, hosting and dns services to Wikileaks. Soon Wikileaks moved from servers based in the USA to Sweden/Switzerland.

For now Wikileaks seem to be safe, however, they also started a call for mirrors.

Interested to see which people in the world are supporting Wikileaks, I had the idea to visualize the Wikileaks mirrors on Google Earth.

* update 09-12-2010 8pm CET *
- Added Google Earth web plugin links

* update 10-12-2010 10am CET *
- Got featured on ReadWriteWeb / NY Times, thanks guys!
- Stay tuned for more updates!

* update 23-12-2010 10pm CET *
- Embedded Google Earth web plugin in demo section

Data mining

In order to get the data, I went to the official Wikileaks website which had all mirrors listed on this page: mirrors. I wrote a small PHP script that would open up the mirror page and scanned the document for URLs. If a mirror was found, it would be stored in a file. At this moment there are about 1334 mirrors on the website.

Data manipulation

At this point I only have the URLs of the mirrors, but how do I know where these servers are located?

In order to know where the server is located, I used GeoLite City which is a service from MaxMind. GeoLite City allows you to resolve most IPs to a geospatial location. Of course it doesn’t gives the exact location, but usually it is able to show in which City the server is located, which is good enough for my purpose.

After obtaining the GeoLite City database (there is a free version!) I used PHP to write a script that would first resolve the URL to an IP address (PHP function: gethostbyname() ) and then use this IP address to look up the longtitude and latitude.

The last step was converting the data into the KML format (and adding some artificial altitude information) for use with Google Earth.

Data visualization

Below are some of the early results.

Currently the main server (http://46.59.1.2) seem to be located in Sweden, when we view the data in Google Earth we can see that a lot of mirrors are actually located in Europe.

Europe
Red pin: Wikileaks server
Yellow pins: Wikileaks mirrors
Greenlines: Connections from the Wikileaks server to a Wikileaks mirror

USA
Wikileaks mirrors in the USA

Sweden
Since some of the servers are at the same location (probably sharing the same data center), we can click on a pin and it will expand to show all mirrors located at this data center.

Screencast / Video

Short screencast (Watch it in HD 720p, in fullscreen mode)

Online Demo

To view this demo you will need to install the Google Earth Browser Plug-in

KML source

Want to try it out locally on your computer?

No problem! Here is how:

• Make sure you’ve installed Google Earth.
• Next download and unzip the file following file
  Wikileaks Google Earth KML files (2942)
• Open the KML files one by one.
• Enjoy!

Recently I started to organize the footage I made and collected over the years and decided to put some of it online. To maintain the overview, a new section has been added to the site: Research Projects. It includes most of the projects I have been working on at the University of Amsterdam (UvA) and Harvard University.

Each project includes a short description, pictures and a video. Enjoy!

(Click the project title for more information)

Interactive Networks

This project introduces the Interactive Network concept and describes the design and implementation of the first prototype.

Twilight

Twilight is an interactive graph exploration tool for multi-touch systems. Twilight provides a flexible environment that can be used to visualize and analyse graphs and networks found in the computational science.

Treepanorama

This projects involves the visualization of large phylogenetic tree structures such as the ones found in the Tree of Life. By combining high performance computer graphics with multi-touch input interaction methods, his project will create an interactive exploration environment that allows us to view the data interactively and in different representations. This research will lead into a better understanding of the evolutionary tree.

Involv

INVOLV is a research project that combines cutting-edge interactive technology with emerging information visualization techniques to create innovative explorations for large hierarchical data sets

Phylogenie

This application is designed to be a collaborative activity to teach undergraduate students about phylogeny and to prevent misconceptions about evolution. The system guides the students through a set of steps required to construct a phylogenetic tree based on morphological and DNA sequence data.

Since this project is still active, more media content will be released in future!

* update: 27 November 2010 *

Old footage from personal projects: Touch tracer and Real time fluid dynamics running on the UvA-MTT

Touch Tracer v0.3

Real time fluid dynamics

So what exactly is the Kinect? Is it similar to the PS2 eyetoy?

Actually it is much more advanced than the PS2 eyetoy. Unlike the eyetoy, the Kinect contains 2 cameras. One RGB camera which is used for ‘normal’ images and a depth camera is used to figure out the position of objects in its view. A nice explaination (and an overview of the components) can be found at ifixit.

Because Adafruit required the contest winner to open source his source code, this means that others can now enjoy hacking their Kinect as well! Currently the code is available at github and renamed to libfreenect (irc: #openkinect @ freenode): https://github.com/OpenKinect/libfreenect/

Getting the code to compile might be a bit tricky and it involves CMake to create the project files. Running it on Linux is very trivial (just make sure you’ve installed all dependencies), on Windows and Max OS X there are some extra steps involved to compile the library and demo application.

For the windows version you will need libusb, glut and pthreads. Also, don’t forget to select the win32 branch when you do a checkout from git. After creating the visual studio project files you will need to manually fix the path to the include and lib directories (The current CMake file is broken).

If you can’t be bothered, I have compiled a windows binary (vs2k8): Kinect-VC9-Release.zip

Before running the binary, make sure you’ve installed the drivers from github (First the XBOX motor, then camera and audio). To control the motor in the Kinect, you can use this code: NUI_Motor.cpp

My experiments

Last Friday I actually bought myself a Kinect. In the video below you can see me running the Kinect on my machine. For now it is just retrieving both camera streams and putting it on display.
Basically the library (libfreenect) is providing you with two images through a callback. One depth image and one RGB image. The depth image actually maps a depth to a certain color. In this video for example, the color red/white means something is really close to the camera and green/blue is further away.

Hopefully I will have some nice apps later this month .

Kinect hacks by others

3D mapping by Oliver Kreylos

ofxKinect 3D draw by Memo

Multitouch hack by Florian Echtler

Kinect Point Cloud by cc laan

Contest

P.s.: Before I forget, Matt Cutts (Google) started another contest. Check it out at: http://www.mattcutts.com/blog/open-kinect-contest/

Seth Sandler a good friend of mine, released his new website sparkon.net which is a website that he describes as “a social platform for people that are sparked (inspired) by creative and emergent technology”.

Personally I like the way how he organized the site. Basically he created a portal that allows you to rapidly find (multitouch) applications, open source programming frameworks, and community projects that are out there, all in one single place!

Check it out and don’t forget to add your own projects as well!

Interactive Tabletops and Surfaces 2010

If you’re doing research on interactive tabletops and surfaces, you might want to check out this years ITS 2010 conference. This year it will be hosted in Saarbrücken, Germany!

Check out the details below (more information after the break).

CALL FOR PARTICIPATION

5th Annual ACM Conference on Interactive Tabletops and Surfaces 2010

ITS 2010
November 7-10, 2010
Saarbrücken, Germany

http://www.its2010.org
————————————————————————-

The Interactive Tabletops and Surfaces 2010 Conference (ITS) is a
premiere venue for presenting research in the design and use of new
and emerging tabletop and interactive surface technologies. As a new
community, we embrace the growth of the discipline in a wide variety
of areas, including innovations in ITS hardware, software, design, and
projects expanding our understanding of design considerations of ITS
technologies and of their applications.

Building on their success in previous years, ITS again features Papers
and Notes presentations, as well as tutorials, posters, and
demonstrations tracks. For the first time, ITS 2010 will also include
a doctoral symposium.

ITS 2010 will bring together top researchers and practitioners who are
interested in both the technical and human aspects of ITS technology.
On behalf of the conference organizing committee, we invite you to
begin planning your submissions and participation for this year’s
conference!

CALL FOR PARTICIPATION: PAPERS AND NOTES

The use of interactive surfaces is an exciting and emerging research
area. Display technologies, such as projectors, LCD and OLED flat
panels, and even flexible display substrates, coupled with input
sensors capable of enabling direct interaction, make it reasonable to
envision a not-so-distant future in which many of the common surfaces
in our environment will function as digital interactive displays. ITS
brings together researchers and practitioners from a variety of
backgrounds and interests, such as camera and projector based systems,
new display technologies, multi-touch sensing, user interface
technologies, augmented reality, computer vision, multimodal
interaction, novel input and sensing technologies, computer supported
cooperative work (CSCW), and information visualization.

The intimate size of this single-track symposium provides an ideal
venue for leading researchers and practitioners to exchange research
results and experiences. We encourage submissions on (but not limited
to) the following topic areas as they relate to interactive tabletops
and surfaces:

* Applications
* Gesture-based interfaces
* Multi-modal interfaces
* Tangible interfaces
* Novel interaction techniques
* Data handling/exchange on large interactive surfaces
* Data presentation on large interactive surfaces
* User-interface technology
* Computer supported collaborative systems
* Middleware and network support
* Augmented reality
* Social protocols
* Information visualizations
* Interactive surface hardware, including sensing and input
technologies with novel capabilities
* Human-centered design & methodologies

CONFERENCE COMMITTEE

Conference Co-Chairs
Johannes Schöning
Antonio Krüger

Conference Program Committee Co-Chairs
Daniel Wigdor
Michael Haller

MENTORESHIPS

To help those with less experience in academic publication, ITS 2010
offers the help of our two Mentors Chairs. The mentoring program is
intended primarily for authors from non-academics, including industry.

Two types of help are available from the mentor: assistance in framing
and composing your research paper (early feedback), and help and
feedback reviewing an already completed paper (later feedback). In
either case, the mentor will help frame your work in context of other
related work, and make meaningful suggestions to improve the quality
of your contribution. Both types of mentorship have the same
submission deadline.

Although we cannot guarantee that your submission will be accepted by
ITS, mentoring would be helpful for you to turn your ideas into a
proper research paper. If you wish to avail yourself of this program,
you can do so by sending e-mail to both of the Mentor Chairs Lucia
Terrenghi (lucia.terrenghi@vodafone.com) and Changkyu Choi
(changkyu_choi@samsung.com). The submission type will vary depending
on the type of feedback you are seeking.

Early-Stages Feedback (Deadline: 2010.05.25):

Send a 1-page document describing your intended contribution. This
1-page must include author names and affiliation (e.g. company name),
and enough information for the chairs to understand both what it is
that you have done.

Completed Paper Feedback (Deadline: 2010.05.25):

Send your completed paper to both mentors chairs by May 25.

For both early and paper feedback, you will receive a response by June 14.

SUBMISSIONS
Papers/Notes: We invite paper submissions of two kinds: Papers (10
pages) and Notes (4 pages). Papers must present original, highly
innovative, prospective and forward-looking research, possibly in one
or more of the themes given above. Notes must also report novel and
complete research, but where the scope and scale of the contribution
is more focused and succinct than papers. Submissions must be
submitted as a single PDF file in the ACM format through the
submission system. A template for submissions can be found on the ITS
website (http://www.its.2010.org).

All accepted submissions will be presented at ITS 2010 and appear in
the ITS proceedings and be archived in the ACM digital library.

IMPORTANT DATES

Paper / Note Mentorship Requests (optional): May 25, 2010
Paper / Note Mentorship Feedback Received (optional): June 14, 2010
Paper / Note Submissions: June 23, 2010
Paper / Note Author Notifications: August 27, 2010
Paper / Note Camera-Ready Deadline: October 1, 2010

The calls for the posters, demos, tutorials, and the doctoral
symposium will be published soon, please check our website for further
information.

ITS 2010 Conference: November 7-10, 2010

http://www.its2010.org

I haven’t visited the NEMO for a while (since I’m still in the USA) but my dad visited the science center last month and took these nice pictures:

[Show as slideshow]

Thanks dad!

A few weeks ago I attended TEI’10 hosted at the MIT Medialab.

“TEI, the conference on tangible, embedded, and embodied interaction, is about HCI, design, interactive art, user experience, tools and technologies, with a strong focus on how computing can bridge atoms and bits into cohesive interactive systems.”

To keep it simple, I decided to just posted a lot of media instead of a summary of the TEI ’10 conference… enjoy !

Inspiring projects

Unlike most conferences I have been, this conference was not only a place where scientific work was presented but also where artists could show their work (art meets science).

The following two projects are clearly an example for this:

• SMSlingshot (link contains cool video!) by Patrick Tobias Fischer, Christian Zöllner and Thilo Hoffmann.
• Electronic Popables: exploring paper-based computing through an interactive pop-up book by Jie Qi and Leah Buechley.

Some of my pictures shot at the conference

[Show as slideshow]

12345►

More pictures of the conference

Pictures created by Jean Baptiste Paris on Flickr.

Studio sessions

Pictures created by Seth Hunter of the studio session on surface computing
(I should be in there somewhere…).

Video studio sessions

Nice video compilation of the different studio sessions that took place at the MIT Medialab.

Blog

A blog about the TEI 2010 conference

Keynotes

• John Frazer
• Vik Muniz

Unfortunately I could not attend the conference on interactive tabletop and surfaces, so I missed the great keynotes from: Ken Perlin, Chia Shen and Bill Buxton.
Fortunately a few fellow researchers and friends blogged about the event.

PyMT

PyMT is a python module designed for developing multi-touch enabled applications. The current aim is to provide an environment which allows developers to do quick and easy interaction design, and rapid prototype development.

The projects was initially started by Thomas Hansen but now also contains contributes from: Juan Pablo Hourcade, Sharath Patali, Matiue Virbel, Thiago Serra. Their submitted paper is available in the proceedings from this years conference.

For those who are interested, a nice set of videos demonstrating the toolkits capabilities can be found on Vimeo.

Since it’s open source, everyone can download and contribute to the project. The website can be found here: http://code.google.com/p/pymt/. A nice example of an application created with PyMT by Sharath Patali can be found here: NUIPaint.

TUIO

As one of the designers of the TUIO protocol and the ReacTable, Martin Kaltenbrunner presented his work “reacTIVision and TUIO: A Tangible Tabletop Toolkit” at the conference. For those who are interested, Martin created a nice summary of the conference on his personal blog which can be found here: http://modin.yuri.at/blog/?p=117

More interesting links…

More information about the conference can be found in the topic at the NUI Group forums started by Johannes Schöning (contains some videos and docs), the official youtube channel and on Twitter.

ITS 2010?

Hope to see you next year in Saarbrücken, Germany!

Lots happened the past year, the most notable projects I have worked on as a scientific programmer were “Interactive Networks” (a collaboration with the SNE department) and “Twilight” (a graph visualization and exploration tool for multi-touch devices). I am still busy creating some short videos from the HD footage we shot, so expect them in the near future.

I am very glad that I was offered the opportunity to participate in both projects. Therefore I would like the following people who made it possible for me to work for the University of Amsterdam (UvA):

• Dr. Robert Belleman (SCS)
• Prof. Peter Sloot (SCS)
• Dr. Cees de Laat (SNE)

I would also like to thank:

• Paul Melis for his help with constructing the multi-touch table, software support and the pleasant time at the SVVR Lab.
• Dr. Elena Zudilova-Seinstra for her help and advice while I was writing my masters thesis.
• Dr. Jaap Kaandorp and Dr. Alban Ponse for giving me the opportunity to study at the UvA.
• Rudolf Strijkers for the fun time we had in Amsterdam and in Austin/Houston during SC08. And also for the stressful period while writing the paper for ICCS.

Finally I would like to thank Dr. Dick van Albada, Edwin Steffens, Erik Hitipeuw, Alexis Salin, my fellow students at the UvA and everyone I forgot.

A new challenge

After a very brief meeting in Amsterdam during the IEEE Tabletop 2008, I got in touch again with Dr. Chia Shen during the IDC 2009. Shen is a Senior Scientist and Program Manager at the Initiative in Innovative Computing (IIC) at Harvard University. She is well known for her work as Senior Research Scientist at MERL (the Mitsubishi Electric Research Labs).

During the breaks at IDC 2009, we had a talk about the work I was doing at the UvA and my involvement in the NUI Group. After showing some work I have been doing on network visualization (Twilight) for multi-touch devices it seemed that there were some similarities between the work I was doing at the UvA and the research that was going on at Harvard University. While discussing some of the ongoing projects, I was asked whether I was interested to join the SDR Lab as a research fellow. After some consideration, I proudly accepted her job offer.

The next months I will be working on several interesting projects. One of the tasks will be continuing the development of INVOLV, a multi-touch application developed by Michael Horn and Chia Shen in collaboration with the Encyclopedia of Life (EoL), the Tree of life (ToL) and many biology professors and scientists since May 2008. The application visualizes the data of the EoL and the ToL in a unique way using a Voronoi diagram and is used for learning and discovery.

A video of Michael Horn explaining the INVOLV project can be found below:

Living in the USA

It has only been a week since my arrival in the USA, but so far it has been a pleasant experience.

I have not made a lot of photos yet, but this one was made nearby the place where I am staying (safety first?):

More photos will be posted on Flickr @ flickr.com/photos/multigesture/

To help out fellow multi-touch users and developers, the NUI Group community decided to create a book sharing information about multi-touch technologies. Originally we intended to release this book during the IDC2009, unfortunately we were only able to finish a concept version. This concept version was available at the NUI Group booth.

NUI Group press release:

“We are proud to announce the release of the first edition of the NUI Group Community Book – Multi-Touch Technologies. This book comprises state of the art articles from our Community Wiki, as well as several chapters written by leading experts in the field specifically for this publication. Detailing basic hardware technologies as well as the process of developing software suited for multi-touch systems, this book provides a technical introduction to the world of multi-touch.

This book aims to provide an introduction to the multi-touch world for all backgrounds of people, and also include sufficient information to serve as a general “handbook” for those acquainted with the field, including code examples, suppliers, and building methodology.

Multi-Touch Technologies is a unique publication in its domain in that it is released at no charge under the Creative Commons 3.0 Attribution Share-Alike license – all content found in this book is free to be re-distributed and modified under a similar license, with attribution.”

View the full press release.
Download your copy here: First_Edition_Community_Release.pdf

Comments and suggestions can be posted in the feedback topic @ NUI Group forums.

* update 15 July *
Mirror of the NUI Book:
English version
Chinese version (special thanks to Kelvin / mt2a for translating the book, feedback can be posted here).

Google Summer of Code 2009

Like last year, the NUI Group will be coaching students to work on open source multi-touch projects. This year, I will be helping out by mentoring a student who will add hand tracking to Community Core Vision (formally known as tBeta).

Project: Recognition, Tracking and Association of Hands, Fingers, and Blobs: A Community Core Upgrade
Student: Thiago de Freitas Oliveira Araujo
Mentor: Laurence Muller

Follow his progress on his blog and his code on nuicode or google code.

Hand/Fingers occlusion testcase

Matching a hand model (using Active Appearance Models)

More videos can be found here.

Watch it in HD on Youtube or Vimeo!

Changelog for v3:

• Fixed compatibility issue with Adobe AIR 1.5 (Fullscreen bug)
• This version should autodetect your screensize to support any screen resolution.
• Added a Google Maps object
• Added support for adding pictures on the fly (for example if you want send a picture from your cellphone with bluetooth, read the readme.txt for the instructions)
• Added a exit-button, reset view-button, canvaslock and displaylock buttons.
• Temporary removed DCIM support due to slowdowns

Want to try it out yourself?
Download it now: Multitouch Media Application Pro v3 (12186) - 2.25 MB

Please read the install instructions in readme.txt (and download the latest Adobe AIR 1.5.x).

note: If you want to try out the bluetooth feature, I recommend installing BlueSoleil to handle the pairing of the devices and file transfers.