Kinect viewer with HTML5 canvas

I wanted to create a web-based Kinect viewer that would work on multiple platforms, including Android mobile phones and iPhones. To this end, I created a HTML5 canvas particle viewer for Kinect data using my very limited javascript knowledge and a lot of Mr. doob's knowledge.

Kinect HTML5 particle image

Take a look at the HTML5 Kinect particle image viewer.

Using libfreenect I captured the depth and color data, used Imagemagik and an Automator script to process and stich the images, and threejs to render.

You can extract data from an image by rendering it to a hidden canvas element, and then reading the canvas image data. To avoid errors, the image data should only be read after the full image has been loaded, as per the callback function in loadTexture.

function getImageData( image ) {
var canvas = document.createElement( 'canvas' );
canvas.width = image.width;
canvas.height = image.height;
var context = canvas.getContext( '2d' );
context.drawImage( image, 0, 0 );
return context.getImageData( 0, 0, image.width, image.height );
}

function getPixel( imagedata, x, y ) {
var position = ( x + imagedata.width * y ) * 4, data = imagedata.data;
return { r: data[ position ], g: data[ position + 1 ], b: data[ position + 2 ], a: data[ position + 3 ] };
}

var texture, imagedata;
texture = THREE.ImageUtils.loadTexture( "textures/sprites/spark1.png", new THREE.UVMapping(), function ( event ) {
imagedata = getImageData( texture.image );
} );


To run the system locally, avoiding some of Chromes very annoying security settings, you can just use python to run a simple webserver:

python -m SimpleHTTPServer


I recorded a short data sequence and rendered to sprites to create this final 3d home move with the Kinect (too slow for my phone to render). I believe this is the first web-based kinect video ever made, youtube3d anyone?

Animated 3D kinect data with HTML sprites

Next time, I might try WebGL...