Kinect viewer with HTML5 canvas

I wanted to create a web-based Kinect viewer that would work on multiple platforms, including Android mobile phones and iPhones. To this end, I created a HTML5 canvas particle viewer for Kinect data using my very limited javascript knowledge and a lot of Mr. doob's knowledge.

Kinect HTML5 particle image

Take a look at the HTML5 Kinect particle image viewer.

Using libfreenect I captured the depth and color data, used Imagemagik and an Automator script to process and stich the images, and threejs to render.

You can extract data from an image by rendering it to a hidden canvas element, and then reading the canvas image data. To avoid errors, the image data should only be read after the full image has been loaded, as per the callback function in loadTexture.

function getImageData( image ) {
var canvas = document.createElement( 'canvas' );
canvas.width = image.width;
canvas.height = image.height;
var context = canvas.getContext( '2d' );
context.drawImage( image, 0, 0 );
return context.getImageData( 0, 0, image.width, image.height );
}

function getPixel( imagedata, x, y ) {
var position = ( x + imagedata.width * y ) * 4, data = imagedata.data;
return { r: data[ position ], g: data[ position + 1 ], b: data[ position + 2 ], a: data[ position + 3 ] };
}

var texture, imagedata;
texture = THREE.ImageUtils.loadTexture( "textures/sprites/spark1.png", new THREE.UVMapping(), function ( event ) {
imagedata = getImageData( texture.image );
} );


To run the system locally, avoiding some of Chromes very annoying security settings, you can just use python to run a simple webserver:

python -m SimpleHTTPServer


I recorded a short data sequence and rendered to sprites to create this final 3d home move with the Kinect (too slow for my phone to render). I believe this is the first web-based kinect video ever made, youtube3d anyone?

Animated 3D kinect data with HTML sprites

Next time, I might try WebGL...

2011 Wrapup

Looking back, 2011 was certainly a quieter year than 2010. Personal life got busy in the later half of the year, so less was posted. My job also took a less technical turn as I moved away from software development.
By far my most popular posts concerned my analysis of some popular demo scene effects and WebGL:

• The most popular posts were on Ripple effect, Raycasting planes, Twist effect, many more demoscene effects explained.
  
• The self-explanatory demo, was well received, scratching into the top of the month for pouet for a while, and coming 3rd at the Mozzila demoparty. This animates the explanation behind a number of the demo effects.
  
• Slides on a guest lecture that cover some of the details of these effects again.
  

Otherwise, a survey / summary post on exe compression for multiple platforms was well read, and my Civ 5 casual game mod proved popular too.

I will try to post a bit more about work and the mining industry, not because these are popular posts, but because there is so little information about this field on the web. I hope to post more about robotics and graphics (raytracing), and will try to open source and post about some of my past work.