Tuesday, August 16, 2011

Kinect and OpenKinect on Apple Mac OSX

Adrian, in Kinect-3D!
The Kinect is a revolutionary sensor for robotics - the low price point has seen an explosion of interest and applications in this field. (Hopefully we will see the same for the Neato XV11 low cost laser scanner one day). I was very impressed by the data the Kinect returned, it is a fantastic device.

To get started, just follow the OSX install instructions at OpenKinect. You will need MacPorts. A copy of the instructions is here:
sudo port install git-core
sudo port install libtool
sudo port install libusb-devel
git clone https://github.com/OpenKinect/libfreenect.git
cd libfreenect
mkdir build
cd build
ccmake ..
cmake ..
make
sudo make install
(Press 'g' to generate your cmake make files). Very simple.

Now check under system profiler to see that the Kinect device is detected.

I had the following error when I tried to read from the device:
Isochronous transfer error: 1 

Thankfully, this was easy to solve with a fully self-contained installer from AS3 kinect OSX download package. You should now be able to read the color image, depth map, and command the servo.

The next step will likely be to interpret the data in some form. The Point Cloud Library will be very useful for processing the data. A tutorial on Point Cloud Library is available, although many features are breezed over.

But to begin with PCL, you will need Eigen, a fast vector and matrix math library. (Also used in MRPT)
First download Eigen, and unzip it. Building Eigen is quite straight forward:
cd eigen-eigen-3.0.1
mkdir build
cd build/
cmake ../
sudo make install

PCL has a self installing package, and will install itself to /usr/local/lib/libpcl_*
Here is a simple test program from the tutorial:


#include
#include "pcl/io/pcd_io.h" 
#include "pcl/point_types.h" 

int main (int argc, char** argv) { 
    pcl::PointCloud cloud; 
    //Fill the cloud data 
    cloud.width = 5
    cloud.height = 1
    cloud.is_dense = false
    cloud.points.resize(cloud.width * cloud.height); 
    
    for (size_t i = 0; i < cloud.points.size(); ++i) { 
        cloud.points[i].x = 1024 * rand() / (RAND_MAX + 1.0); 
        cloud.points[i].y = 1024 * rand() / (RAND_MAX + 1.0); 
        cloud.points[i].z = 1024 * rand() / (RAND_MAX + 1.0); 
    } 
    
    pcl::io::savePCDFileASCII("test_pcd.pcd", cloud); 
    std::cerr << "Saved " << cloud.points.size () << " data points to test_pcd.pcd. " << std::endl; 
    
    for (size_t i = 0; i < cloud.points.size(); ++i) 
        std::cerr << "    " << cloud.points[i].x << " " << cloud.points[i].y << " " << cloud.points[i].z << std::endl; 
    
    return(0); 



This can be compiled directly with:
g++ pcd_write.cpp -c -I /usr/local/include/pcl-1.0/ -I /usr/local/include/eigen3/ 
g++ pcd_write.o /usr/local/lib/libpcl_io.dylib 

Finally, attached is a youtube video showing some very impressive visual SLAM surface reconstruction with the Kinect (See KinectFusion project page).


Of course, you will probably want some image stabilization technology to get smooth input videos, so why not try an Owl?


Happy Hacking!

Friday, August 12, 2011

ROS - Robot Operating System

Not so long ago there were a number of robotics packages that provided a variety of functionality of various tasks, such as OpenCV for computer vision, MRPT for mobile robots and ORCOS for general functions. Recently, this has changed as Robot Operating System (ROS) has become the predominant system which has swallowed the functionality of all the other packages.

ROS functionality includes:
Refer to this full list of ROS packages. The functionality encapsulated by ROS is both amazing and disturbing.

I've never been a fan of massive (disorganised!) frameworks that force a work structure onto you, but the robotics world isn't leaving you with much choice anymore. After implementing a graph based information flow architecture in ImprovCV and working closely with DirectShow's graphfilter I'm also keenly aware of the limitations of a graph-based approach to software development. It's not a one-size-fits-all solution, unfortunately ROS leaves little choice.

Thankfully some ROS packages are largely stand alone, so there is still some hope that one day some organisation will put in the effort to streamline and clean up ROS overall.

In any case getting ROS running on Apple Mac OSX 10.6 was not too difficult, however I lost my install notes in an untimely crash, so I'll only give a brief overview. ROS installation relies on MacPorts. I'd been holding off on installing MacPorts as it has historically been unstable, and doesn't play nicely with already installed libraries. MacPorts is now stable and well tested, but it still doesn't play nice with the rest of your system.

To install ROS I followed the ROS diamondback OSX installation instructions, all without any major dramas. I did have to alter the CMakeLists.txt as suggested in the troubleshooting section.

MacPorts gave me a few headaches, some of the errors I encountered were:
  • Error: Target org.macports.install returned: no destroot found

    To solve, use:
    sudo port clean NAME
  • Error: NAME already exists and does not belong to a registered port

    To solve, use:
    sudo port -f activate NAME

Once ROS is all installed you can source setup.sh. Remember to add in any additional packages to ROS_PACKAGE_PATH.

All in all, I wouldn't recommend running ROS on OSX, although the base ROS install works without any major dramas, installing each individual package is a bit of a struggle.

I believe ROS is an appropriate name, it really should be a stand alone operating system. I would strongly advise setting up a separate linux OS dedicated to your ROS install, running ROS ontop of OSX is quite problematic.

Wednesday, August 10, 2011

Communications and Messaging Frameworks

A common problem for any distributed software project is handling inter process communications and messaging. Luckily, a number of frameworks already exist to solve this issue. Robotics projects tend to have a range of requirements, from providing a higher level cognitive architecture framework (typically a set of interacting components or nodes) to low-level realtime communications (e.g. CANbus, RT ethernet).

Some robotics packages already provide a particular architecture, such as ImprovCV and ROS which are both filter based, or MRPT which provides some IPC primitives. However, this problem is hardly limited to robotics and many real time distributed networking/processing systems have similar communications requirements. (e.g. computer games, or traffic management systems / plant control systems).

The communications frameworks can be broadly grouped into three categories:
  1. Integrated middlewear for a larger package
  2. (hard) Real time control orientated frameworks
  3. (soft) Real time messaging libraries
From a games perspective, the people at second life have a nice evaluation of several messaging systems. Beginning with soft real time messaging, some interesting libraries include:
  • Google protocol buffers, for serialisation. I mention it, because it is popular.
  • MessagePack is a language agnostic message serialisation system similar to Google protocol buffers - just faster and better.
  • CORBA allows objects to be sent around a network, often used in business applications. It is an open standard and language agnostic as it includes an Interface Definition Language to define components.
  • Microsoft DCOM / .NET remote and its many variants are similar to CORBA, just the Microsoft variant.
  • Apache ActiveMQ another enterprise orientated message broker with support for a large number of scripting languages (Python, PHP, Ruby, Perl, etc).
  • RakNet manages distributed objects, primarily targeted towards computer games (used in Unity).

  • Open Torque Networking Library has its origins in the Torque game engine designed for massively multiplayer games.

  • YAMI4 is a fairly advanced but easy to use peer to peer messaging framework for control systems that is easy to extend to a pub/sub system. POSIX, Windows, Java, C++, well worth a look.

  • 0MQ, zero MQ is a messaging system that is socket based, and also has pub/sub extensions.

  • Socket.IO hardly qualifies for a framework, but is a great enabler for realtime HTML communications, so it certainly deserves a mention.

there are many more networking libraries available, but these are the ones that stand out for this category. For hard real-time control tasks the choices are a bit more limited, however libraries of note are:
  • Real time CORBA, is CORBA modified to be a bit more lightweight and able to support real time requirements. TAO ACE is worth looking into if your following this path.
  • Data Distribution Service (DDS) is similar to Real time CORBA, however it has the advantage of lessons learnt from CORBA and provides simpler fine grained control over quality of service and intelligent networking. RTI DDS is the framework we used for the WAMbot TARDEC/DSTO MAGIC 2010 project. OpenDDS is an open source implementation.
  • Real time data base (RTDB) is an open source framework that provides a lower-level communications interface for C/C++ over Xenomai. This is the framework we used for the Team Annieway entry to the DARPA Urban Challenge.
  • LCM is a socket-based IPC library used by Team Michigan for MAGIC 2010.
  • IPC a low level inter process communications system for embedded systems used by NASA.
  • Dynamic Data eXchange (DDX) is a large open source distributed software architecture developed by CSIRO and has been previously integrated with ROS.
Finally, there are a number of application specific frameworks, for Robotics this includes ROS, MS Robot Studio, Player/Stage, JAUS and more general packages such as Labview and Matlab. I think it is prudent for every software developer to be familiar with at least three tiers of communications frameworks, from large scale frameworks (e.g. CORBA) to light-weight frameworks (e.g. RackNet) to low-level frameworks (e.g. Socket.IO). Ideally, one for each key application area (e.g. Hard real time, Soft real time, Business applications, Web).

Friday, July 29, 2011

Half year catchup on Graphics, GPUs, Compilers, etc.

Another slow month on the blog. More than half way through the year, so its time to catch up on the backlog of news. Only covering graphics, games, physics and GPGPU and compilers. Expect a number of posts on robotics soon!
Finally, here is the SIGGRAPH 2011 technical papers highlights video, it contains a number of interesting advances in physics simulations and modeling.

Monday, June 27, 2011

CAD on OSX with Blender and Rhino - DWG

The majority of the engineering world use AutoCAD-like products for creating drawings, with typical exchange formats of DWG, DXF, SAT or IGES. (I wrote some tips on using AutoCAD and SolidEdge here) Unfortunately on OSX there has been historically little choice. I asked Stefan Boeykens who runs the CAD 3D blog for suggestions, and compiled a list of possibilities:
  • AutoCAD have a Mac OSX version. Obviously 100% DWG support, the downside is the price (Retails for $3,995, 30 day trial)
  • iCADmac is a mac port of progeCAD, that is actually very good on OSX (still a poorer cousin of AutoCAD). I use this for 2D CAD work on OSX. (Retails ~$1000, 30 day trial)
  • ARES commander is a cross platform CAD package for Windows,OSX,Linux. So there should be no compatibility issues between. I've not tried it out myself. (Retails $795, 30 day trial)
  • Blender is a free cross platform 3D package, that does have DWG/DXF plugin support. Unfortunately, the DWG support is not very good, and it is difficult to get the plugin to work under OSX and Linux. (free, Open Source)
  • Rhino 3D, a free trial version for OSX at the moment. (Retail expected ~$1000, free unrestricted trial while in beta)
Although Stefan mentioned some issues with Rhino3D, it worked perfectly with all my CAD files, and it is still an alpha product at the moment. Rhino3D supports a lot of file formats to a high standard:
  • 3dm (Rhino format)
  • 3ds (3d Studio, original not Max)
  • dwg (AutoCAD - I tested the 2007 version)
  • dxf (AutoCAD)
  • fbx (MotionBuilder)
  • iges (IGES)
  • lwo (Lightwave)
  • obj (Wavefront)
  • sat (ACIS) [Export only]
  • stl (Stereolithography)
  • vrml (VRML, also wrl file format)
In the end, I only used Rhino3D to convert the data and used Blender for doing most of my data manipulation, as I already have a number of scripts for it (See Convex hull generation for Blender). Every time I get back to using Blender after a pause it takes me a while to get used to the interface, here are some common commands I use when manipulating objects to conform to a given coordinate system:
  • space - add an object
  • g - move (follow by x,y,z to select an axis)
  • r - rotate (follow by x,y,z to select an axis)
  • a - selects all objects
  • h - hide objects
  • . - zooms so you can see all
  • 0 - switches to camera view (F9 - edit panel, you can set the FOV by altering 'D')
  • n - displays the current transformation matrix (To find the global coordinates you should view the Object in edit mode)
  • SHIFT+c - centres the cursor
  • View->View properties - allows you to manually enter the cursor position
  • Object->Transform->Center Cursor - allows you to re-define the center of the object
  • CTRL+a - collapses the rotation/scale transformation matrices. (see above for setting the position)
  • CTRL+p - to parent objects, allowing you to move them in groups
This Blender keyboard shortcut map always comes in handy too!

Friday, June 17, 2011

Deform (textured interference) effect in WebGL





'deform' interference effect
This effect is one of my all time favourites. It is somewhat similar to the XOR bitplasma effect with which I started the tutorial series based on iq's shader toy. This effect is an 'interference' of two points based on the subtraction of the radii and angles.

Jumping straight to the solution; the code we are trying to decipher is:

float a1 = atan(p.y-offset1.y,p.x-offset1.x);
float r1 = sqrt(dot(p-offset1,p-offset1));
float a2 = atan(p.y-offset2.y,p.x-offset2.x);
float r2 = sqrt(dot(p-offset2,p-offset2));

vec2 uv;
uv.x =(r1-r2);
uv.y =(a1-a2);


r1

So why does it look the way it looks?
r2

Lets start with the radius part. If we just look at one radius (r1) we would expect to see a steadily increasing value radiating out from our offset point (in all my examples this will be at 0.3)

And this is exactly what we see when we plot this. If we plot the inverse (negative r2, offset to -0.3) we should see the inverse, that is, a steadily decreasing value radiating from the offset point.
r1-r2

So if we combine these (r1-r2) we get a nice smooth surface transition between these two patterns. Where does this blending come from?

Lets take it back a step and look at our problem in just one dimension. Our definition of the radius is sqrt(x*x+y*y), in 1D we can plot x*x and sqrt(x*x):

Coming back to the generation of the surface, we can plot two interfering 1D functions, that is sqrt((x-0.3)*(x-0.3)) and sqrt((x+0.3)*(x+0.3)), and the difference (i.e. the first subtract the second). This gives us a ramped step between the functions as each reaches its turning point.

We can see the same feature in the surface, if we view it from an appropriate angle.
r1-r2, alternative view
We still haven't discovered the cause of the smooth blending that gives such a nice visual effect. When we add a constant (i.e. offset) to the function sqrt(x*x+k), notice how we now have the turning point at sqrt(k) and have a parabola instead. This is effectively evaluating sqrt(x*x+y*y) for some constant y=sqrt(k).

So, if we now consider the two extremes, again plotting in 1D but considering the 2D case for both our x-offsets of =-0.3 and 0.3 we get a set of blended curves (dark blue/red is y=-0.3, light blue/purple is y = 0.3).


And that is where our full surface comes from.



Left: atan(x) Right: atan(1/y)
Now we want to understand the blend in the angles. The generating function is nothing more complex than atan(x/y). Below is a graph of atan(x), followed by atan(1/y). Note: There are different viewing angles for each function so it is easier to see the curvature.

If we combine these two equations into one (i.e. atan(x/y)) we get:

atan(x/y)


Now, we just need to understand the interference pattern that is generated from two of these functions interacting. I'm going to take a shortcut here and just let an animated gif tell the story. This animation shows a discretized version of the angles (first in steps of ~60 degrees, then ~30, etc.) interacting for the equation a1-a2 from the code snippet.

a1-a2 discretized and animated (open as a separate image)


From the gif you can see how the curves evolve. To understand these better we can take a look at how this curve pattern comes about for a constant difference angle. If we modify the end of our original shader code to:

uv.y = (a2-a1);
float k = 0.0;
if ((uv.y > 0.3) && (uv.y<0.31)) k = 1.0; gl_FragColor = vec4(col.r+k,col.g+k,col.b+k,1.0);


The resulting image is still the original effect, but one of the curves is highlighted (in white). This represents a curve where (a2-a1) ~= 0.3

We can understand this curve by generating a set of triangles in which two vertices are fixed, and the difference between two angles is held constant. By altering one angle the final vertex will generate the sweeping curve in the original image (See Law of Sines). In the example below, we vary a1 and hold a2 as (a1 - 30 degrees).



This generates our final explanatory image of the sweeping triangles:


Finally! The full effect has been deciphered, click the button below to see it in action. (You will need a WebGL enabled browser)

Your browser doesn't appear to support the HTML5 <canvas> element.

Monday, June 13, 2011

Demoparties

Australia is hosting another demo party, this time in Sydney called Flashback 2011. It will be interesting to see what gets released.

For a more international focus there is the Mozilla Demoparty with WebGL, CSS, GIF animation, and Audio competitions. The 'event' started on the 10th of June, and submissions close on the 30th of July. Take a look at the original announcement of the Mozilla Demoparty incase you want to organize a local contributing event.

If your looking for a modern tracker to make your audio you can try the open source Schism Tracker (Impulse Tracker clone). (Daniel Gagan has a nice blog post on various methods for implementing HTML5 audio loops) For your WebGL needs, there are a number of libraries/engines to try:
Should be enough to get you started. Of related interest is a description of Brandan Jones' WebGL renderer for id's Rage levels.