Tuesday, August 16, 2011

Kinect and OpenKinect on Apple Mac OSX

Adrian, in Kinect-3D!
The Kinect is a revolutionary sensor for robotics - the low price point has seen an explosion of interest and applications in this field. (Hopefully we will see the same for the Neato XV11 low cost laser scanner one day). I was very impressed by the data the Kinect returned, it is a fantastic device.

To get started, just follow the OSX install instructions at OpenKinect. You will need MacPorts. A copy of the instructions is here:
sudo port install git-core
sudo port install libtool
sudo port install libusb-devel
git clone https://github.com/OpenKinect/libfreenect.git
cd libfreenect
mkdir build
cd build
ccmake ..
cmake ..
sudo make install
(Press 'g' to generate your cmake make files). Very simple.

Now check under system profiler to see that the Kinect device is detected.

I had the following error when I tried to read from the device:
Isochronous transfer error: 1 

Thankfully, this was easy to solve with a fully self-contained installer from AS3 kinect OSX download package. You should now be able to read the color image, depth map, and command the servo.

The next step will likely be to interpret the data in some form. The Point Cloud Library will be very useful for processing the data. A tutorial on Point Cloud Library is available, although many features are breezed over.

But to begin with PCL, you will need Eigen, a fast vector and matrix math library. (Also used in MRPT)
First download Eigen, and unzip it. Building Eigen is quite straight forward:
cd eigen-eigen-3.0.1
mkdir build
cd build/
cmake ../
sudo make install

PCL has a self installing package, and will install itself to /usr/local/lib/libpcl_*
Here is a simple test program from the tutorial:

#include "pcl/io/pcd_io.h" 
#include "pcl/point_types.h" 

int main (int argc, char** argv) { 
    pcl::PointCloud cloud; 
    //Fill the cloud data 
    cloud.width = 5
    cloud.height = 1
    cloud.is_dense = false
    cloud.points.resize(cloud.width * cloud.height); 
    for (size_t i = 0; i < cloud.points.size(); ++i) { 
        cloud.points[i].x = 1024 * rand() / (RAND_MAX + 1.0); 
        cloud.points[i].y = 1024 * rand() / (RAND_MAX + 1.0); 
        cloud.points[i].z = 1024 * rand() / (RAND_MAX + 1.0); 
    pcl::io::savePCDFileASCII("test_pcd.pcd", cloud); 
    std::cerr << "Saved " << cloud.points.size () << " data points to test_pcd.pcd. " << std::endl; 
    for (size_t i = 0; i < cloud.points.size(); ++i) 
        std::cerr << "    " << cloud.points[i].x << " " << cloud.points[i].y << " " << cloud.points[i].z << std::endl; 

This can be compiled directly with:
g++ pcd_write.cpp -c -I /usr/local/include/pcl-1.0/ -I /usr/local/include/eigen3/ 
g++ pcd_write.o /usr/local/lib/libpcl_io.dylib 

Finally, attached is a youtube video showing some very impressive visual SLAM surface reconstruction with the Kinect (See KinectFusion project page).

Of course, you will probably want some image stabilization technology to get smooth input videos, so why not try an Owl?

Happy Hacking!

Friday, August 12, 2011

ROS - Robot Operating System

Not so long ago there were a number of robotics packages that provided a variety of functionality of various tasks, such as OpenCV for computer vision, MRPT for mobile robots and ORCOS for general functions. Recently, this has changed as Robot Operating System (ROS) has become the predominant system which has swallowed the functionality of all the other packages.

ROS functionality includes:
Refer to this full list of ROS packages. The functionality encapsulated by ROS is both amazing and disturbing.

I've never been a fan of massive (disorganised!) frameworks that force a work structure onto you, but the robotics world isn't leaving you with much choice anymore. After implementing a graph based information flow architecture in ImprovCV and working closely with DirectShow's graphfilter I'm also keenly aware of the limitations of a graph-based approach to software development. It's not a one-size-fits-all solution, unfortunately ROS leaves little choice.

Thankfully some ROS packages are largely stand alone, so there is still some hope that one day some organisation will put in the effort to streamline and clean up ROS overall.

In any case getting ROS running on Apple Mac OSX 10.6 was not too difficult, however I lost my install notes in an untimely crash, so I'll only give a brief overview. ROS installation relies on MacPorts. I'd been holding off on installing MacPorts as it has historically been unstable, and doesn't play nicely with already installed libraries. MacPorts is now stable and well tested, but it still doesn't play nice with the rest of your system.

To install ROS I followed the ROS diamondback OSX installation instructions, all without any major dramas. I did have to alter the CMakeLists.txt as suggested in the troubleshooting section.

MacPorts gave me a few headaches, some of the errors I encountered were:
  • Error: Target org.macports.install returned: no destroot found

    To solve, use:
    sudo port clean NAME
  • Error: NAME already exists and does not belong to a registered port

    To solve, use:
    sudo port -f activate NAME

Once ROS is all installed you can source setup.sh. Remember to add in any additional packages to ROS_PACKAGE_PATH.

All in all, I wouldn't recommend running ROS on OSX, although the base ROS install works without any major dramas, installing each individual package is a bit of a struggle.

I believe ROS is an appropriate name, it really should be a stand alone operating system. I would strongly advise setting up a separate linux OS dedicated to your ROS install, running ROS ontop of OSX is quite problematic.

Wednesday, August 10, 2011

Communications and Messaging Frameworks

A common problem for any distributed software project is handling inter process communications and messaging. Luckily, a number of frameworks already exist to solve this issue. Robotics projects tend to have a range of requirements, from providing a higher level cognitive architecture framework (typically a set of interacting components or nodes) to low-level realtime communications (e.g. CANbus, RT ethernet).

Some robotics packages already provide a particular architecture, such as ImprovCV and ROS which are both filter based, or MRPT which provides some IPC primitives. However, this problem is hardly limited to robotics and many real time distributed networking/processing systems have similar communications requirements. (e.g. computer games, or traffic management systems / plant control systems).

The communications frameworks can be broadly grouped into three categories:
  1. Integrated middlewear for a larger package
  2. (hard) Real time control orientated frameworks
  3. (soft) Real time messaging libraries
From a games perspective, the people at second life have a nice evaluation of several messaging systems. Beginning with soft real time messaging, some interesting libraries include:
  • Google protocol buffers, for serialisation. I mention it, because it is popular.
  • MessagePack is a language agnostic message serialisation system similar to Google protocol buffers - just faster and better.
  • CORBA allows objects to be sent around a network, often used in business applications. It is an open standard and language agnostic as it includes an Interface Definition Language to define components.
  • Microsoft DCOM / .NET remote and its many variants are similar to CORBA, just the Microsoft variant.
  • Apache ActiveMQ another enterprise orientated message broker with support for a large number of scripting languages (Python, PHP, Ruby, Perl, etc).
  • RakNet manages distributed objects, primarily targeted towards computer games (used in Unity).

  • Open Torque Networking Library has its origins in the Torque game engine designed for massively multiplayer games.

  • YAMI4 is a fairly advanced but easy to use peer to peer messaging framework for control systems that is easy to extend to a pub/sub system. POSIX, Windows, Java, C++, well worth a look.

  • 0MQ, zero MQ is a messaging system that is socket based, and also has pub/sub extensions.

  • Socket.IO hardly qualifies for a framework, but is a great enabler for realtime HTML communications, so it certainly deserves a mention.

there are many more networking libraries available, but these are the ones that stand out for this category. For hard real-time control tasks the choices are a bit more limited, however libraries of note are:
  • Real time CORBA, is CORBA modified to be a bit more lightweight and able to support real time requirements. TAO ACE is worth looking into if your following this path.
  • Data Distribution Service (DDS) is similar to Real time CORBA, however it has the advantage of lessons learnt from CORBA and provides simpler fine grained control over quality of service and intelligent networking. RTI DDS is the framework we used for the WAMbot TARDEC/DSTO MAGIC 2010 project. OpenDDS is an open source implementation.
  • Real time data base (RTDB) is an open source framework that provides a lower-level communications interface for C/C++ over Xenomai. This is the framework we used for the Team Annieway entry to the DARPA Urban Challenge.
  • LCM is a socket-based IPC library used by Team Michigan for MAGIC 2010.
  • IPC a low level inter process communications system for embedded systems used by NASA.
  • Dynamic Data eXchange (DDX) is a large open source distributed software architecture developed by CSIRO and has been previously integrated with ROS.
Finally, there are a number of application specific frameworks, for Robotics this includes ROS, MS Robot Studio, Player/Stage, JAUS and more general packages such as Labview and Matlab. I think it is prudent for every software developer to be familiar with at least three tiers of communications frameworks, from large scale frameworks (e.g. CORBA) to light-weight frameworks (e.g. RackNet) to low-level frameworks (e.g. Socket.IO). Ideally, one for each key application area (e.g. Hard real time, Soft real time, Business applications, Web).