A bunch of links on the nearest-neighbor problem, for higher dimensions:
And, for good measure a set of comparisons of optical flow algorithms:
I'll probably stick to the OpenCV defaults anyway, but its nice to know there are options. It would seem Bruhn et al. is the most accurate..