How to Put LiDAR and Point Clouds in Their Place

“Large LiDAR datasets usually have power over their users, but with what you’ve done with FME 2011, users will now have power over their LiDAR data.” That’s what Gene Roe of LiDARNews.com said to me last week during a quick conversation about the new point cloud functionality in FME 2011. It’s true. You can now do a lot with your massive point cloud datasets and I’ll get to that after sharing what we’ve learned about point clouds.

The Right Way to Approach Point Clouds
Regardless of whether you call it LiDAR or point clouds, this data type poses substantial data management issues. The key breakthrough comes when you began to see point clouds for what they truly are – misbehaved (dare I say “evil”?) rasters. Yes, rasters can be large – horribly large even. Yes, rasters can have many bands. Yes, the raster pixel values can have different data types. But even though it took us multiple years to finally beat rasters into submission with FME, I’m sorry – rasters just aren’t very evil. You can always easily find the value for a particular coordinate by doing some simple math. And really, how scary is that?

But point clouds? You have files with even more horribly-huge numbers of samples in them. And these points aren’t spaced in any regular way. In fact, there’s no way of really knowing how they’ll be laid out spatially until you read them all (thank goodness for the spatial index that the libLAS folks provided, is all I’ll say). For a given spot on the earth, you could have multiple readings! Each reading is a trove of information, potentially including color values, intensity, classification, return #, as well as a precise x, y, and z. So what we have is a very rich dataset, increasingly less expensive to collect and increasingly more common, but very unruly to manage.

So how did we wrestle down the point cloud menace? For one thing, we applied the hard earned lessons we learned from our raster experience and modeled point clouds in FME just as we model rasters – handling each point cloud as an atomic unit in our workflow. We went out and talked with a large number of our users to find out just where their point cloud pain points were. We documented these scenarios, then went away and built the infrastructure. We also called in expert help – in particular, Howard Butler of libLAS (which we used for its spatial index and file I/O and chipping), as well as the folks at Pointools (who worked with us to add POD file reading and writing).

And we worked very hard to make sure this new data type integrated with all the other data types of FME in a reasonable fashion, which turned out to be a much larger task than we imagined at first, but also opened the door to a wide range of powerful data integrations (like the one in the video below where I read, clip, reproject, and create a surface model from LiDAR data, and then combine it with imagery to create an interactive 3D PDF).

GIS Folks: Take Control of Your Point Cloud Data
The net result? Well, I hadn’t thought of it this way until after speaking with Gene, but I believe that in GIS shops everywhere, the once mighty and powerful point cloud files are now cowering with fear, knowing that at any time their new masters may be sending them out on a transforming date with FME 2011 and that they’ll be coming back cut into pieces, split into parts, vertically reprojected, scaled and offset far from home, or worst of all, thinned into a pale shadow of their former selves; at last completely ready to have their intrinsic value wrung out of them by the application of their users’ choosing. Point clouds, say goodbye to your PhD In Horribleness.

Do you have unruly point cloud data in your shop? If so, let me know if the functionality described in my video above, or in Dmitri’s Point Cloud Lab, addresses your point cloud pain points. I’d love to hear what you plan to do with your point cloud data – drop me a line.

Bad Horse had allowed LAS files into the Evil League
of Evil Data solely on the basis of their sheer evil size.