The Quest for the Ultimate Format
“All of this has happened before and all of it will happen again.”
The new Battlestar Galactica series has quite a following at Safe, and we’re all anxiously awaiting the final two episodes. But as we reflect on how the show’s recurring prophesy that “all of this has happened before and all of it will happen again” plays out, I was reminded that this statement also applies to the world of spatial data formats.
Discussion surrounding Alex Willmer’s recent blog posting “Shapefile 2.0 Manifesto” asks what, if anything, can and/or should be done about the fact that the most popular format seems to be the humble shapefile, a specification which ESRI made public more than a decade ago. The conversation reveals that the various shortcomings of the shapefile format can be an irritant to some users who lament its ubiquitousness. And so they search for a better alternative, a new format which will address these shortcomings and, they hope, become as popular as shapefiles are today.
Industry veteran Geoff Zeiss was talking with me and some customers recently at a conference when this topic came up. He stated with conviction that from his experience, there is an unwritten law of spatial data that new formats will continue to be developed and that nothing can be done to stop that. Our graph of formats supported in FME since we began does drive home the fact that there continues to be more and more data formats developed.
Why? Because as technology evolves, as needs evolve, as data types evolve, the formats that served one purpose or another fall short in the new situation, and so new formats are born. But the old ones don’t go away. And that is why I am confident that we’ll be using shapefiles ten years from now. We might be using other things too, and maybe we’ll have a slightly newer take on shapefiles that addresses some of the issues folks have today, but they will be there. And so will new formats, new GML dialects, new SQLites, new web and raster and CAD and BIM formats that we can’t even imagine today.
And ten years from now, there will be a posting somewhere about how the world would be a better place if we could just settle on a new format that solves whatever issue we have at the time. And I’ll comment on it then too. “All of this has happened before and all of it will happen again.”
Just as Galactica (so far) represents the heartbreaking failed quest for a home (Earth), perhaps the quest for the ultimate format is similarly and ultimately doomed to end unsuccessfully, and we’ll just all have to learn to survive with what we have. On the other hand, perhaps there will be a happy ending for Galactica and they will find their earth, and we will find our ultimate spatial data format. But I’m not holding my breath…
