The National Information Exchange Model: An XML-based Framework for Data Sharing
I am at the GEOINT 2011 conference this week in San Antonio. One of the major things we’re talking about is NIEM, and how FME 2012 can help people work with NIEM data. Indeed, looking at FME 2012, one might think that it was built with NIEM in mind.
What is NIEM?
NIEM or National Information Exchange Model is a multi-agency framework which defines how to share data. To say that the vision of NIEM is ambitious is an understatement. However, the benefit if successful is massive, and could greatly increase the efficiencies of communication across and outside of government.
The interesting thing about NIEM is that this framework is not trying to dictate data models that all must use to participate as other projects attempt to do. Rather, NIEM leaves the definition of these data models to experts through the definition of IEPD’s (Information Exchange Package Documentation)
This delegation of model definition through IEPD’s is very different from other approaches and I would argue increases its chance for success. Other data infrastructure projects that I can think of (spatial or otherwise) mandate the data model that must be used by all, resulting in either a hugely complex model that is difficult for anyone to use, or a model that is so simple, that it is useful to nobody.
I am constantly amazed at how so many people don’t view complex data models as a barrier to sharing data. The folks behind NIEM do understand this as from my experience thus far, I have found the data models to be refreshingly simple and even pragmatic.
NIEM – Another Reason To Love XML
The number of systems that must interoperate with the data of NIEM is truly mind-boggling. It encompasses everything from local law enforcement, to 911 systems, to fire, to Amber Alert warnings.
Rather than define a new data encoding the folks behind NIEM chose an open standards approach that can be embraced by all. It should come as no surprise that the data exchange protocol for NIEM is XML!
When I first looked at NIEM I got discouraged as the data model itself is absolutely massive with 1000’s of different data types! Looking a little closer I found that to perform a particular task – say “Amber Alert”, “Commercial Vehicle Collision”, or “TrackIEPD” – I have to understand only the classes that are relevant to what I am trying to do, and these are relatively easy for any domain expert to understand.
Different organizations use different systems and this is not going to change. Indeed, we do not want this to change as it is important for all organizations to be able to adopt the system that does the best job for them and to be able to continue to innovate.
Indeed with FME 2012 it is easy to work with or produce the NIEM (XML) data enabling FME to easily move the data to and from different systems.
NIEM – The Need for Event Driven Architecture
Looking at NIEM and it becomes clear that much of the data being disseminated through NIEM is very time sensitive. Getting an Amber Alert far and wide as soon as possible is critical to successfully recovering a child. In fact, pick any emergency event and getting information to the responders that need it as soon as possible is critical to saving lives.
To support a fast response, systems are needed which are architected to consume events and trigger the appropriate event processing. As events can come fast and furious in a full blown emergency it is important that the system design reflect an ability to separate the event logging/recording from the event processing to avoid losing or dropping events. It should come as no surprise that FME 2012 Server has an event driven architecture, enabling it to do precisely what is described here.
NIEM – From Pull to Push, and the Power of Social Media
There are many different event processing and notification systems, and again expecting organizations to adopt one standard is not going to happen. To be successful any event system must be able to consume events from other event systems, and must also be able to push events into other event systems. The ability to “ripple” events across a disparate network of systems is what NIEM is all about. It is left as an exercise for the reader to identify such a product. (Hint: It must love XML and be able to move data in about 300 different systems.)
In a public emergency the power of social media cannot be ignored. We have all heard how Twitter has been used by authorities to locate people in an emergency.
Why not use social media to notify the public? It isn’t a big jump to view Twitter as a public event notification system. I could argue that each “tweet” is in fact an event. Using hash tags I can spread a message far and wide. Tweets are small (like event notices) and support location. For some types of NIEM events, notifying the public is key. It would be hard to come up with a more cost effective way than to simply embrace the existing infrastructure of social media.
NIEM, GEOINT, and Your Challenges
If you are at the GEOINT conference and would like to talk about NIEM and what we are excited about, then I invite you to visit me at our booth, or catch my presentation today at 4:30pm when I’ll be sharing a few demos.
If you are working with NIEM data or want to work with NIEM data let me know your challenges and comment below. After all NIEM data is XML and you know how I feel about XML.
