The importance of the OGC & open standards
The OGC, Open Geospatial Consortium, is the leading international organization for coordinating standards related to geospatial information. One of their central goals is to promote data access, sharing, and interoperability. In other words, make data more FAIR – Findable, Accessible, Interoperable, and Reusable.
The OGC partners with a wide range of organizations, from universities to government agencies like NASA, USGS, NRCan and UKEA to private companies such as Google, ESRI and Microsoft. The OGC also hosts a very active and vibrant user community that meets regularly to discuss a wide range of topics of specific interest. These include industries such as utilities or local governments and technical areas such as 3D, remote sensing, or IoT/sensors.
The standards made by the OGC are considered “open standards”. This means they are freely accessible, publicly available, vendor-neutral, and not constrained by license fees or patents. They’re important because they reduce barriers for everyone to access and share data. They also provide a common means to structure and exchange information. This reduces technical risk and makes it easier to develop systems that more easily interoperate.
Note that open standards does not mean open source. The two are often associated with each other and can be confused. Open standards can be supported by both closed source proprietary software and open source software. It just so happens that open source software has a natural affinity for supporting open standards, perhaps because they both share a similar underlying philosophy of openness.
Why Open Standards?
In the rough and tumble world of application development and data analysis, it can be tempting to only write code for the solution you need at a given time. However, this ad hoc code and resulting custom data structures gets to be more and more unwieldy as time goes on.
We’ve all seen mountains of Excel spreadsheets, shapefiles and database versions that can be overwhelming to make use of, especially when one department or organization needs to exchange data with another. There is also a ‘many-to-many’ problem. For every organization involved in sharing or accessing data, you have an increasing amount of data translations that need to take place. Say there are 10 organizations working together. This means 10 x 10 possible data transfers taking place, resulting in 100 possible translators needed to move the data.
By defining a common open standard for data exchange, you have a common exchange point. Everyone can develop one translator from their internal data model or standard to the external community standard. This way everyone can exchange data with minimal effort.
Reducing Data Constraints
Another motivating factor for open standards is the desire to open up competition. Without open standards, users can be held hostage by vendors who define closed formats that only their tools can access. By utilizing open standards, users can choose best of breed tools for each application and move data between them with open methods. It also promotes security and privacy because developers can code to an open specification without having to share sensitive data.
The trend towards automation and the rich information products they support makes data sharing and the standards needed to support it all the more important.
In the past, many systems worked in isolation within siloed environments. A good example of this is a flood or climate forecast model. The isolation of data systems, structures, and services is fine for abstract research when results are published and reviewed over months or years. However, today we face increasing demands to integrate across multiple models and applications. It’s not enough to only have one good climate or forecast model. Multiple models need to be integrated together so comprehensive assessments can be made based on all the latest available data, and forecasts can be combined to assess cumulative risk. This is just one example of where open standards are essential to scaling the complexity of problems that our information systems can address.
The Importance of Sharing Data
It is true that it is possible to collect and digitize your own data in order to produce the map you need for some geospatial applications, like performing your own GPS survey and making a map from it. However, the more complex the problem you are faced with, the greater the need to obtain data from other sources.
Two classic fields where data sharing is essential are disaster response and environmental management. In both cases, analysts need access to a wide array of data sources from a wide variety of domains (meaning any application area such as industry, subject, or theme) and technologies to answer questions and support decision makers. When it comes to these domains, lack of information or delays in obtaining information can cost lives, lead to infrastructure damage, or result in resource or habitat loss.
Selecting an Open Standard
Sometimes when designing a system, there is no prescribed standard that you must support. In this case, one of the key things to understand when choosing a standard is what type of standard you need and what use cases you need to support. Data standards can be divided into four primary use types: storage, application, exchange and streaming.
Example of each of these are as follows:
- Storage: OGC Geopackage, PostGIS
- Application: KML, IMDF
- Exchange: CityGML, WaterML, GeoTIFF
- Streaming/Services: GeoRSS, GeoJSON, WFS, 3D Tiles
If you select the wrong standard for a specific use case you may end up fighting the constraints of that standard. For example, if you try to store very large datasets in GML you may have a lot of trouble reading the whole dataset just to access one feature since GML is not spatially indexed. If you publish your data to an application-oriented format such as KML or PDF, you may have difficulty using that data in an exchange mode since they are typically a lot easier to write to than read from. So, to reiterate, it is fundamental to understand the underlying purpose of the standard to make sure it aligns with your use case.
Other Standards
The main focus of this article is OGC Open Standards, but it is worth mentioning that there are many other standards related to geospatial data. These include but are not limited to:
- BSI (Building Smart International) for BIM data
- INSPIRE for geospatial data in the EU
- ISO (International Standards Organisation) for a wide variety of spatial and other standards
- IETF for JSON & GeoJSON
- W3C for XML, HTML etc.
Recently, many of these standards groups have been looking to collaborate and coordinate between standards to improve compatibility. Examples of this include the new OGC API services which involves coordination between OGC and OpenAPI. There has also been increasing coordination between OGC and BSI on GeoBIM topics.
Using OGC Standards
To use data supported by OGC standards, you’ll need access to tools that support OGC standards. There are many options out there that support a range of open standards for different use cases such as:
- Web: Google Earth
- GIS: ArcGIS, QGIS
- Data Integration: FME
Finding OGC Standard Data
Not sure where to look for open-source OGC standard data? The answer is: everywhere. Fortunately, many data publishers prefer to avoid vendor lock in so they like to make their data available in an open standard format in addition to any proprietary format they might support.
Government agencies such as USGS, NOAA, and EU INSPIRE continually update datasets and make them available via publicly accessible geoportals. It can also be handy to do a web search like “US flood WFS” or any other sector/service keyword combination. You will be amazed at how easy it is to locate data sources and services related to various OGC standards.
The above shows data from FEMA DFIRM Flood Mapping WFS. Learn more how this data is used in presentation FME for Disaster Response
Note that when you are looking for data it is important to understand the purpose or use you wish to make of the data and to choose the best standard for that use. For example, many agencies publish both WMS and WFS. WMS is often the easiest to use because it generally returns an image file such as PNG. While this map image may be useful as a background map, it usually is not helpful if you are looking for feature attribute or geometry information. In the latter case you would be better off locating a WFS or OGC API Feature service.
Publishing Data with an OGC Standard
Once you become familiar with OGC standards, you may come to the point where you wish to publish your own data according to one of the standards. Many agencies require standards compliance in order to submit data to them. Some examples include:
- BC Forests ESF GML
- EU INSPIRE GML
- FAA AIXM
Generally speaking, producing data compliant with an open standard typically requires more in depth understanding than it does to consume open standard data.
You need to understand both the subject matter involved (ex. 3D buildings in the case of CityGML) and the technology used for the encoding (ex. GML in the case of CityGML). Visual based tools like FME or ESRI Data Interoperability make this process a lot easier because they allow you to import the standard’s data model directly into your data production tool and then it’s mostly a matter of transforming and mapping your source data to match that schema.
Challenges with Using Open Standards
While it is true that standards can often simplify data exchange, the wider the application for a given standard, the more complex it can become, and the more challenging it can be to implement.
For example, using the standard GPX XML to store a GPS point is much simpler than a standard like INSPIRE GML due to the nature and purpose of the standard. Even within a standard, there are simple and complex use cases. GML is a very large standard that supports scores of geometry types and it’s likely that no one has ever implemented the full standard. However, GML profiles exist to simplify implementation so only selected aspects of the standard are used. For example, Simple Features only implements the simplest geometries such as points, lines and polygons. The idea of profiles is very important when working with standards because it allows users to benefit from the ability to exchange data using the standard, without having to implement all the complexities of that standard.
Integrating Standards with Other Data
Ideally, all our data would be compliant with one standard or another making data interoperability that much easier for everyone. The reality is we will always have customized applications built on proprietary software that interacts with proprietary, application-oriented data models.
This is simply a natural result that comes from making tools to fit specific workflows for specific solutions. The ongoing challenge is the need for tools which can simplify the process of transforming data from internal application specific models to external community standard models. For example, many OGC standards employ nested or object oriented data models that can be difficult to reconcile with relational models typically found in GIS systems. Model based transformation tools that have support for both proprietary formats and open standards are ideally suited for supporting these types of transformations.
One of the best examples of a data integration tool that can do this is FME. FME can also support automation so that proprietary systems and external client applications can be kept seamlessly in sync even though the data models they rely on are radically different. An example of this might be a proprietary CAD or GIS system that publishes data to geopackage on S3 which in turn is used to publish KML and OGC API REST services to client applications.
What’s Next with Open Standards
Recently, there has been a big push at the OGC and other open standards groups like Open API to modernize standards and bring them up to par with current trends in IT and web infrastructure. REST protocols and JSON encodings are examples of current deployment approaches that are much more web, mobile, and cloud friendly. Another strong trend is cloud support and cloud optimization as well as offline processing support through standards such as OGC Geopackage.
Naturally, there is a continued strong focus on enterprise service-oriented architectures with more of a scalable message and data stream focus and less emphasis on file based processing.
Aside from using open standards to consume and produce data, you can also get involved in an open standards community near you. The OGC meets quarterly in what are called Technical Committee meetings. These have a combination of member-only and open events that anyone can attend for little or no cost.
There are specialized meetings hosted by the various standards working groups, often with fascinating plenary sessions that look at the latest trends from engaging keynote speakers. A recent plenary had an engaging session sponsored by NASA with astronaut Stephen Bowen.
Safe Software & the OGC
Safe Software, the makers of FME, have been an OGC partner for many years and recently upgraded to become a Technical Member.
Standards support has always been central to our goal of promoting data sharing, integration and automation since the use of standards simplifies the problem of data integration considerably. It is almost always easier to move data between two well known standards, than to move the equivalent data between ad hoc data structures.
Because of this, there has always been a close alignment between the goals of Safe and the mission of the OGC. Safe has also participated in numerous OGC pilots and plugfests including IndoorGML Pilot, Maritime Limits and Boundaries Pilot, OGC Testbeds, CityGML plugfest, and a number of domain (maritime, aviation, utilities, 3D) and standards working groups (CityGML, MUDDI, IndoorGML).
Safe has had a long term commitment to open standards support, and given the recent trends, we have taken the opportunity to both quickly adopt the latest generation of emerging OGC API REST/JSON based web services such as OGC API Features, as well as address gaps in our existing open standards support.
We are also making an increasing effort to better document, register, and communicate the depth of our open standards support and provide better learning resources for those who want to work with them. While we have some of the best support for GML in the industry, including dynamic application schema and complex geometry support, this has often been rather understated.
We live in a rapidly changing world with growing challenges including climate change, population growth, resource scarcity, and hazard management that only seem to grow more severe, unpredictable and complex over time.
On the other hand, a proliferation of data sources and data collection services means we have access to more data than ever to help us tackle these problems.
The central challenge to address this is data integration and automation. We must understand how we can extract what we need from the deluge of available data with minimal effort, and bring it into common reference and semantic models so that it can be used to support real world problem solving in a timely fashion.
Open standards, such as those provided by the OGC, provide a fundamental foundation upon which this type of dynamic data integration can be built. The evolution of open standards promises to serve as a key stepping stone to help us address the daunting challenges of both the present and future, whatever they may be.
To learn more about open standards, both the OGC and others, see these resources:
