FME Blog

11 Reasons You Should Have a Data Integration Plan

Tiana Warner

April 9, 2014•9 min

Data integration means combining information from various sources into something useful. A data integration plan efficiently manages data and makes it available to those who need it.

A data integration plan – what is it and why should you have one? Data integration means combining information from various sources into something useful. It’s about efficiently managing data and making it available to those who need it. Both a technical and a business process, IBM defines it as “discovery, cleansing, monitoring, transforming and delivery of data from a variety of sources.”

Below are 11 reasons why integrating data and having a data integration plan are essential to technical and business practices.

1. Every data type has its strengths

Every data format was designed for a reason. Each one represents information in a way no other format can, with unique attributes, metadata, structure, and schema. Integrating data from different formats adds various levels of specialty to the dataset.

data integration plan to make the most of formats like lidar — Integrating data from various sources, like AutoCAD, TIFF, and a LAS point cloud, enables you to leverage the best of every format. Image: 5 Ways to Optimize Your LiDAR Data

For instance, CAD and GIS data can be integrated to enhance a CAD drawing with specialized GIS information and attributes. The City of Henderson is an example of a CAD-GIS data integration project, where they created a digital plan submission portal for construction projects.

2. Take advantage of specialized applications

Similarly, every application was designed for a reason. (Though this may be hard to believe, given the existence of Shark Fingers.)

Take advantage of specialized applications for a data integration plan — These are just a handful of popular applications that handle data in highly specialized ways.

Let me modify that to say, every piece of software that works with data represents, analyzes, and transforms information in a specialized way. By integrating data into a format accepted by that application, you’re giving yourself the power to open and use your data in that software.

For example, North Ayrshire Council used data integration to streamline HR operations by centralizing data in Salesforce and Microsoft Office.

3. Reduce data complexity

Reduce data complexity for a data integration plan — A data integration solution simplifies the interaction between diverse systems, like in this 3D PDF.

April Reeve describes it well: “The number of potential interfaces between applications in an organization is an exponential function of the number of applications. Thus, an organization with one thousand applications could have as many as half a million interfaces …”

Thrashing around in that web of connections will only get you tangled. Forming a data integration plan, on the other hand, is like entering that web with a machete. Data integration is about managing complexity, streamlining these connections, and making it easy to deliver data to any system. This might involve creating a data hub that’s easy to publish to and subscribe to.

For example, Cochranville Ag Service used data integration to enable users to access and download complex data from their preferred environment of Salesforce.

4. Increase the value of data through unified systems

Bringing disparate datasets together increases the value of the information. Examples include:

merging with and leveraging external data (e.g. from vendors);
combining data of varying natures (structured, unstructured, spatial, tabular, web, raster, big data, etc.);
applying spatial information to non-spatial data;
combining databases from different repositories;
creating a uniform schema for a group of datasets without metadata standards;
bringing collections of data from different sources into the common GML structure for compliance with INSPIRE (see examples).

BlueDot, for example, created an early warning system that depends on a huge volume of data to predict the potential spread and impact of nearly 200 diseases around the world. The data is collected in real-time and fed into a data warehouse where artificial intelligence can then analyze the data and alert to any outbreaks.

5. Make data more available

Centralizing your data makes it easy for anyone at your company (or outside of your company, depending on your goals) to retrieve, inspect, and analyze it.

Easily accessible data means easily transformed data. People will be more likely to integrate the data into their projects, share the results, and keep the data up to date. This cycle of available data is key for innovation, knowledge-sharing and continuing to develop your data integration plan.

For example, Balfour Beatty VINCI uses innovative in-house apps to make validated data available across the enterprise.

6. Easy data collaboration

Easy collaboration — Data integration makes it easier to collaborate on potentially overwhelming information.

With accessibility comes easier collaboration. When a data integration plan is in place, anyone who works with your data will find it easier to use brain power now. They can actually use the data in the format they require. Whether collaboration involves sharing among internal teams and applications, or across organizations, integrated data is more complete because it has more contributors.

For example, the Norwegian Environment Agency with the UN Environment Programme World Conservation Monitoring Centre created a spatial tool that helps to protect biodiversity. The tool functions as open software to allow multiple countries and stakeholders to use it.

7. Understanding data means smarter business decisions

Integrated data means transparent processes within your company. With a data integration plan, people are offered the flexibility to use your data in whatever system, you’re giving them the opportunity to better understand the information. It’s much easier – and more informative – to navigate through organized repositories that contain a variety of integrated datasets.

Applying location intelligence to your dataset, for instance, makes it spatially comprehensive and offers new levels of insight around that dataset, which leads to better decision-making.

For example, Fortis BC uses data integration to automate the assessment of current wildfire threats to its buildings, customers, and gas and electric assets. Integrating with other data types can also mean conforming to interoperability standards to save on business costs. Further, open source data formats offer many advantages for businesses.

8. Data integrity

Data integration technology should cleanse and validate the information passing through. Obviously, we all want our data to be robust and high quality. A data integration plan ensures data is free of errors, inconsistencies, and duplication.

For example, the City of Oshkosh’s Child Abduction Response Team, enabled real-time mapping for faster and more accurate search and rescue responses. A data integration strategy ensures City of Oshkosh’s Child Abduction Response Team’s critical data is accurate, up to date and is of high quality.

9. Keep all systems synchronized

Make your data live — Integrating with cloud systems is one way of enjoying the benefits of live data.

A data integration plan makes it easy to keep information up to date automatically. One input can propagate across all integrated systems, keeping your data current. In fact, your data can even be real-time if a server or cloud solution is part of the integration strategy. Automation can keep data both up to date and integrated seamlessly, especially in response to an event or whenever new data arrives.

For example, Surrey Heath Borough Council uses data integration to ship legacy data to the cloud, run scheduled data validation workflows, display live RSS feeds, and synchronize data across the organization in near real-time. With up-to-date data across the enterprise, Surrey Heath can focus on analysis and planning to better services to the community.

10. Connect to data streams

A data integration plan enables you to use automation to connect to real-time data sources. Data streaming is the continuous transfer of data. This occurs at a high speed. Data streams are always collecting data from different and many sources. Hence, automating this process makes it easier for you to keep all your information up to date and integrated as you gain new and real-time data of all kinds.

11. Automate sharing

Data integration workflows allow information to be delivered automatically to who needs it and in the format it is needed in. Data can be shared on demand in open data portals, apps, or automatically delivered via email. Once an integrated data solution is in place, all your current information is updated and integrated across all systems, applications, and so on. Since everything is connected, you can automate sharing what, who and when.

By having an active data integration plan, you avoid the dreaded data silos and gain the boundless potential outlined above.

Any data type can live in a wide variety of sources. Developing an integration strategy—both the technical and business aspects—is critical to assuring your organization’s data reaches its maximum potential. Learn more about data integration and how FME technology can help your organization develop a data integration plan – no matter how big or small your data is, spatial or tabular, structured or unstructured, open or proprietary, or all the above.