6 Data Integration Tips for Electric, Gas & Water Utilities

Transmission and distribution network data is a lifeblood for utilities companies. Because the data is maintained by various departments and contractors, best-of-breed software is often used by each group. This can make it difficult to acquire a complete picture of the distribution network.

From speaking with users in the utilities space, I’ve gained many insights for integrating utilities data, which I’ve boiled down into the six tips below.

1. Bring legacy data into the current dataset

Starting at the beginning is a very good place to start. It’s a no-brainer to port over your existing dataset when you’re starting up with a new system, as Marcin Goreki at Atmos Energy did.

But even if you’re not starting fresh, archived distribution network data can provide great value. Business intelligence can be gained by comparing old data to the latest information and performing analytics to project the future. For example, it can point to marketing opportunities where there is rich transmission network infrastructure but increasing customer churn, or it can examine data for locations where distribution bottlenecks and low churn rates are both present, indicating where infrastructure updates could boost profit.

Integrating this old data with your current data involves getting acquainted with the old data model and determining how it can best be ported over. As one of my co-workers says: “Figuring out how  the source and destination data models should connect is the hard part. The easy part is moving it over!”

2. Translate external data into the current systems

This is the easy part: converting information from one system to another for a complete view of the transmission network.

Many utilities find this problem hard to solve. I hear far too many stories from new FME users about previous workflows with several partial solutions stitched together, and data quality diminishing with each new translation. Like the game of telephone, far too much is lost in translation!

Alternately, a custom coded solution is developed. But when modifications are (soon) required, costly redevelopment effort ensues. Worse, the original contractor may be too busy or unavailable to make the changes, resulting in a new contractor who must painstakingly review the system and recommend changes.

But it can be so easy! Reliable integration involves moving all of the data in its fullest form over from its source system into the destination, making it usable to others while preserving its full glory.  I speak to so many customers who are excited that they just created a system that’s visually readable, can be easily modified by the least technical individual on the team, and stays up-to-date despite inevitable system changes and updates.

3. Keep distribution network data up-to-date

“We needed this yesterday!”

Data integration doesn’t just involve one-off translations that run when the need is urgent.

True integration comes from identifying an automation schedule that works for everyone within the company. This may be as simple as a nightly update, or as complex as on-demand updates, triggered by business logic, that set off notifications to various people and systems.

When Cindy Tuttle at San Antonio Water System described the impact of automated workflows, she said: “We would have needed to double our team to accomplish what we did with a few people’s effort. In fact, we estimate the money saved in our first year alone is nearly one million dollars.”

4. QA is essential

No matter where data is moving, it’s only worth its weight in truth. If data is inaccurate, incomplete, or only partially populated, the translation efforts are wasted. By setting up quality assurance tests and running them before data is imported, errors can be identified and fixed before problems arise.

Again, automation is key. The best QA tests are the ones where the individuals submitting the distribution network data can run the tests themselves. This is because they can receive feedback on their data immediately, and if necessary, correct errors – saving a lot of costly back-and-forth. Clay Wise of  Hampton Roads Sanitation District did just that and commented: “We can now keep our centralized database current in a fraction of the time it would take to do this manually.”

5. Provide self-serve access to transmission network data

For data to be valuable, it must be accessible – in useful forms – to the people who need it. Increasingly this means providing a self-serve process where data can be downloaded in multiple formats for use in a variety of databases and CAD or GIS systems. This frees data managers up from responding to ongoing emailed requests, and provides the end user with instant information.

6. Tap into the power of “where”

I never cease to be amazed when I encounter large, multi-million dollar companies who have yet to leverage location information. Network distribution data is inherently tied to the earth. Most systems recognize the location aspect of the data, and ensuring that this information is maintained as part of the dataset helps not only day-to-day functions of the business, but also marketing, growth opportunities, and operational efficiency.

Integrating utilities data in real life

I recently caught up with Lassi Tani from Spatialworld at the FME International User Conference. He helped client Helsingin Energia, one of the largest energy companies in Finland, get started with integrating and analyzing their business data. Their most popular output so far is easy-to-use PDF maps for decision makers. Listen in on my video interview with him:

As Lassi said: “Think of what you want to have, don’t think of what you can do. Because FME can do everything.”

What tips would you give to data managers who need to integrate transmission network data?
Share in the comments below.