5 Data Integration Workflows for Utilities

Utility companies, including small utilities and cooperatives, work with a huge range of data systems, from legacy databases, to spatial information collected in the field, to business records. All of this data is vastly different in nature and structure, but all of it needs to work together in order to enable business operations and decisions. Here at Safe Software, we see utility companies using FME to integrate and distribute data for:

• design and planning,
• operational efficiency and cross-team communication,
• asset management,
• outage and leak management,
• distribution network maintenance,
• sharing data with the public and stakeholders,
• and much more.

Let’s look at the 5 main data workflows we see for utilities, along with real-world examples of these workflows in practice.

1. Synchronize data across systems and formats

To form a complete picture of all available information and facilitate decision-making, data needs to be integrated from various systems – including across teams, from external sources, and from both geospatial and business software like Esri, Autodesk, Smallworld, Bentley, Oracle, ERP software, and others. Plus, all of these systems need to stay up to date.

This is why a lot of utilities agencies implement data integration workflows, whether to bring data from various sources into one central repository, or to ensure all disparate systems stay synchronized. An application integration workflow enables every team to use the best fit-for-purpose applications, including databases, engineering, operations, and finance, while keeping everything connected.

For distribution network maintenance and operations, it’s helpful to centralize data into a single database, making it easier to analyze the data, share it with stakeholders, generate reports, and more. Running this data integration workflow on a schedule or as new data arrives ensures the central database is updated on an ongoing basis and critical network information is always available.

Here are some examples of utilities that use data integration workflows to centralize information.

• Hampton Roads Sanitation District implemented an online data contribution service to help centralize wastewater collection data from 14 localities. Data is automatically validated and loaded into the repository’s data model for use by the GIS team.

• Traverse City Light and Power uses FME to move core business data between several disparate databases and applications, including Outage Management (OMS), work order system, utility billing/CIS, Integrated Voice Response (IVR) system, and an energy efficiency application.
• Grant PUD uses FME to automate data integration across all departments of their organization, including building a connection between finance and engineering, facilitating communication with their Customer Information System, data validation for accounting, and more.
• Forsmark Power Group worked with our partner, Sweco, to integrate nuclear power plant data into a single database and share the results with authorized users via a secure web map. Data is integrated from AutoCAD, Shapefile, Excel, Revit, and IFC, and stored in an Oracle Spatial database. The system meets security standards and regulatory requirements.
  

2. Data validation and quality control

Data integrity is critical, and so implementing an automated quality control workflow is a common task for utility companies. FME has transformers to identify, repair, and filter out bad data, like the GeometryValidator and AttributeValidator. Validation reports can be generated as PDFs, spreadsheets, HTML pages, or in other formats, and automatically sent to stakeholders.

• Learn how: Data validation and QA tutorial

It helps to validate data before integrating it into the system by creating a self-serve data validation portal for field crews to upload and validate their submissions. Running data validation workflows regularly also ensures the integrity of all datasets.

Quality control is a critical part of every data integration workflow. Here are two examples of companies who use FME for their everyday tasks, with data validation being a key part of it.Georgia Power uses FME to automate many everyday data integration tasks, from simple format conversions to data validation and analysis. Automating these processes saves hours of time and ensures data is accurate. Their Metering Services team also uses FME for several tasks, like to locate specific installed meters that need replacement, to compare several databases and detect financial discrepancies, to process a large database and use business intelligence to generate a “Morning Report”, to detect anomalies in databases, and more.

PNM uses FME to automate many data integration tasks in their GIS department, including data conversion, extraction, reporting, analysis, and quality control. Many workflows are run on a schedule to ensure stakeholders always have access to up-to-date data.

3. Migrate data between GIS, CAD, and other systems

Data migration is a common task, not just to move data from legacy systems, but also to convert and integrate new data that arrives every day. For example, many utilities need to convert data from CAD to GIS or from GIS to CAD on a regular basis.

FME is also great at facilitating the process of migrating to the Esri Utility Network. Learn how to do this in our tutorial, “Esri ArcGIS Pro Utility Network Migration Tools”, and in this Esri Utility Network Migration training course.Central Contra Costa Sanitary District migrated their GIS data model into the Esri Local Government Information Model. FME helped them manage and transform the data model without losing data integrity, including attributes, line segments, and sewer main features.

Enstar Natural Gas Company & Alaska Map Science used FME to migrate AutoCAD drawings into their new GIS mapping system (Esri ArcSDE Geodatabase). FME enabled this CAD-to-GIS translation without losing any information.

Louisville Gas & Electric integrated pipeline design data from CAD into their GE Smallworld database. They used FME to integrate the incoming data, perform quality control, rubber sheet the data to a common landbase, and insert it into GE Smallworld.

4. Deliver and distribute critical information

It’s important to make information accessible by sharing data in various formats, empowering the end user to get data in whatever format they need – whether that’s an easily shared PDF, a rich GIS data model, a spreadsheet, or a dataset for loading into business intelligence software. We often see companies creating data portals for the public, design groups, contractors, or other clients to access. For example, Sunesys built a OneCall Automated Response System.

• Open Data portals: Learn more about creating a data portal in this webinar and ebook.

• FME examples: Learn how to build a data distribution service in FME.

For emergency and risk management, making data accessible means responders and agents can be equipped with critical information no matter where they are. This empowers them to deal with weather emergencies, outages, leaks, and maintenance using the most up-to-date information. For example, you can configure a data integration workflow to receive incoming data from sensors and field personnel and respond according to business rules to alert the right people.

Critical information can also be sent to customers, like to inform them of outages and anticipated uptimes in their area on easily accessible web maps. For example, Manitoba Hydro made power outage data available to customers through an interactive web map.

FortisBC used FME to assess wildfire threats to gas and electric assets, enabling their Operations Managers to take immediate action. The workflow involved integrating wildfire data with their Smallworld environment on an ongoing basis, and making it available in various formats for Emergency Responders to access.

5. Embrace new technology – drones, AR, smart grids, and more

One of the most exciting things about FME is how easy it is to experiment with cutting-edge technology and data types. It’s straightforward to integrate new sources like drone imagery, or to reroute your data to different output formats like game systems, or to apply various analytics and transformations to your data, like machine learning algorithms.

Here are a few new technologies that utilities are experimenting with, plus some ideas and inspiration:

• Connect to imagery from drones/UAVs, like what Alabama Power Company is doing.
• Connect to satellite imagery and data like Landsat, Sentinel, and Planet.
• Output data to augmented reality to visualize underground infrastructure on your mobile device.
• Build an interactive visualization that can be explored in a browser.
• Output data to a game environment like Unreal Engine to let stakeholders walk around and explore infrastructure in a virtual world.
• Connect to sensor data to build a smart grid, facilitating network monitoring, management, analytics, and planning.

Connect to cognitive services to gather intelligence from your data, like natural language processing and computer vision algorithms.Sharper Shape uses FME to build solutions for their power distribution clients, for example, to identify and prioritize the removal of vegetation and other objects that could impede client networks. When the European Space Agency (ESA) wanted to identify the value of satellite data for power distribution asset management, Sharper Shape employed FME for the task.

*

Hopefully you found these examples inspiring as we explored what utilities around the world are accomplishing using data integration workflows. Automatic quality control, analysis, data conversions, report generation, data distribution, and so much more can be accomplished when data is harmonized, information is made accessible, and new technology is embraced. By running these powerful workflows on schedule or in response to an event, these companies are automating tasks that used to take hours, days, or even weeks of manual effort. This not only reduces the risk of human error, but also frees up staff to focus on building innovative solutions.

To learn more about data integration for Utilities, watch our webinar, “Meeting the Demand for Quality Utility Data”. Our Small Utilities subscription model is designed specifically to help small utilities and cooperatives deploy FME, while our Enterprise subscription model is a good option for larger utilities. Contact us to learn more.

Further reading:

• [Blog] 8 Inspiring Ways Utilities are Automating Data Workflows
• [Blog] 6 Data Integration Tips for Electric, Gas & Water Utilities

In addition, stay tuned for a new utilities webinar on June 3rd. Registration coming soon on our webinars page.