What if you could process 1.09 million handwritten records in just 26 hours instead of 19 years? AI is revolutionizing data pipelines, making them faster, smarter, and more efficient than ever before—and enabling previously impossible projects like this one.
In this blog, we’ll explore real examples of automating complex data workflows using Google Gemini and FME. We’ll also look at how Google uses FME to efficiently transform and integrate external data into Google Maps.
For a deeper dive into the topics discussed below, including demos and workspaces, watch our webinar: All-Data Any-AI Integration: Innovations with FME and Google.
Google & FME: Transforming External Data for Google Maps
Mapping the world is an ongoing challenge that requires integrating vast amounts of ever-changing data. Google Maps not only generates its own data but also ingests information from users and third-party sources. This includes images, addresses, indoor maps, road closures, building footprints, transit data, and much more.
Since external providers often deliver data in different formats, Google must continually transform these varied schemas into its internal model. This is where FME plays a critical role.
FME Flow serves as a bridge between external providers and Google Maps’ internal data pipelines, ensuring seamless transformation and integration.
Here are three key ways Google benefits from FME:
- Real-time data streams. FME Flow Hosted retrieves high-velocity data via a secure WebSocket, enabling live status updates for charging ports across a European country.
- API integration. When human operators need to compare an external dataset with internal models, FME Flow fetches and processes the required data.
- Google Cloud Platform automation. FME Flow streamlines data processing by triggering workflows whenever new data is added to Google Cloud Storage.
Ultimately, Google’s goal is to maximize automation and integration while minimizing the need for human intervention. AI and FME are essential to achieving this efficiency.
AI for Schema Mapping: Standardizing Inconsistent Data
Handling data from multiple sources presents a significant challenge: schema inconsistencies. For example, address data may arrive as a single string in one dataset but be split into multiple fields in another. How can organizations efficiently standardize such disparate data formats?
To solve this, we combined FME’s dynamic workspace capabilities with an AI agent powered by Google Gemini. Here’s how it works:
- Automation triggers the workflow – An FME Flow Automation monitors incoming files and determines whether the data is spatial or tabular.
- Data conversion – An FME Workspace converts the data into a standardized CSV format.
- AI-powered standardization – The CSV is sent to Gemini, which intelligently parses, categorizes, and resolves inconsistencies in the data.
- Final output – The transformed dataset is now clean, structured, and ready for downstream use.
AI was beneficial for schema mapping here because it:
- Drastically reduced the processing and QA time.
- Handled complex data ambiguities with precision.
- Processed large data volumes easily.
- Reduced manual effort and cost.
This is just the beginning—AI-powered automation will continue to enhance scalability and efficiency in data pipelines.
Use Case: Digitizing a Million Historical Records with Gemini Flash
As part of a UK power network digitization project, Avineon-Tensing needed to extract structured data from 1.09 million handwritten service record cards dating back to 1907. Manually transcribing this data would have taken a full-time employee 19 years—but AI did it in just 26 hours.
Gemini Flash is a multimodal AI model that can process text, images, video, and audio. Its high token limit makes it well-suited for large-scale data extraction tasks. Compared to other AI models, Gemini Flash is both cost-effective and highly efficient for lower-intelligence tasks.
For this project, Avineon-Tensing integrated FME with Gemini Flash to fully automate the digitization process.
This project is a game-changer for historical data digitization, proving that AI-powered automation can make large-scale data transformation fast, cost-effective, and scalable.
Learn more about FME from Avineon-Tensing: participate in their Free FME Skills Booster training. They are also running an FME AI Skill Booster on March 4, 2025.
Become an “All-Data, Any-AI” Industry Leader
To stay ahead in a data-driven world, organizations must embrace all data types, locations, and velocities:
- All Data Types: GIS, CAD, BIM, 3D models, satellite imagery, point clouds, augmented & virtual reality, and any AI services and models. Spatial data is critical for unlocking real-time insights.
- All Data Locations: For optimal performance, data should be processed where it resides, whether on-premises or in the cloud.
- All Data Velocities: Real-time data streams are more prevalent than ever, requiring organizations to adapt to fast-moving data.
FME was designed to handle all data formats and integrate with any AI service, making it an essential tool for modern organizations that seek to maximize the value of their data.
By embracing AI-powered automation and FME, businesses can streamline data integration, enhance decision-making, and unlock the full potential of their data pipelines.
