Disease surveillance is an essential tool used by public health officials to monitor disease activities, predict outbreaks, and inform actions
Seven days before the World Health Organization announced the emergence of a novel coronavirus, BlueDot had already sent out the first warning of COVID-19. How did they do it?
Emergency responses have never had a greater need for the ability to detect and prepare for disease outbreaks. Before COVID-19, BlueDot was already helping governments, businesses, and healthcare organizations safeguard lives and livelihoods with their Early Warning System.
BlueDot uses AI to track, locate and conceptualize infectious disease spread. Its proprietary machine learning algorithms detect credible information from hundreds of thousands of sources globally. How do they get the data ready for AI? The answer is: data integration, and to do it, they chose FME, the spatial data integration platform.
“FME has been an integral solution for orchestrating data processing at BlueDot,” says Adriano Nicolucci, BlueDot’s Senior Manager of Data Engineering. With their ambitious project which relies on a huge array of datasets, BlueDot needed an enterprise integration solution that supports diverse data types and applications, meets constantly changing data requirements, and works with both cloud and on-premise databases.
Enterprise integration makes it possible
BlueDot’s early warning system depends on a huge volume of data to anticipate the potential spread and impact of nearly 200 diseases.
The data collected comes from various global sources, including government data, official health notifications, health forums, news articles, and remote sensing data. They leverage FME to integrate health-related data into their system and automate data processing workflows by having them either run on schedules or triggered by specific events. For example, they use FME to orchestrate the data processing job to scrape disease related information from websites. This way, they can capture and process tons of data in real-time.
However, only tracking the disease is not enough to prepare for outbreaks. BlueDot also tracks datasets regarding the movement of people, like flight schedules and historical travel data.
With FME, these two kinds of datasets are integrated and fed into their data warehouse (AWS), where artificial intelligence takes over to analyze the data and detect credible information.
Managing peaks in processing demands
Besides the huge volume of diverse datasets, BlueDot must manage constantly changing data requirements. As a result of the pandemic, an explosive amount of information related to COVID-19 entered their system, which dramatically increased their data processing needs.
Every new insight was critical, and BlueDot needed to ensure they would not miss anything.
Using FME Flow, they could use as many engines as needed across multiple machines to manage peaks in processing demands.
Scaling through enterprise integration
Working with different types of applications, BlueDot leverages FME to build enterprise integration patterns as they provide standardized data integration logic and are easy to set up. With automation capabilities in FME, they can easily set up data processing jobs to be triggered by events in decoupled applications or services. Since the patterns are built into the automation with no coding needed, they can easily build complex enterprise integration solutions in a more efficient and scalable way.
Making data accessible on-premises and on-cloud
BlueDot uses FME to feed data into their data warehouse where artificial intelligence takes over to analyze the data and detect credible information, and then synchronizes processed data with their products so that their health expert team can further assess the data and identify potential health threats to the public. With FME’s support for cloud databases, they are able to support their clients by delivering data and analytics insights to clients’ preferred cloud data storage locations, such as Amazon S3, ArcGIS Online and Snowflake. Those analytics insights include social distancing metrics from the neighbourhood to the national level as well as risk of infectious disease importation from known hot spots.
With an enterprise integration solution in place, BlueDot is able to monitor nearly 200 diseases around the world on a daily basis in order to identify disease activities and contextualize outbreaks. By automating data processing workflows, they can free up time for the team to focus on other projects. Being data-prepared at the beginning of an outbreak is vital to timely emergency response, helping governments, healthcare organizations, businesses and individuals mitigate risks and safeguard lives.
