Ministry for Primary Industries & Locus drive environmental monitoring efficiency

A need to eliminate data management inefficiencies

Didymo, an invasive freshwater pest, has been causing damage to lakes and rivers in the South Island of New Zealand since 2004. Biosecurity New Zealand (the biosecurity arm of the Ministry for Primary Industries (MPI)) had been responsible for managing the sampling data related to Didymo through the Institute of Environmental Science and Research (ESR). The data, including site status, lab results, and visual results, was stored in a SQL database and made available to the public through a website. 

Outsourcing the management of this data to external parties had resulted in some inefficiencies, prompting Biosecurity New Zealand to seek an in-house solution.

Streamlining processes and increasing Didymo impact awareness

To facilitate the in-house management of Didymo sampling data, the initial step involved migrating the data. A backup of the SQL Server was provided and subsequently restored on the UAT SQL server. Given the data’s presence in multiple tables, MPI’s Data and Information team employed FME to cleanse and transform it into a geodatabase format.

What followed was the hosting of data in ArcGIS Online, where sampling site points and related visits information were stored, maintaining a 1-to-many relationship. 

New sites and visits could now be created using ArcGIS Experience/Survey123, a browser-based platform. Views were implemented on the master layer to restrict access to specific user groups, and dashboards were developed for visualizing the data. FME Flow webhook automation was employed to streamline certain processes, such as updating site status values based on new visit submissions.

The FME-powered data migration process ensured efficient data wrangling and conversion to a geodatabase, facilitating best practice data management and analysis. The utilization of ArcGIS Online as the new solution allowed for improved data visualization through dashboards and a superior user experience. 

The integration of Survey123 with custom URL hyperlinks supported the pre-population of survey questions with relevant data, and FME’s usability made it easier for colleagues to understand and update site status values based on complicated criteria. 

By implementing FME Flow webhook automation, the processing time for updating site status values when new visits were submitted was drastically reduced from 1 minute to an impressive 16 seconds. Additionally, the Didymo Samples Hub Public Dashboard (still in UAT), provides a publicly accessible view of the sampling data, ensuring transparency and increasing awareness about the impact of Didymo on water quality and biodiversity.