In the last three months, I have been exploring scenarios that brought AI capabilities into FME workflows.
Safe Software added some AI functionality to a few transformers, which should help with designing workspaces. I found my passion in making AI a part of the workflows, that is, when a workspace calls the API for enhancing the results of the translations.
I tried about a dozen different scenarios ranging from picking a geographic projection to texturing 3D geometries. In some scenarios, AI performs really well and I certainly can recommend using it for gains in productivity, quality, or knowledge acquisition. In other cases, the technology is not ready just yet, but we can say with all certainty, it is still very promising, and maybe in a few months, it really makes sense to revisit the scenarios and see how well they’re performing with all the advances in AI development.
All my findings were published on LinkedIn, before we had this wonderful new blogging platform, so I invite you to go over the article summaries below and if you find them interesting, dive into the details by following the links.
For most of the articles, DALL-E and I created images that light-heartedly and humorously depict my collaboration with AI. Have a look, you may enjoy them.
1. Texturing
The article discusses an experiment using DALL-E through OpenAI API to generate and apply textures in FME, facilitated by a custom transformer named OpenAITextureSetter available through FME Hub. This method simplifies the texture creation process, allowing for easy adjustments through FME. However, the API has some limitations, which include constraints on image sizes and proportions, which may affect texture usability on various geometries.
Another texturing article reflects on challenges I faced while using DALL-E 3 through the OpenAI API for creating textures, specifically focusing on struggles with generating seamless textures for walls, doors, and roofs. While wall textures showed promising results, achieving seamlessness seems to be impossible at the moment. Doors presented issues due to mismatched proportions with available image sizes in the OpenAI API, leading to results that cannot be used directly. Roof textures posed the greatest challenge, with problems like perspective views of a whole roof or even an entire neighborhood, extra pixels around images and inconsistent background coloring, which wouldn’t allow clipping the image to the data extents. Despite all my attempts, creating realistic top-down roof textures remains a big challenge.
2. FME Log Messages to Voice Reports
This experiment leverages OpenAI’s API within FME to convert FME log messages into audio notifications sent via Slack. By selecting a raster transformer known for detailed error messages, the experiment showcases text-to-speech capabilities. A JSON script, incorporating the error message and transformer name, is sent to OpenAI API, producing an MP3 file. This audio file is then distributed to a Slack channel using SlackConnector. Highlighted as particularly useful for extensive or complex FME workflows, this functionality represents an inventive use of AI to enhance message communication in FME projects.
3. Color Generation
In the third installment about OpenAI and FME, the focus shifts to coloring 3D objects for enhanced realism. The basic color generation within FME is not satisfactory for 3D geometries. We can use OpenAI’s API as a solution for acquiring detailed and cohesive color sets for 3D models. By setting different color types (diffuse, ambient, specular) based on OpenAI’s output, the process significantly improves the visual quality of 3D objects. The example given involves adjusting the colors of pipes in a Revit model based on their types, demonstrating OpenAI’s potential to streamline and enhance FME’s data transformation tasks.
4. Projection Selector
The article discusses leveraging OpenAI API with FME for cartographic projection selection. This method uses OpenAI to generate or find existing projections for real-time reprojection in FME, based on criteria like angle, distance, area preservation, and map purpose. A detailed prompt, created with ChatGPT’s help, requests a suitable projection in WKT format. The prompt is processed through FME to OpenAI, yielding a projection definition for direct use. The experimental OpenAIReprojector transformer is available on FME Hub.
5. 3D features from description
The article explains how to use OpenAI API with FME to create 3D geometries from textures, focusing on wrought-iron fences. By specifying a dark fence against a white background, transparency is applied to white pixels, allowing for clear fence designs. I recommend using a third-party tool called Potrace for raster-to-vector conversion, which creates smoother outcomes compared to standard FME transformers. The process includes extruding and scaling the vectorized fence for visualization, complemented by an AI-generated 19th-century Saint Petersburg building facade for augmented reality scenes.
6. Workspace description
This was a really exciting experiment that showed OpenAI capabilities to understand the contents and the purpose of an FME workspace submitted as a screenshot to the API. The workspace I submitted was laid out nicely and annotated well, nevertheless, the analysis performed by theAI is really impressive. The analysis highlighted the workspace’s moderate complexity, structured design, and potential applications in urban planning, gaming, VR, and architectural presentations. The AI response emphasizes the importance of making workspaces understandable to both humans and AI.
7. Kernel Matrices
In this article, I discuss an experiment, in which I used the OpenAI API to set parameters for RasterConvolver. The discovered issues include the API’s difficulty in generating large kernel matrices (7×7 and larger) and inconsistent results for custom filters beyond standard ones like sharpening or edge detection. Despite these challenges, I remain optimistic about future AI advancements.
8. Urban Planning
This article explores the use of DALL-E 2 for augmenting aerial imagery by filling gaps in orthophotos with contextually relevant imagery. I conducted experiments by integrating OpenAI with FME, manipulating zoning layers in the City of Coquitlam for various scenarios, such as inserting a snowman in a residential zone or replacing a city block with a high-rise building. The article emphasizes the potential of AI in urban planning.
9. Scripting
In this article, I share my insights into using OpenAI API alongside FME for automating file operations, eliminating the need for traditional scripting skills. Initially facing challenges in creating a consistent script for renaming files based on their creation date, I was able eventually to develop a complex yet effective prompt for OpenAI API, integrating it with FME’s SystemCaller transformer. This approach simplified the user experience by hiding technical complexities and allowing users to just type what file operations they would like to perform. This approach should be also applicable for using PowerShell on Windows and Bash scripts on Linux and MacOS.
10. Snowflake
In this scenario, I introduce a unique FME and OpenAI integration that enabled users to design custom Christmas cards. By starting with a simple snowflake formula from ChatGPT, I designed a workflow to create diverse, snowflake-bow-knot hybrid shapes. Users can adjust parameters to vary the shapes, which are then combined with thematic backgrounds and foregrounds generated by OpenAI based on provided prompts. This approach yields unique images every time, which are then assembled into cards using MapnikRasterizer. The service was available for the Christmas season only, but to see a similar approach in action, check the article about GeoArtAI below.
11. GeoArtAI
This was a journey of blending AI’s imaginative power with FME’s robust data handling to create location-inspired artwork. By feeding geographic coordinates into OpenAI, I developed a service that transforms these inputs into vivid, artistic representations of places around the world. This project leverages detailed prompts to guide AI in crafting images that reflect the essence of selected locations, influenced by variables like season, time of day, and landscape type. This work underscores the potential of AI to extend the boundaries of artistic creation, providing a platform for those without traditional artistic skills to realize their visions.
12. Geospatial Bias
In this article, I share a story of confronting geospatial bias within AI systems through my efforts to generate an accurate depiction of rural Lithuania using the GeoArtAI web service. Despite providing accurate coordinates, the AI repeatedly misidentified the location as Dubai, UAE, illustrating a tendency towards favoring more commonly known or discussed locations. This experience led me to coin the term “Geospatial Bias”, defined as a cognitive or computational tendency to prioritize certain geographic locations over others, despite clear geographic data indicating otherwise. The story underscores the importance of precision in prompts and the need for mechanisms to validate AI-generated results, highlighting the broader implications of reliance on AI for geographic information.
