APIs and webhooks are interfaces that allow an application to communicate and exchange information. Passing information to or from an API or webhook can happen on demand, on a schedule, or in real-time whenever new information is available.
Let’s discuss the difference between APIs and webhooks, how connecting to them works, and how to set up automated workflows to integrate them and process the information in a customized way.
APIs vs. Webhooks: To Poll or To Listen?
Connecting to an API (Application Programming Interface) involves authenticating with the API, sending a request or “polling” it as needed, and getting back the requested information. A REST API is a type of protocol that uses a URL to call a resource and usually involves defining the HTTP method (i.e. to GET, POST, DELETE, or PUT information), plus a header and a body.
Webhooks use a URL to listen for new data rather than polling for it. It works by allowing the user to register a URL with the application or service, and then the application or service will push data to that URL whenever an event occurs. With webhooks, the URL receives information automatically instead of requiring the user to manually check as with a typical API. This is advantageous because you don’t need to repeatedly poll the API for new information, which can be taxing on both the client and server.
Connecting to an API means polling for data, while webhooks mean listening for data.
Think of a webhook like a delivery service that rings your doorbell when a package arrives, as opposed to the API letter carrier who leaves it in your mailbox for you to retrieve at your leisure. The difference is whether you receive the package as it arrives, or go check your mailbox at intervals.
Why would you choose to check your mailbox when you can receive the package the moment it arrives? Well, not every application offers webhook “delivery services”, so sometimes you have no choice. But even if the option is there, sometimes an API is favourable if you only need to check for new information at intervals (e.g. to update a web app every Monday, or to pull new data if/when a client submits a request). APIs also tend to offer more comprehensive interfaces than webhooks and can be used to send data to the application.
How to Connect to APIs and Webhooks without Coding
The simplest way to connect to an API or webhook is to leverage a third-party tool like FME. Create a custom FME Workspace to connect data and process it, then use FME Server to run the workspace automatically and facilitate an ongoing application integration workflow.
The most common method of connecting to an API in FME is with the HTTPCaller transformer, which formats the HTTP request automatically without any coding involved. FME also has transformers that connect to APIs for specific services like ArcGIS Online, Autodesk A360, Cesium ion, SharePoint Online, and a lot of custom transformers on FME Hub, so one of those might be more suitable and even easier than the HTTPCaller.
Connecting to a webhook instead of an API involves using a trigger in FME Server, which we’ll talk about in the section below. But whether an API or webhook is used to connect to an application, the Response Body (i.e. the requested data that comes back) is most often in JSON. This is where the JSONFragmenter transformer is useful for parsing it. If the response comes in another format like XML, FME also has transformers to process that.
The next step is to take action based on the information that’s received. FME can read 450+ data formats and has hundreds of transformers, so the possibilities for data integration and transformation are endless. If the API uses pagination (i.e. it sends the data in ‘pages’ instead of all at once), FME can be used to loop through the pages and process the data as it comes.
With an FME Workspace created to read, process, and output the data, the next step is to automate it using FME Server.
This FME Workspace sends a request to an API using an HTTPCaller and processes the result using a JSONFragmenter.
What about authentication?
Authentication differs for each API and is specified in the API’s documentation. For example, there is HTTP authentication, web tokens, and OAuth 2.0. In FME, Web Connections are a safe and convenient way to store and reuse connection parameters, making it easy to log into web services and connect to APIs.
Automating the Workflow
Running workflows automatically is the key to reducing manual effort and keeping applications integrated. In FME Server, Automations is a graphical interface that makes it easy to build event-driven workflows. For example, Automations can achieve the following:
- On a daily schedule, poll the API to fetch up-to-date information.
- When a client email arrives, poll the API for the requested data.
- When a file is added to a directory, poll an API for updated data.
- Process and share data as it is received from a webhook.
Note the difference in how they are used: a webhook is a trigger that can launch an FME Workspace to process the information, while polling an API is an action that takes place within an FME Workspace and is done in response to a trigger (e.g. a schedule or event). FME Server can also deliver data to an application through a webhook, which is discussed in detail in the article “Webhooks and FME Server”.
This user is building an FME Server Automation that uses a webhook as a trigger to kick off the workflow.
Tip: APIs change, and an Automation can be configured to send an alert to the server admin if the workflow fails due to a change in the API.
Get Started with APIs and Webhooks in FME
Whether the goal is to connect to an API to request up-to-date information as needed or to connect to a webhook to listen to new data, FME makes this possible without needing to code. Here are some related webinars, tutorials, and training resources:
