Bringing the Power of the Cloud to FME Desktop

Calling all FME Desktop users…

• Do you often run FME workspaces in FME Desktop and run out of compute power?
• Would you like to free up your desktop when you run large, long jobs?
• Do you wish you could be notified when your job completes?
• Do you have a need for FME Server but can’t afford the up-front costs or don’t have the infrastructure to deploy on premises?
• Do you already have some of your data in the cloud?
• Would you like to have the power of the cloud at your fingertips?

If you answered yes to any of these, FME Cloud can help. There are a lot of benefits that FME Server brings you, but I am not going to tackle these here: I am purely going to look at leveraging FME Cloud as a tool to enhance existing FME Desktop workflows. We’ll walk through a scenario that provides you with an extremely cost effective way to leverage the benefits of FME Cloud.

What is FME Cloud?

FME Cloud is essentially just a new way to deploy FME Server. Rather than having to find hardware and install FME Server locally, we manage the hardware for you and there are no up-front costs. You just pay for the FME Server on an hourly basis. What this means is you can provision an extremely powerful FME Server for one hour to perform a task for you and then shut it down.

Making FME Cloud easy to use

With FME Cloud, you can launch as many hosted FME Servers as you wish using a launch wizard in the web interface.

What makes things especially interesting is we have created an API on top of FME Cloud that enables you to provision FME Cloud capacity programmatically. Using the API, I have built a series of custom FME Cloud transformers that let you do the following:

• FMECloudInstanceLauncher – Allows you to launch an FME Cloud Instance.
• FMECloudInstanceController – Pause/start/reboot or terminate your FME Cloud instance.
• FMECloudBalanceChecker – Returns the total remaining credits and an estimate of the current monthly balance.

For the first time, FME capacity can be provisioned on demand to meet your needs!

Coupling FME technology with throwaway compute power and relaxed licensing really opens the doors up. To showcase this, I have built a workspace that does the following:

This is an extreme example, as we are launching, running, and terminating an instance in one workflow. Crucially, it ties the FME process directly to the hardware so you only need to pay for the hardware when you run the process. This means you can dramatically reduce server idle time, which saves you $$$.

Using these transformers in various combinations, you can see how easy it is to provision FME capacity, and then using the FME Server REST API, configure the server and run the job. Here are a few workflows I can think of that might be useful:

• Offload computationally demanding jobs to FME Cloud and free up your local hardware for other tasks. Our largest machine has 8 cores and 68GB of RAM, so it is very capable.
• Automatically launch instances to parallelize jobs across multiple engines and even multiple instances to slash processing times.
• Unpause a previously configured server that performs a series of operations on awakening.
• Check the FME Cloud balance for the current month and then pause running instances if you go over a certain threshold.

Conclusion

Even without all of the benefits of FME Server, this is a game-changer for FME. You can provision huge amounts of FME processing capacity to run your data intensive jobs, with an extremely simple and measurable pricing model. Instances from 1 core with 3.7GB of RAM up to 8 cores and 68GB of RAM can be launched to cover all of your pricing/processing needs.

If anyone has any workspaces they are thinking of running on FME Cloud, email them with the data to support@safe.com. We now have a test-suite setup that will tell us the instance size and number of engines that are optimal for that specific workspace.