FME Evangelism Weekly Issue #7
Introduction
Back after a little break… well I did say this would be weekly-ish.
Again this issue is more related to use of FME than new development, since we’re still in the final phases of the 08 release.
Contents
- Training Resources for FME 2008
- New Format: Oracle GeoRaster Reader
- RasterRotationApplier Transformer and @ApplyRasterRotation function
- Random Number Generation via a Web Service
- New Format: Directory and File Pathnames Reader
- FME Group on Facebook
- Data Cleanup and Validation
1) Training Resources for FME 2008
The training materials for FME 2008 are complete (phew!) and will be debuted at the FME User Conference training courses.
In order to achieve our two main aims – training as many users as possible, and providing value for certified trainers – for 2008 we have a number of different training packages aimed at different levels of skill and expertise.
- At the top of the pile are certified trainers. They receive the full set of materials including advanced modules, instructor notes and training certificates, plus receive other certification benefits such as exposure on our web site.
- One step down, users who have applied to our certification program receive more a set of detailed materials to help them bring their training skills up to a certifiable level.
- At the next level, resellers and educational establishments can download a set of basic materials for general educational use by non-certified trainers.
- At the lowest level, users are able to download a set of basic self-study materials as PDFs.
The Online Learning page of the Safe web site will connect you to all public training resources, including the FME Tutorial and the Self-Study materials (nb: the 2008 materials are still awaiting uploading), while the other specialist levels are available either on demand or through the reseller and certification portals.
2) New Format: Oracle GeoRaster Reader
The Oracle GeoRaster reader is a new format for FME2008.
This reader nicely complements the Oracle GeoRaster writer that already exists, since raster data can now be both imported and exported from an Oracle database using FME.
This is an entirely separate reader to the Oracle vector reader, but is documented within the same section of the Readers and Writers Manual.
The table pickers for our Oracle readers are able to tell between vector and raster tables, and will only show the table when it contains the relevant type of data.
The latest builds also include the ability to choose a specific pyramid to read.
The Oracle GeoRaster reader supports most of the numeric and color interpretation types on both bands and palettes supported by FME. It does not support some of Oracle’s more advanced types such as 128 bit Complex Numbers. A search envelope may also be applied to the reader to restrict data retrieval to an area of interest.
The reader supports SRS georeferencing, including rotation of raster features.
3) RasterRotationApplier Transformer and @ApplyRasterRotation function
These two items allow a user to apply a pre-defined raster rotation to a raster feature.
We (Safe) use this function in various places, and have exposed it so that users may also make use of it.
The reasoning behind this is slightly complex – even to a raster user – so here’s a brief outline.
A rotated raster feature – for example a feature from Oracle GeoRaster – is typically stored horizontally with an attribute that records the rotation.
When FME reads the data it keeps it in this structure until it needs to process the feature in a way where rotation would matter – for example clipping.
Sometimes you – the user – may wish to force immediate rotation of the raster in order to inspect or process the data in some way.
For example, the bounding box of a rotated raster may be greatly different to the bounding box of a raster with rotation value.
When you apply this value you can do so in a number of interpretations. As usual these include Nearest Neighbour, Bicubic, Biliniear, Average4 and Average16
Setting a nodata value for the raster is recommended since rotation usually results in areas that are outside of the main raster values.
Note that the value of the rotation is stored as a trait of the geometry when the data is read, is not accessible as a format attribute and bears no relation to fme_rotation
To set a value for this trait you can use the Rotator transformer.
4) Random Number Generation via a Web Service
For quite some time, many forward-looking FME users have suggested the future of spatial data translation and transformation lies in accessing web services. Perhaps the generation of random numbers was not quite what they had in mind, nonetheless it is a good example of a web service that FME can access to carry out a specific (and perhaps specialist) task.
Chief FME Inquisitor Dmitri Bagh has been experimenting with this particular web service and has some great examples.
See http://www.fmepedia.com/index.php/Using_Random.org_service for an index to his workspaces and transformers.
The key benefits of this service – over the in-built RandomNumberGenerator transformer – are:
True Randomness: Dmitri explains why generators such as are found in TCL are not truly random
Random Strings: Strings include characters as well as numbers, making this great for Alpha-Numeric IDs or random passwords
Unique Values: A set of guaranteed unique values is a useful tool for assigning non-duplicate IDs (or just picking lottery numbers)
Dmitri’s transformers can generate either a number of new features with random values, or assign random values to existing features.
The really clever part of his work is to get all of the required values with a single HTTP request; the reason being that 1 request for 1,000 random numbers is way quicker that 1,000 requests for a single number (2 seconds as opposed to 7 minutes)! It’s a very interesting case of FME Best Practice.
Kudos to: The folks at random.org for making such a great service available
5) New Format: Directory and File Pathnames Reader
An experienced FME user will recognize that a new feature in one part of FME – although important in that area – can often be used for many other purposes. Often the uses to which a new function is put are not something that we at Safe had even considered. I’ve a feeling that the Directory and File Pathnames format will turn out to be such a case.
This new format takes a specified folder and returns a list of all its sub-folders, files, or both.
A feature is output for each file/folder in the list, the attributes of which include path name, file name, file extension and type (directory or file).
The intended use of this is for some component of FME Server.
However, I’ve a strong feeling that some of our more experimental users will find different ways to use this; particularly in conjunction with the RasterReader or AttributeFileReader transformer. Maybe as a QA tool, custom logger or… who knows what?!
Any FME user who is a member of Facebook may be interested to know that there is a specific Facebook group set up by some of our FME users.
It looks like a great place to communicate with users on a more personal level, for example sharing photos of Safe related travels or activities.
The link to the group is: http://www.facebook.com/group.php?gid=8670341457
7) Data Cleanup and Validation
If you have browsed the Safe blog recently, you’ll have noticed an article by Pro Services head Mark Stoakes on using FME for data cleanup and quality assurance.
Outside of format translation, this must be one of the top uses of FME, so this is a very important subject.
The article – available here as a PDF – explains to what degree FME meets all of the different requirements of the IS19100 international quality standards, and highlights some example projects where the Pro Services department has produced custom solutions for various clients and data types.
For more information on data validation solutions and other services offered by Safe’s Professional Services team, please visit Safe’s website safe.local/services.
8) FME Artwork Example
This egg-box design was created by Dmitri as he experimented with some 3D surfaces.
Dmitri says: “I wanted to make an example of a TriangleStrip. I made a box, tiled it, replaced with points, assigned elevation as the product of a row and a column multiplied first by sin or cos, and built TIN. Now I wonder whether I can use CSGBuilder for making eggs 🙂 I’ll let you know.
You can find the whole set of FME inspired art work on fmepedia at:
http://www.fmepedia.com/index.php/Category:FME_Art
Brief Notes
- Long format names continued to be tweaked to find a consistent naming structure – but should all be finished now
- A fix is implemented for the Concatenator transformer to prevent attributes being wrongly published when a custom transformer is duplicated
- There have been a couple of WMS fixes to make sure SRS and HTTP Proxies are all being used correctly
- Unicode handling for the S-57 reader has been implemented
- The Oracle Raster Writer now allows easier pyramid creation by using default values for settings the user has not specified
- A double-click on a Workbench published parameter now brings up a dialog to edit the value, rather than define the parameter. Right-Click > Edit will now bring up the parameter editing dialog. This is a change in behaviour that may catch a few users by surprise.
This week’s Weekly was written to the tune (if you can call it that) of…
Florence Foster Jenkins singing the Queen of the Night aria from Mozart’s Magic Flute.
