Batchtransform multiple shapefiles with FME

Question

I would like to batch-transform multiple shapefiles which are stores in several different folders.

I used the batch deploy with the option "recreate source folder structure" which works well for creating the folder structure.

What's messed up are the attributes of the shapefiles. The "feature types to read" seem to be bound to the shapefile I used to insert the shapefile-reader.

Is there something I can do to transform the shapefiles with the correct attributes (and without additional FME-specific attributes)?

( I used a workbench with shapefile reader-->csmap-reprojector --> shapefile writer but tested also a version with a creator, feature-reader, csmapreprojector and feature-writer).

• JimT: have you tried a schema reader and then a feature reader?
• Alex P: Have you Checked on the "Merge Feature Type" box in the Reader with "Merge Filter set to * ? Have you also set the Reader with the "Dynamic Schema Definition" box checked on?
• Mark Ireland: It would need a dynamic writer too. Better to recreate the workspace at this point, using the dynamic option in the generate dialog. That should solve the issues.

Answer 1

So the method to do this is fairly straightforward. I created a quick demo video here: https://www.screencast.com/t/7FIU8u4o

Written down the steps are:

1. Generate a workspace. Use Select Multiple Folders/Files to pick a source folder and choose the subfolders option. Make sure the Workflow is set to Dynamic:

enter image description here

2. Add any transformers you want to the workspace, eg you had a CSMapReprojector. Preferably you don't use any that change/use attributes, since they will vary per file.

3. Save the workspace and choose Run> Batch Deploy. In the Batch Deploy wizard be sure to set the Recreate Source Folder Structure option:

enter image description here

FME now runs and translates your data. Because it's a dynamic translation, the output has the correct attributes and not the attributes of just one of the files.

Again, see the above video for a demo/proof of success.

• Dale Lutz: In addition to Mark's tip, there is a Tutorial at knowledge.safe.com/articles/1050/… that walks through the process in more detail.

Answer 2

You are using FME (Feature Manipulation Engine) to batch-transform multiple shapefiles stored in different folders. However, you are encountering issues with the attributes of the shapefiles, specifically with the "feature types to read" being bound to the shapefile used to insert the shapefile reader. You want to transform the shapefiles with the correct attributes and without any additional FME-specific attributes.

To address this issue, you can follow these steps:

1: Open your FME Workbench and ensure that the "Recreate source folder structure" option is enabled in the batch deployment settings. This will ensure that the output folder structure matches the input.

2: Add a Shapefile Reader to your workspace. Set the reader's parameters to read the shapefiles and their attributes. Make sure to set the "Feature Types to Read" parameter to "Read from Dataset" instead of selecting a specific shapefile. This allows FME to dynamically read the feature types from each input shapefile.

3: Connect the Shapefile Reader to the desired transformation or reprojector transformer (e.g., CSMapReprojector) in your workspace.

4: Add a Shapefile Writer to your workspace. Set the writer's parameters to write the transformed shapefiles. Ensure that you enable the "Don't write FME-specific attributes" option in the writer's parameters. This will prevent any additional FME-specific attributes from being written to the output shapefiles.

5: Connect the transformation or reprojector transformer to the Shapefile Writer.

6: Save your workspace and run the batch transformation process. FME will now read each input shapefile dynamically, preserving their attributes, perform the desired transformation, and writing the output shapefiles without any additional FME-specific attributes.

By following these steps, you should be able to batch-transform your shapefiles while maintaining the correct attributes and excluding any unwanted FME-specific characteristics in the output.and in addition you can check this https://www.kychub.com/dynamic-workflows/ and you can use this tool.