Removing spatially duplicate features using ModelBuilder?

Question

I am creating a model to remove features which are duplicated within a polygon feature class. Most of the features are duplicated once, but about a quarter have up to three or four exact copies. I do not have ArcInfo, so I cannot use Delete Identical.

My model should contain the following:

1. Make Feature Layer on the input feature class specified by the user to output Output_FL
2. Get Count to count the number of rows within the feature class
3. Create a variable counter and initialise to 0
4. Add a While iterator where counter < row count
5. Select Layer by Attribute on Output_FL to create a NEW_SELECTION where OBJECTID = counter
6. Select Layer by Location on Output_FL where the relationship is ARE_IDENTICAL_TO
7. Select Layer by Attribute on Output_FL to REMOVE_FROM_SELECTION where OBJECTID = counter
8. Delete Features from Output_FL
9. Increment counter

I have finished all the steps except step 9. I am not sure where/how I should increment the counter. I'm also not sure if I should put a Copy Features tool somewhere before the while loop as a backup mechanism since this model will be modifying the input data, and not everyone will make a backup of their data before running the model.

• user19717: How do you integrate 'Feature OID' in the SQL query ??? I try "%Feature OID%" = OID and other type of request and its not working
• fatih_dur: You may want to have it a go the solution in gis.stackexchange.com/a/219606/28687

Answer 1

You should try this, Add XY coordinates to your featureclass, then Dissolve on the X,Y fields. This should remove your stacked, duplicate features. You can then join the attributes from your source file. You may need to concatenate X,Y to form a single field for the join.

• Get Spatial: +1 - I may have duplicated her model, but this is a far simpler solution.
• Cindy Jayakumar: I first tried intersecting the feature class once, which extracted the polygons where it overlapped itself, then joined it to the original feature class and deleted the matches. I forgot that it would extract each duplicate as well, so this method did not yield my intended outcome. I am going to try your method first as it seems it will be the fastest, before attempting the model again.
• Cindy Jayakumar: I used the Shapes to Centroids tool from XTools Pro so that each duplicate poly would have the same centre point. I added the field CONCAT_XY to my original polygon feature class. After adding XY coordinates to the centroids, I used a spatial join to join it to my original FC. I then dissolved based on CONCAT_XY, imported my original table structure from the XML schema, and loaded all attributes. Out of ~29 000 features, I now have ~24 500, which means about ~4 500 polys were duplicated. Thank you for the guidance!
• sparky: This is simple yet genius! Thank you.

Answer 2

Here is a model that looks like it duplicates the process you have listed above. There are a couple of things different than the procedure you have outlined above.

Instead of using a while loop to increment through the features, there is a Feature Selection Iterator that loops directly over the features in a featureclass. What is nice about this iterator is that there are two outputs, the selected feature, and an attribute value of that feature. If you don't specify the attribute, it defaults to the Object ID, which is what you need to deselect the initial feature.

The value that is returned is used as part of the expression for the Select Layer by Attribute tool. It is referenced as an Inline Variable. This removes the initial feature from the selection.

The last interesting bit is the use of Feedback in the iteration. This is represented by the blue line from the "Output Feature Class", to the "Input features". This allows you to tell the input features to use the output feature class as their source, after the initial parameter definition. This removes the need for a counter to effect a loop of some sort.

I have not tested this, and there are a couple of places I am not sure of, for example the correct way to set up an expression using the OID field on the layer. This may be tricky because the field name has to be surrounded with brackets or double quotes depending on if it is coming from a personal or a file GDB respectively.

Hopefully this will give you some ideas to work with to structure your model.

Here is a link to a screenshot of the model: Remove Duplicates Model

Remove Duplicates Model

• blah238: What is the 'Select Identical Features' tool in this model? There is no built-in tool by that exact name (there is a Find Identical tool, but this also requires ArcInfo).
• Get Spatial: @blah238, It is actually the Select by Location tool, with the relationship being "Is Identical To". I'm going to update the model with some notes, then update the post.
• Cindy Jayakumar: Could you please provide a link to the image of your model? All the ideas you have given sound great!
• Get Spatial: @Cindy - If you right click on the image in the answer, you should be able to view it. It will be larger. You can also just right click and save the image.
• Cindy Jayakumar: That's the problem, I cannot see that there is an image in your answer, I only know there is because you mentioned it, there isn't even a red X or "Image cannot be displayed" or alternate text to indicate that there is an image supposed to be there. This has been a problem for me when accessing this site from work, I can never "see" the images people refer to in their questions and answers. I don't think it's a browser problem, as I have the latest versions of Firefox and IE, both are set to "Show Pictures" and still I can't see anything.

Answer 3

You may want to look at XTools and ETGeoWizards. XTools has a tool called Find Duplicates in the Table section. ETGeoWizards has a tool called remove Exact Duplicates under the Geoprocessing section. You can use both tools in a Model in Model Builder.

• Cindy Jayakumar: We unfortunately do not have a licence for ETGeoWizards, and we have been having major problems with our IT department,as in they remove things from our machines randomly such as XTools, which I cannot install again, and anyway, no licence for Pro. So far we have been manually merging/deleting all the overlap errors (thousands and thousands and thousands...) and needless to say, it is not working out. I also want to create this model so that it is readily available to us for future projects.
• Cindy Jayakumar: erm... Now that I have XTools installed again, I used the Find Duplicates tool, which gave the same results as my model. Thanks for pointing it out.