Remove objects with duplicate properties from Swift array

Question

The question here involves removing duplicate objects from an array:

Removing duplicate elements from an array in Swift

I instead need to remove objects that are not themselves duplicates, but have specific duplicate properties such as id.

---

I have an array containing my Post objects. Every Post has an id property.

Is there a more effective way to find duplicate Post ID's in my array than

for post1 in posts {
 for post2 in posts {
 if post1.id == post2.id {
 posts.removeObject(post2)
 }
 }
}

• dfrib: stackoverflow.com/questions/25738817/…
• Cristik: First result in google search for "Remove duplicate objects in an array swift" goes to @dfri's link...
• Oscar Apeland: @Cristik The linked reply did not answer my question, as I want to check for duplicate properties in an object, not duplicate objects.
• Cristik: @OscarApeland please then update the question, nor from the title or the contents cleary results that you need the array filtered by a property, not by the object itself.
• dfrib: @OscarApeland That was never my intention, my apologies if you perceived it as such. I simply marked this question (in it's current form) as a duplicate, mostly due to it's title. As Cristik writes, edited questions can possibly be re-opened again. Next time, possibly be more specific, e.g. asking how to use the linked approach (duplicates of "regular" array) to apply to your array of objects, specifically their property id. Finally, we're all here to learn and teach each other, never be afraid to asks questions, we also learn from how we ask, how to ask next time.

Answer 1

I am going to suggest 2 solutions.

Both approaches will need Post to be Hashable and Equatable

Conforming Post to Hashable and Equatable

Here I am assuming your Post struct (or class) has an id property of type String.

struct Post: Hashable, Equatable {
 let id: String
 var hashValue: Int { get { return id.hashValue } }
}
func ==(left:Post, right:Post) -> Bool {
 return left.id == right.id
}

Solution 1 (losing the original order)

To remove duplicated you can use a Set

let uniquePosts = Array(Set(posts))

Solution 2 (preserving the order)

var alreadyThere = Set<Post>()
let uniquePosts = posts.flatMap { (post) -> Post? in
 guard !alreadyThere.contains(post) else { return nil }
 alreadyThere.insert(post)
 return post
}

• Oscar Apeland: Thank you for your answer. Sadly, order matters to me, as I want the posts to show up chronologically.
• Luca Angeletti: @OscarApeland: No problem, I added another section (Solution 2) where the original order is preserved.
• user3441734: when creating Set, the Set internally check if it contains(post). than you do the same again. I don't see any advantage of this approach. Even though my own answer was down-voted, conformance to Hashable and creating the Set is not necessary and finally less effective. please, check my answer ...
• Luca Angeletti: @user3441734: I didn't downvote your question. I think you are talking about my Solution 2. The alreadyThere.contains in my code is mandatory because I need to know if the post I am evaluating is already present in the array I am building and returning. You cannot remove that part. if you want unique values.
• J. Goce: Ive been coding in Swift for almost 2 years and I only understand Hashable and Equatable with your explanation :) I thought it was a super function I will not able to understand. thanks

Answer 2

You can create an empty array "uniquePosts", and loop through your array "Posts" to append elements to "uniquePosts" and every time you append you have to check if you already append the element or you didn't. The method "contains" can help you.

func removeDuplicateElements(posts: [Post]) -> [Post] {
 var uniquePosts = [Post]()
 for post in posts {
 if !uniquePosts.contains(where: {0ドル.postId == post.postId }) {
 uniquePosts.append(post)
 }
 }
 return uniquePosts
}

• help-info.de: Code-only answers are considered low quality: make sure to provide an explanation what your code does and how it solves the problem.

Answer 3

A generic solution which preserves the original order is:

extension Array {
 func unique(selector:(Element,Element)->Bool) -> Array<Element> {
 return reduce(Array<Element>()){
 if let last = 0ドル.last {
 return selector(last,1ドル) ? 0ドル : 0ドル + [1ドル]
 } else {
 return [1ドル]
 }
 }
 }
}
let uniquePosts = posts.unique{0ドル.id == 1ドル.id }

Answer 4

based on Danielvgftv's answer, we can rewrite it by leveraging KeyPaths, as follows:

extension Sequence {
 func removingDuplicates<T: Hashable>(withSame keyPath: KeyPath<Element, T>) -> [Element] {
 var seen = Set<T>()
 return filter { element in
 guard seen.insert(element[keyPath: keyPath]).inserted else { return false }
 return true
 }
 }
}

Usage:

struct Car {
 let id: UUID = UUID()
 let manufacturer: String
 // ... other vars
}
let cars: [Car] = [
 Car(manufacturer: "Toyota"),
 Car(manufacturer: "Tesla"),
 Car(manufacturer: "Toyota"),
]
print(cars.removingDuplicates(withSame: \.manufacturer)) // [Car(manufacturer: "Toyota"), Car(manufacturer: "Tesla")]

Answer 5

My solution on Swift 5:

Add Extension:

extension Array where Element: Hashable {
 func removingDuplicates<T: Hashable>(byKey key: (Element) -> T) -> [Element] {
 var result = [Element]()
 var seen = Set<T>()
 for value in self {
 if seen.insert(key(value)).inserted {
 result.append(value)
 }
 }
 return result
 }
}

Class Client, important have the class like Hashable :

struct Client:Hashable {
 let uid :String
 let notifications:Bool
 init(uid:String,dictionary:[String:Any]) {
 self.uid = uid
 self.notifications = dictionary["notificationsStatus"] as? Bool ?? false
 }
 static func == (lhs: Client, rhs: Client) -> Bool {
 return lhs.uid == rhs.uid
 }
}

Use:

arrayClients.removingDuplicates(byKey: { 0ドル.uid })

Have a good day swift lovers ♥️

Answer 6

There is a good example from this post

Here is an Array extension to return the unique list of objects based on a given key:

extension Array {
 func unique<T:Hashable>(map: ((Element) -> (T))) -> [Element] {
 var set = Set<T>() //the unique list kept in a Set for fast retrieval
 var arrayOrdered = [Element]() //keeping the unique list of elements but ordered
 for value in self {
 if !set.contains(map(value)) {
 set.insert(map(value))
 arrayOrdered.append(value)
 }
 }
 return arrayOrdered
 }
}

for your example do:

let uniquePosts = posts.unique{0ドル.id ?? ""}

Answer 7

my 'pure' Swift solutions without Post conformance to Hashable (required by Set )

struct Post {
 var id: Int
}
let posts = [Post(id: 1),Post(id: 2),Post(id: 1),Post(id: 3),Post(id: 4),Post(id: 2)]
// (1)
var res:[Post] = []
posts.forEach { (p) -> () in
 if !res.contains ({ 0ドル.id == p.id }) {
 res.append(p)
 }
}
print(res) // [Post(id: 1), Post(id: 2), Post(id: 3), Post(id: 4)]
// (2)
let res2 = posts.reduce([]) { (var r, p) -> [Post] in
 if !r.contains ({ 0ドル.id == p.id }) {
 r.append(p)
 }
 return r
}
print(res2) // [Post(id: 1), Post(id: 2), Post(id: 3), Post(id: 4)]

I prefer (1) encapsulated into function (aka func unique(posts:[Post])->[Post] ), maybe an extension Array ....

Answer 8

Preserving order, without adding extra state:

func removeDuplicates<T: Equatable>(accumulator: [T], element: T) -> [T] {
 return accumulator.contains(element) ?
 accumulator :
 accumulator + [element]
}
posts.reduce([], removeDuplicates)

Answer 9

(Updated for Swift 3)

As I mentioned in my comment to the question, you can make use of a modified Daniel Kroms solution in the thread we previously marked this post to be duplicate of. Just make your Post object hashable (implicitly equatable via id property) and implement a modified (using Set rather than Dictionary; the dict value in the linked method is not used anyway) version of Daniel Kroms uniq function as follows:

func uniq<S: Sequence, E: Hashable>(_ source: S) -> [E] where E == S.Iterator.Element {
 var seen = Set<E>()
 return source.filter { seen.update(with: 0ドル) == nil }
}
struct Post : Hashable {
 var id : Int
 var hashValue : Int { return self.id }
}
func == (lhs: Post, rhs: Post) -> Bool {
 return lhs.id == rhs.id
}
var posts : [Post] = [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 1), Post(id: 3), Post(id: 5), Post(id: 7), Post(id: 9)]
print(Posts)
/* [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 1), Post(id: 3), Post(id: 5), Post(id: 7), Post(id: 9)] */
var myUniquePosts = uniq(posts)
print(myUniquePosts)
/* [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 3), Post(id: 5), Post(id: 9)] */

This will remove duplicates while maintaining the order of the original array.

---

Helper function uniq as a Sequence extension

Alternatively to using a free function, we could implement uniq as a constrained Sequence extension:

extension Sequence where Iterator.Element: Hashable {
 func uniq() -> [Iterator.Element] {
 var seen = Set<Iterator.Element>()
 return filter { seen.update(with: 0ドル) == nil }
 }
}
struct Post : Hashable {
 var id : Int
 var hashValue : Int { return self.id }
}
func == (lhs: Post, rhs: Post) -> Bool {
 return lhs.id == rhs.id
}
var posts : [Post] = [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 1), Post(id: 3), Post(id: 5), Post(id: 7), Post(id: 9)]
print(posts)
/* [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 1), Post(id: 3), Post(id: 5), Post(id: 7), Post(id: 9)] */
var myUniquePosts = posts.uniq()
print(myUniquePosts)
/* [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 3), Post(id: 5), Post(id: 9)] */

Answer 10

This works for multidimensional arrays as well:

for (index, element) in arr.enumerated().reversed() {
 if arr.filter({ 0ドル == element}).count > 1 {
 arr.remove(at: index)
 }
}

Answer 11

In swift 3 refer below code:

let filterSet = NSSet(array: orignalArray as NSArray as! [NSObject])
let filterArray = filterSet.allObjects as NSArray //NSArray
 print("Filter Array:\(filterArray)")

Answer 12

use a Set

To use it, make your Post hashable and implement the == operator

import Foundation
class Post: Hashable, Equatable {
 let id:UInt
 let title:String
 let date:NSDate
 var hashValue: Int { get{
 return Int(self.id)
 }
 }
 init(id:UInt, title:String, date:NSDate){
 self.id = id
 self.title = title
 self.date = date
 }
}
func ==(lhs: Post, rhs: Post) -> Bool {
 return lhs.id == rhs.id
}
let posts = [Post(id: 11, title: "sadf", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 1; c.month = 1; c.year = 2016; return c}())!),
 Post(id: 33, title: "sdfr", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 3; c.month = 1; c.year = 2016; return c}())!),
 Post(id: 22, title: "sdfr", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 1; c.month = 12; c.year = 2015; return c}())!),
 Post(id: 22, title: "sdfr", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 1; c.month = 12; c.year = 2015; return c}())!)]

Create set from array with duplicates

let postsSet = Set(posts)

This is unordered, create a new array, apply order.

let uniquePosts = Array(postsSet).sort { (p1, p2) -> Bool in
 return p1.date.timeIntervalSince1970 < p2.date.timeIntervalSince1970
}

---

Instead of making your Post model hashable, you could also use a wrapper class. This wrapper class would use the post objects property to calculate the hash and equality.
this wrapper could be configurable through closure:

class HashableWrapper<T>: Hashable {
 let object: T
 let equal: (obj1: T,obj2: T) -> Bool
 let hash: (obj: T) -> Int
 var hashValue:Int {
 get {
 return self.hash(obj: self.object)
 }
 }
 init(obj: T, equal:(obj1: T, obj2: T) -> Bool, hash: (obj: T) -> Int) {
 self.object = obj
 self.equal = equal
 self.hash = hash
 }
}
func ==<T>(lhs:HashableWrapper<T>, rhs:HashableWrapper<T>) -> Bool
{
 return lhs.equal(obj1: lhs.object,obj2: rhs.object)
}

The Post could be simply

class Post {
 let id:UInt
 let title:String
 let date:NSDate
 init(id:UInt, title:String, date:NSDate){
 self.id = id
 self.title = title
 self.date = date
 }
}

Let's create some post as before

let posts = [
 Post(id: 3, title: "sadf", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 1; c.month = 1; c.year = 2016; return c}())!),
 Post(id: 1, title: "sdfr", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 3; c.month = 1; c.year = 2016; return c}())!),
 Post(id: 2, title: "sdfr", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 1; c.month = 12; c.year = 2015; return c}())!),
 Post(id: 2, title: "sdfr", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 1; c.month = 12; c.year = 2015; return c}())!),
 Post(id: 1, title: "sdfr", date: NSCalendar.currentCalendar().dateFromComponents({let c = NSDateComponents(); c.day = 3; c.month = 1; c.year = 2016; return c}())!)
]

Now we create wrapper objects for every post with closure to determine equality and the hash. And we create the set.

let wrappers = posts.map { (p) -> HashableWrapper<Post> in
 return HashableWrapper<Post>(obj: p, equal: { (obj1, obj2) -> Bool in
 return obj1.id == obj2.id
 }, hash: { (obj) -> Int in
 return Int(obj.id)
 })
}
let s = Set(wrappers)

Now we extract the wrapped objects and sort it by date.

let objects = s.map { (w) -> Post in
 return w.object
}.sort { (p1, p2) -> Bool in
 return p1.date.timeIntervalSince1970 > p2.date.timeIntervalSince1970
}

and

print(objects.map{0ドル.id})

prints

[1, 3, 2]

• Oscar Apeland: Thank you for the massive effort. I chose to accept the other answer because your solution used Date to sort, and my post objects didn't have a date property initially. (Stupid API..). I wish I could accept yours too.
• vikingosegundo: That is the point in my last example: by using a configurable equality you could also check the index in the array. Also the wrapper object could have a property to store the index in the array.

Answer 13

This is a less-specialized question than the more popular variant found here: https://stackoverflow.com/a/33553374/652038

Use that answer of mine, and you can do this:

posts.firstUniqueElements(\.id)

Answer 14

Instead of using a hashable object, you could just use a set. Take an attribute value that you want to remove duplicates for and use that as your test value. In my example, I am checking for duplicate ISBN values.

do {
 try fetchRequestController.performFetch()
 print(fetchRequestController.fetchedObjects?.count)
 var set = Set<String>()
 for entry in fetchRequestController.fetchedObjects! {
 if set.contains(entry.isbn!){
 fetchRequestController.managedObjectContext.delete(entry)
 }else {
 set.insert(entry.isbn!)
 }
 }
 try fetchRequestController.performFetch()
 print(fetchRequestController.fetchedObjects?.count) 
 } catch {
 fatalError()
}

Answer 15

Swift 3.1 Most Elegant Solution (Thanx dfri)

Apple Swift version 3.1 (swiftlang-802.0.51 clang-802.0.41)

func uniq<S: Sequence, E: Hashable>(source: S) -> [E] where E==S.Iterator.Element {
 var seen: [E:Bool] = [:]
 return source.filter({ (v) -> Bool in
 return seen.updateValue(true, forKey: v) == nil
 })
}
struct Post : Hashable {
 var id : Int
 var hashValue : Int { return self.id }
}
func == (lhs: Post, rhs: Post) -> Bool {
 return lhs.id == rhs.id
}
var Posts : [Post] = [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 1), Post(id: 3), Post(id: 5), Post(id: 7), Post(id: 9)]
print(Posts)
/* [Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 1), Post(id: 3), Post(id: 5), Post(id: 7), Post(id: 9)] */
var myUniquePosts = uniq(source: Posts)
print(myUniquePosts)
/*[Post(id: 1), Post(id: 7), Post(id: 2), Post(id: 3), Post(id: 5), Post(id: 9)]*/

• dfrib: Consider adding a comment to the original answer next time (prompting for a Swift X.X update) prior to copying and re-posting the answer with only a minor update. In case the original answer's author doesn't answer the prompt for an update (and you do not have sufficient rights to update the answer yourself), at that time it could be appropriate to duplicate the answer with an update for a newer language version. As this Q&A currently stands, we now have two exactly duplicated answers, as I've now updated my original answer to Swift 3.

Answer 16

struct Post {
 var id: Int
}
extension Post: Hashable {
 var hashValue: Int {
 return id
 }
 static func == (lhs: Post, rhs: Post) -> Bool {
 return lhs.id == rhs.id
 }
}

and additional extension

public extension Sequence {
 func distinct<E: Hashable>() -> [E] where E == Iterator.Element {
 return Array(Set(self))
 }
}