Convert array of arrays to array without duplicates

Question

I have the following array, and I want to convert it to an array without duplicates.

const a = [
 {
 items: [
 { outerKey: { innerKey: 1 } },
 { outerKey: { innerKey: 2 } },
 { outerKey: { innerKey: 3 } },
 ],
 },
 {
 items: [
 { outerKey: { innerKey: 3 } },
 { outerKey: { innerKey: 5 } },
 { outerKey: { innerKey: 5 } },
 ],
 },
 {
 items: [
 { outerKey: { innerKey: 1 } },
 { outerKey: { innerKey: 8 } },
 { outerKey: { innerKey: 2 } },
 ],
 },
];
const finalArray = [...new Set(a.reduce((acc, curr) => { acc.push(curr.items.map(item => item.outerKey.innerKey)); return acc;} , []).flat())];
console.log(finalArray);

Is there a better approach than this? considering there could be > 1000 items.

Answer 1

Adding all the elements to a Set should be generally the most efficient way. However, before you add them to the Set you are creating lots of intermediate arrays. It will likely be more efficient to just iterate over the elements and add them to the set one by one:

let set = new Set();
a.forEach(o => {
 o.items.forEach(i => {
 set.add(i.outerKey.innerKey);
 });
});
const finalArray = [...set];
console.log(finalArray);

Not sure if forEach() is the fastest way to iterate over the array elements in whatever runtime you are using. See here for a discussion of how to iterate arrays.

• Falco: I think this is the fastest and most readable solution. If I show this code without comments to someone and ask what it does, he can clearly see it at firs glance.

Answer 2

From a code review standpoint, to start with, a better approach than this line:

const finalArray = [...new Set(a.reduce((acc, curr) => { acc.push(curr.items.map(item => item.outerKey.innerKey)); return acc;} , []).flat())];

would be to separate it out onto separate lines to make it more easily readable:

const finalArray = [...new Set(
 a.reduce(
 (acc, curr) => {
 acc.push(curr.items.map(item => item.outerKey.innerKey));
 return acc;
 },
 [])
 .flat()
)];

or something of the sort - better to have long, easy-to-read code than to conserve lines at the expense of readability - leave that to the automatic minifiers at the end of a build process. More than 4 or 5 operators and function calls on a single line is usually a red flag.

The algorithm you're using is slightly inefficient in that it's iterating over every subarray twice (once to map the items to each .innerKey, and again to flatten each subarray), but that's not so bad. A similar approach but using a slightly more appropriate method would be to use .flatMap instead of using .reduce and .flat at the end:

const a=[{items:[{outerKey:{innerKey:1}},{outerKey:{innerKey:2}},{outerKey:{innerKey:3}}]},{items:[{outerKey:{innerKey:3}},{outerKey:{innerKey:5}},{outerKey:{innerKey:5}}]},{items:[{outerKey:{innerKey:1}},{outerKey:{innerKey:8}},{outerKey:{innerKey:2}}]}];
const finalArray = [...new Set(
 a.flatMap(
 ({ items }) => items.map(
 item => item.outerKey.innerKey
 )
 )
)];
console.log(finalArray);

(Using .reduce when the accumulator stays the same object is arguably inappropriate anyway - see this video by V8 developers on the subject)

Using a Set to begin with in the other answer instead of making intermediate arrays is even better.

But:

considering there could be > 1000 items.

On modern computers, 1000 items in a data structure is nothing. (If they were elements in the DOM, that'd be another matter, but if they're just array items, it's almost certainly not something to worry about.) Better to write clean, readable code, and then if you find something doesn't seem to be running as fast as you'd like, run a performance test to identify the bottleneck(s) and fix them. (The bottleneck will almost certainly not be a section like this.)