Remove duplicates in 2d array

Question

I want to remove duplicate row in a 2d array . i tried the below code .but it is not working . please help me .

Input :

1,ram,mech
1,ram,mech
2,gopi,csc
2.gopi,civil

output should be :

1,ram,mech
2,gopi,csc
2.gopi,civil

Code :

package employee_dup;
import java.util.*;
public class Employee_dup {
 public static void main(String[] args)
 {
 boolean Switch = true;
 System.out.println("Name ID Dept ");
 String[][] employee_t = {{"1","ram","Mech"},{"1","siva","Mech"},{"1","gopi","Mech"},{"4","jenkat","Mech"},{"5","linda","Mech"},{"1","velu","Mech"}};
 int g = employee_t[0].length;
 String[][] array2 = new String[10][g];
 int rows = employee_t.length;
 Arrays.sort(employee_t, new sort(0));
 for(int i=0;i<employee_t.length;i++){ 
 for(int j=0;j<employee_t[0].length;j++){ 
 System.out.print(employee_t[i][j]+" "); 
 } 
 System.out.println(); 
 } 
 List<String[]> l = new ArrayList<String[]>(Arrays.asList(employee_t));
 for(int k = 0 ;k < employee_t.length-1;k++)
 {
 if(employee_t[k][0] == employee_t[k+1][0])
 {
 System.out.println("same value is present"); 
 l.remove(1);
 array2 = l.toArray(new String[][]{});
 } 
 }
 System.out.println("Name ID Dept ");
 for(int i=0;i<array2.length;i++){ 
 for(int j=0;j<array2[0].length;j++){ 
 System.out.print(array2[i][j]+" "); 
 } 
 System.out.println(); 
 }
 }
}
class sort implements Comparator {
 int j;
 sort(int columnToSort) {
 this.j = columnToSort;
 }
 //overriding compare method
 public int compare(Object o1, Object o2) {
 String[] row1 = (String[]) o1;
 String[] row2 = (String[]) o2;
 //compare the columns to sort
 return row1[j].compareTo(row2[j]);
 }
}

First I sorted the array based on column one ,then tried to remove duplicates by checking the first column elements and seconds column elements but it is not removing the required column but remove other columns.

• sovas: Arrays.sort(employee_t, new sort(0)); What do you mean by 'new sort(0)' ?
• kkflf: Do you want help to fix your code, or can I post an answer of how you could do this a lot more clean?
• SomeDude: Hint : You don't need to sort to remove duplicates.
• Anonymous: Your code was quite unreadable due to wrong indentation. This time I fixed it for you; next time please have your IDE do it, it’s much easier.
• sovas: @OleV.V. Thanks, when i posted a comment, the code of sort method wasn't included.

Answer 1

You may give this solution a try:

public static void main(String[] args) {
 String[][] employee_t = {
 {"1","ram","Mech"},
 {"1","ram","Mech"},
 {"1","siva","Mech"},
 {"1","siva","Mech"},
 {"1","gopi","Mech"},
 {"1","gopi","Mech"} };
 System.out.println("ID Name Dept");
 Arrays.stream(employee_t)
 .map(Arrays::asList)
 .distinct()
 .forEach(row -> System.out.printf("%-3s%-7s%s\n", row.get(0), row.get(1), row.get(2)));
}

Output

ID Name Dept
1 ram Mech
1 siva Mech
1 gopi Mech

How it works: comparing arrays does rely on instance equality and not on comparing contained elements by equals. Hence converting each row of your 2D array into a List will enable you to compare lists, which takes equals of the elements contained into account.

The Java Stream API does provide a method distinct which relies on equals and will remove all duplicates for you.

Answer 2

Based on your code. Maybe it is not the BEST solution but it works.

public static void main(String[] args) {
 System.out.println("Name ID Dept ");
 // I added duplicated rows
 String[][] inputArray = {
 { "1", "ram", "Mech" }, 
 { "1", "siva", "Mech" }, 
 { "1", "gopi", "Mech" }, 
 { "1", "gopi", "Mech" }, 
 { "4", "jenkat", "Mech" },
 { "5", "linda", "Mech" }, 
 { "1", "velu", "Mech" },
 { "1", "velu", "Mech" }
 };
 // I will add all rows in a Set as it doesn't store duplicate values
 Set<String> solutionSet = new LinkedHashSet<String>();
 // I get all rows, create a string and insert into Set
 for (int i = 0 ; i < inputArray.length ; i++) {
 String input = inputArray[i][0]+","+inputArray[i][1]+","+inputArray[i][2];
 solutionSet.add(input);
 }
 // You know the final size of the output array
 String[][] outputArray = new String[solutionSet.size()][3];
 // I get the results without duplicated values and reconvert it to your format
 int position = 0;
 for(String solution : solutionSet) {
 String[] solutionArray = solution.split(",");
 outputArray[position][0] = solutionArray[0];
 outputArray[position][1] = solutionArray[1];
 outputArray[position][2] = solutionArray[2];
 position++;
 }
 System.out.println("Name ID Dept ");
 for (int i = 0; i < outputArray.length; i++) {
 for (int j = 0; j < outputArray[0].length; j++) {
 System.out.print(outputArray[i][j] + " ");
 }
 System.out.println();
 }
}

Answer 3

I have posted what I think is a readable and easy to maintain solution.

I decided to use distinct from Stream which is part of Java 8

Returns a stream consisting of the distinct elements (according to Object.equals(Object)) of this stream. - https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#distinct--

Main.class

class Main {
 public static void main(String[] args)
 {
 //Create a list of Employee objects
 List<Employee> employeeList = new ArrayList<Employee>();
 Employee e1 = new Employee(1, "ram", "mech");
 Employee e2 = new Employee(1, "ram", "mech");
 Employee e3 = new Employee(2, "gopi", "csc");
 Employee e4 = new Employee(2, "gopi", "civil");
 employeeList.add(e1);
 employeeList.add(e2);
 employeeList.add(e3);
 employeeList.add(e4);
 System.out.println("Before removing duplicates");
 employeeList.stream().forEach(System.out::println);
 //This is where all the magic happens.
 employeeList = employeeList.stream().distinct().collect(Collectors.toList());
 System.out.println("\nAfter removing duplicates");
 employeeList.stream().forEach(System.out::println);
 }
}

Output:

Before removing duplicates
Employee [valA=1, valB=ram, valC=mech]
Employee [valA=1, valB=ram, valC=mech]
Employee [valA=2, valB=gopi, valC=csc]
Employee [valA=2, valB=gopi, valC=civil]
After removing duplicates
Employee [valA=1, valB=ram, valC=mech]
Employee [valA=2, valB=gopi, valC=csc]
Employee [valA=2, valB=gopi, valC=civil]

Employee.class

//This is just a regular POJO class.
class Employee {
 int valA;
 String valB, valC;
 public Employee(int valA, String valB, String valC){
 this.valA = valA;
 this.valB = valB;
 this.valC = valC;
 }
 public Employee(Employee e) {
 this.valA = e.valA;
 this.valB = e.valB;
 this.valC = e.valC;
 }
 @Override
 public int hashCode() {
 final int prime = 31;
 int result = 1;
 result = prime * result + valA;
 result = prime * result + ((valB == null) ? 0 : valB.hashCode());
 result = prime * result + ((valC == null) ? 0 : valC.hashCode());
 return result;
 }
 @Override
 public boolean equals(Object obj) {
 if(obj instanceof Employee && ((Employee)obj).hashCode() == this.hashCode()){
 return true;
 }
 return false;
 }
 @Override
 public String toString() {
 return "Employee [valA=" + valA + ", valB=" + valB + ", valC=" + valC + "]";
 }
}

• Anonymous: A good idea to have an Employee class to model employees rather than the subarrays. The asker, knowing the domain, should be able to find better variable names than valA, valB and valC.
• kkflf: Yes, I agree that it is terrible variable names. OP did not mention what his values actually represented. :)
• skubski: Any idea what the performance characteristics are from the distinct operation on that stream?
• Anonymous: @skubski, since distinct() cannot rely on hash code or such, I would expect quadratic time, O(n^2). It’s not something I know.

Answer 4

Pre Java - 8 solution. May not be the best way. But a quick solution which works..

String[][] records = {
 {"1","ram","Mech"},
 {"1","ram","Mech"},
 {"1","gopi","csc"},
 {"1","gopi","civil"} };
 List<String[]> distinctRecordsList = new ArrayList<String[]>();
 for(String[] record : records){
 if(distinctRecordsList.size()>0){
 boolean sameValue = false;
 for(String[] distinctRecord : distinctRecordsList){
 int distinctRecordFields = distinctRecord.length;
 if(record.length==distinctRecordFields){
 for(int k=0;k<distinctRecordFields;k++){
 sameValue = record[k].equalsIgnoreCase(distinctRecord[k]);
 if(!sameValue)
 break;
 }
 }else
 throw new Exception("Can't compare the records");
 }
 if(!sameValue)
 distinctRecordsList.add(record);
 }else if(distinctRecordsList.size()==0)
 distinctRecordsList.add(record); 
 }
 Object[] distRecObjects = distinctRecordsList.toArray();
 String[][] distinctRecordsArray = new String[distRecObjects.length][];
 int i=0;
 for(Object distRecObject : distRecObjects){
 distinctRecordsArray[i] = (String[]) distRecObject;
 i++;
 }

Answer 5

Contrary to some other answers I will try to explain what went wrong in your own code and how to fix it within your code (I agree very much with kkflf that an Employee class would be a huge benefit: it’s more object-oriented and it will help structure the code and give better overview of it).

The issues I see in your code are:

• You are not removing the correct element when you detect a duplicate, but always the element at index 1 (the second element since indices count from 0). This isn’t trivial, though, because indices shift as you remove elements. The trick is to iterate backward so only indices that you are finished with shift when you remove an element.
• You are using == to compare the first element of the subarrays you are comparing. If you wanted to compare just the first element, you should use equals() for comparison. However, I believe you want to compare the entire row so 2,gopi,csc and 2.gopi,civil are recognized as different and both preserved. Arrays.equals() can do the job.
• You need to create array2 only after the loop. As your code stands, if no duplicates are detected, arrays2 is never created.

So your loop becomes:

 for (int k = employee_t.length - 1; k >= 1; k--)
 {
 if (Arrays.equals(employee_t[k], employee_t[k - 1]))
 {
 System.out.println("same value is present"); 
 l.remove(k);
 } 
 }
 array2 = l.toArray(new String[][]{});

This gives you the output you asked for.

Further tips:

• Your comparator only compares one field in the inner arrays, which is not enough to guarantee that identical rows come right after each other in the sorted array. You should compare all elements, and also require that the inner arrays have the same length.
• Use generics: class Sort extends Comparator<String[]>, and you won’t need the casts in compare()
• According to Java naming conventions it should be class EmployeeDup, boolean doSwitch (since switch is a reserved word) and class Sort.
• You are not using the variables Switch and rows; delete them.

Answer 6

I have wrote a solution for me. This may not be the best but it works.

public static String[][] removeDuplicate(String[][] matrix) {
 String[][] newMatrix = new String[matrix.length][matrix[0].length];
 int newMatrixRow = 1;
 for (int i = 0; i < matrix[0].length; i++)
 newMatrix[0][i] = matrix[0][i];
 for (int j = 1; j < matrix.length; j++) {
 List<Boolean> list = new ArrayList<>();
 for (int i = 0; newMatrix[i][0] != null; i++) {
 boolean same = true;
 for (int col = 2; col < matrix[j].length; col++) {
 if (!newMatrix[i][col].equals(matrix[j][col])) {
 same = false;
 break;
 }
 }
 list.add(same);
 }
 if (!list.contains(true)) {
 for (int i = 0; i < matrix[j].length; i++) {
 newMatrix[newMatrixRow][i] = matrix[j][i];
 }
 newMatrixRow++;
 }
 }
 
 int i;
 for(i = 0; newMatrix[i][0] != null; i++);
 
 String finalMatrix[][] = new String[i][newMatrix[0].length];
 for (i = 0; i < finalMatrix.length; i++) {
 for (int j = 0; j < finalMatrix[i].length; j++)
 finalMatrix[i][j] = newMatrix[i][j];
 }
 
 return finalMatrix;
}

This method will return a matrix without any duplicate rows.