Java Remove Duplicates from an Array?

Question

I am supposed to read in a file containing many different email addresses and print them out using an array. The problem is I need to eliminate duplicate emails.

I was able to get my try / catch working and print out the email addresses. However, I am not sure how to go about removing the duplicates. I do not have an understanding of hashcode's or how to use a Set yet. Any assistance would be appreciated.

Here is what I have so far:

import java.util.Scanner;
import java.io.*;
public class Duplicate {
 public static void main(String[] args) {
 Scanner keyboard = new Scanner(System.in);
 System.out.println("Enter file name: ");
 String fileName = keyboard.nextLine();
 if (fileName.equals("")) {
 System.out.println("Error: User did not specify a file name.");
 } else {
 Scanner inputStream = null;
 try {
 inputStream = new Scanner(new File(fileName));
 } catch (FileNotFoundException e) {
 System.out.println("Error: " + fileName + " does not exist.");
 System.exit(0);
 }
 String[] address = new String[100];
 int i = 0;
 while (inputStream.hasNextLine()) {
 String email = inputStream.nextLine();
 // System.out.println(email);
 address[i] = email;
 System.out.println(address[i]);
 i++;
 }
 }
 }
}

• jli: Are you looking for the most efficient solution possible? If not, just create a new array and iterate through the old one, adding as you go after checking if the current entry is already in the new array.
• user1241335: @jli just said so before you :P. It might not be the most effective solution, but hell.
• mmmmmm: possible duplicate of How do I remove repeated elements from ArrayList?
• Leigh: @Bean Winz - Welcome to stackoveflow. In future, if your question is a homework assignment, please be sure to add the homework tag.

Answer 1

The Simple solution is that use Set of java,

so set remove duplicate value automatically

and in your code you have array than convert array to set directly using code

Set<T> mySet = new HashSet<T>(Arrays.asList(someArray));

• Bean Winz: thanks. I guess I can try it-i've just never even heard of sets before as I am just beginning programming. So what is the <T> and what does the .asList do?
• Yogesh Prajapati: hi Bean, if you use Set mySet = new HashSet(Arrays.asList(someArray)); than its ok but if you want to create a set of specific datatype so you can give datatype instead of T ex Set<String> set = new Hashset<String>();.
• sparc_spread: The T is a placeholder for whichever datatype you want to use (String in your case). This notation is a bit advanced for someone just starting out so don't worry about it for right now. Learn basics like this collections tutorial first and the T stuff will seem a bit more natural when you encounter it.
• Bean Winz: thanks i just figured that part out. But I am confused on where to put this code. would I put it in my while loop?
• Alex D: No. First build up the array in your while loop, then after the array is finished, convert it into a set. Then go over the set, and print all the addresses out. You can use something like for(String s : mySet) System.out.println(s);

Answer 2

Learn Set. The time it will take you to learn it is less than the time it will take you to code something that doesn't use it.

I'll get you started. Replace this:

String[] address = new String[100];

with this:

Set<String> addresses = new HashSet<String>();

And this:

address[i] = email;

with this:

addresses.add(email);

You don't need the i anymore.

You're done. If you'd like to print everything out:

for (String address : addresses) {
 System.out.println (address);
}

That pretty much covers it. Want everything to be automatically sorted? Replace the HashSet above with TreeSet. Now go read this excellent tutorial so that next time, you can get it all done faster and on your own.

• sparc_spread: Can you tell me a bit more about the constraints of the assignment? I'm surprised that they only let you use arrays, considering that the list of addresses appears to be of arbitrary length (vs. the fixed length of an array).

Answer 3

You can try going through each element in the array, adding it to another one, checking if the 2nd array contains the next item, if it does skip it. Then just replace the 1st array with the 2nd. (ArrayList is better in this case though).

so something like this:

List<String> FinalList = new ArrayList<String>();
for(string temp : adress)
{
if(!FinalList.contains(temp))
 FinalList.add(temp);
}

Answer 4

Read them into a HashSet instead. This will handle duplicates for you.

Set<String> addresses = new HashSet<String>();
addresses.add("[email protected]");
addresses.add("[email protected]");
addresses.add("[email protected]");
System.out.println(addresses.size());

Will print 1.

Answer 5

Use the ArrayUtil class as you need. I have written some methods other than removing duplicates. This class is implemented without using any Collection framework classes.

public class ArrayUtils {
/**
 * Removes all duplicate elements from an array. 
 * @param arr Array from which duplicate elements are to be removed.
 * @param removeAllDuplicates true if remove all duplicate values, false otherwise 
 * @return Array of unique elements.
 */
public static int[] removeDuplicate(int[] arr, boolean removeAllDuplicates) {
 int size = arr.length;
 for (int i = 0; i < size;) {
 boolean flag = false;
 for (int j = i + 1; j < size;) {
 if (arr[i] == arr[j]) {
 flag = true;
 shrinkArray(arr, j, size);
 size--;
 } else
 j++;
 }
 if (flag && removeAllDuplicates) {
 shrinkArray(arr, i, size);
 size--;
 } else
 i++;
 }
 int unique[] = new int[size];
 for (int i = 0; i < size; i++)
 unique[i] = arr[i];
 return unique;
}
/**
 * Removes duplicate elements from an array. 
 * @param arr Array from which duplicate elements are to be removed.
 * @return Array of unique elements.
 */
public static int[] removeDuplicate(int[] arr) {
 return removeDuplicate(arr, false);
}
private static void shrinkArray(int[] arr, int pos, int size) {
 for (int i = pos; i < size - 1; i++) {
 arr[i] = arr[i + 1];
 }
}
/**
 * Displays the array.
 * @param arr The array to be displayed.
 */
public static void displayArray(int arr[]) {
 System.out.println("\n\nThe Array Is:-\n");
 for (int i = 0; i < arr.length; i++) {
 System.out.print(arr[i] + "\t");
 }
}
/**
 * Initializes the array with a given value.
 * @param arr The array to be initialized.
 * @param withValue The value with which the array is to be initialized.
 */
public static void initializeArray(int[] arr, int withValue) {
 for (int i = 0; i < arr.length; i++) {
 arr[i] = withValue;
 }
}
/**
 * Checks whether an element is there in the array. 
 * @param arr The array in which the element is to be found.
 * @param element The element that is to be found.
 * @return True if found false otherwise
 */
public static boolean contains(int arr[], int element) {
 for(int i=0; i< arr.length; i++) {
 if(arr[i] == element)
 return true;
 }
 return false;
}
/**
 * Removes a element from an array.
 * @param arr The array from which the element is to removed.
 * @param element The element to be removed
 * @return The size of the array after removing.
 */
public static int removeElement(int[] arr, int element) {
 int size = arr.length;
 for(int i=0; i< arr.length; i++){
 if(arr[i] == element){
 shrinkArray(arr, i, arr.length);
 size--;
 }
 }
 return size;
}
/**
 * Counts unique elements in an array.
 * @param arr The required array.
 * @return Unique element count.
 */
public static int uniqueElementCount(int arr[]) {
 int count = 0;
 int uniqueCount=0;
 int[] consideredElements = new int[arr.length];
 initializeArray(consideredElements, 0);
 for(int i=0;i<arr.length;i++) {
 int element = arr[i];
 for(int j=i+1;j<arr.length; j++){
 if(element != arr[j] && !contains(consideredElements, element)){
 consideredElements[count++] = element;
 }
 }
 }
 for(int i=0;i< consideredElements.length;i++)
 if(consideredElements[i]!=0)
 uniqueCount++;
 return uniqueCount;
}
}

Answer 6

Please use below code for remove duplicates in an integer array.

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
package test123;
import java.util.ArrayList;
import java.util.HashSet;
/**
 *
 * @author krawler
 */
public class Test123 {
 /**
 * @param args the command line arguments
 */
 public static ArrayList<Integer> removeDuplicates(ArrayList<Integer> list) {
	// Store unique items in result.
	ArrayList<Integer> result = new ArrayList<>();
	HashSet<Integer> set = new HashSet<>();
	
	for (Integer item : list) {
	 
	 if (!set.contains(item)) {
		result.add(item);
		set.add(item);
	 }
	}
	return result;
 }
 public static void main(String[] args) {
	ArrayList<Integer> list = new ArrayList<>();
	list.add(12);
	list.add(12);
	list.add(8);
	list.add(6);
	list.add(4);
	list.add(4);
 list.add(2);
 list.add(1); 
 //int a[]={12,12,8,6,4,4,2,1}
	
	ArrayList<Integer> unique = removeDuplicates(list);
	for (int element : unique) {
	 System.out.println(element);
	}
 }
}
/*run:
12
8
6
4
2
1
BUILD SUCCESSFUL (total time: 0 seconds)*/

Answer 7

If you want to remove duplicates you can try something like this:

String[] address = new String[100]; // the array that contains all addresses
ArrayList<String> uniqueAddresses = new ArrayList<String>(); // create arraylist to contain all non-repeated addresses
for(String addr : address){ // cycle through the entire array
 if(!uniqueAddresses.contain(addr)){ // check if the address already there
 uniqueAddresses.add(addr); // add it
 }
}

Answer 8

remove duplicates from an array

T[] array = {...};

getting a Set without the duplicates
since Java 10

Set<T> set = Set.copyOf(Arrays.asList(array));

the order of the array elements get lost

getting a new array w/o the duplicates

Arrays.stream(array).distinct().toArray(T[]::new);

the order of the array elements is retained

Answer 9

import java.util.Arrays;
import java.util.Scanner;
public class DuplicatesRemoved {
 public static void main(String[] args) {
 Scanner sc = new Scanner(System.in);
 System.out.println("enter size");
 int size = sc.nextInt();
 System.out.println("enter all elements");
 int arr[] = new int[size];
 for (int i = 0; i < arr.length; i++) {
 arr[i] = sc.nextInt();
 }
 duplicateControl(arr);
 if (duplicateControl(arr)) {
 System.out.println("Duplicate Detected");
 int[] removedVersion = removedDuplicate(arr);
 System.out.println("Duplicates Removed In Array: " + Arrays.toString(removedVersion));
 }
 if (!duplicateControl(arr)) {
 System.out.println("No Duplicate");
 }
 }
 public static boolean duplicateControl(int[] arr) {
 if (arr.length == 1) {
 System.out.println("There is only one element in Array,so there is no duplicate ");
 return false;
 }
 for (int i = 0; i < arr.length; i++) {
 for (int j = i + 1; j < arr.length; j++) {
 if (arr[i] == arr[j])
 return true;
 }
 }
 return false;
 }
 public static int[] DuplicatedElements(int arr[]) {
 int size = 0;
 int duplicated[] = new int[arr.length];
 boolean flag;
 for (int i = 0; i < arr.length; i++) {
 flag = false;
 for (int j = i + 1; j < arr.length; j++) {
 if (arr[i] == arr[j]) {
 flag = true;
 }
 }
 if (flag) {
 boolean alreadyAdded = false;
 for (int k = 0; k < size; k++) {
 if (duplicated[k] == arr[i]) {
 alreadyAdded = true;
 }
 }
 if (!alreadyAdded) {
 duplicated[size] = arr[i];
 size++;
 }
 }
 }
 int printDuplicate[] = new int[size];
 for (int i = 0; i < size; i++) {
 printDuplicate[i] = duplicated[i];
 }
 return printDuplicate;
 }
 public static int[] removedDuplicate(int[] arr) {
 int size = 0;
 int removeduplicate[] = new int[arr.length];
 boolean flag;
 for (int i = 0; i < arr.length; i++) {
 flag = false;
 for (int j = i + 1; j < arr.length; j++) {
 if (arr[i] == arr[j]) {
 flag = true;
 }
 }
 if (!flag) {
 boolean alreadyAdded = false;
 for (int k = 0; k < size; k++) {
 if (removeduplicate[k] == arr[i]) {
 alreadyAdded = true;
 }
 }
 if (!alreadyAdded) {
 removeduplicate[size] = arr[i];
 size++;
 }
 }
 }
 int res[] = new int[size];
 for (int i = 0; i < size; i++) {
 res[i] = removeduplicate[i];
 }
 return res;
 }
}

• Community: Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
• Abra: Answers that contain only code are generally regarded as poor quality answers. Also your answer seems quite similar to this answer. Can you explain how your answer provides something that is not in any of the other answers?

Answer 10

the first thing that comes into my head is to sort the array and then to check if the next element equals the current element. if so, delete the current element.

oh and when you don ́t know how many emails are stored in the file, an array is probably not the best way. I ́d take some sort of list, so that i don ́t have to care how many e-mail addresses are in the file.

Answer 11

you can write a function that run on the array and take one email at a time and when ever it find the same address just set it to null. when you're running on the array to print it, make a condition to print the email only if its not null