Remove repeated words from a string

Question

Remove all repeating words from an inputted sentence.

Input will be something like cat dog cat dog bird dog Snake snake Snake and the output should be cat dog bird Snake snake. There will always be a single space separating words.

Output order must be the same as input. (Refer to the example)

You don't need to handle punctuation but capital letter handling is required.

• Alex A.: I recommend waiting to accept an answer for at least a few days. A shorter solution may still come.
• xnor: I expect similar solutions to uniqchars, except that this doesn't ban built-ins that remove duplicates.
• edc65: Seeing the example, there is not special capital letter handling: Snake and snake are treated simply as different
• user46167: @AlexA.: In fact, there already is one. codegolf.stackexchange.com/questions/62044/…

Answer 1

CJam, 7 chars

qS/_&S*

Can probably be much shorter... but whatever I've almost never used CJam. ^.^

q reads input, S/ splits on spaces, _& duplicates and applies a setwise AND (therefore getting rid of duplicates), and S* re-joins on space.

Online interpreter link

• Cruncher: How can you even get much shorter than 7? lol
• Alien G: Some one just did.

Answer 2

Haskell, 34 bytes

import Data.List
unwords.nub.words

Usage example: (unwords.nub.words) "cat dog cat dog bird dog Snake snake Snake" -> "cat dog bird Snake snake".

Answer 3

APL, (削除) 22 (削除ここまで) 20 bytes

{1↓∊∪(∊∘' '⊂⊢)' ',⍵}

This creates an unnamed monadic function that accepts a string on the right and returns a string.

Explanation:

 ' ',⍵} ⍝ Prepend a space to the input string
 (∊∘' '⊂⊢) ⍝ Split the string on spaces using a fork
 ∪ ⍝ Select the unique elements
{1↓∊ ⍝ Join into a string and drop the leading space

Try it online

Saved 2 bytes thanks to Dennis!

• Darth Egregious: I love any answer that uses a non-esoteric, non-golf language.

Answer 4

Ruby, 21 chars

->s{s.split.uniq*' '}

Answer 5

TeaScript, 12 bytes

TeaScript is JavaScript for golfing.

xs` `u()j` `

This is pretty short. It splits on each space, filters out duplicates, then rejoins.

Try it online

• user31556: Is it tee-a script or tee script?
• Downgoat: @MathiasFoster it would be "tee-script"
• intrepidcoder: Does TeaScript have letters reserved for variable names? Most of them appear to be shorthands for built-in properties.
• Downgoat: @intrepidcoder yes all of these: cdfghijklmnopstuvw are reserved for variables, they are all pre-initialized to 0. b is also reserved for a variable name, it is pre-initialized to an empty string

Answer 6

JavaScript (ES6) 33

(see this answer)

Test running the snippet below in an EcmaScript 6 compliant browser (implementing Set, spread operator, template strings and arrow functions - I use Firefox).

Note: the conversion to Set drop all the duplicates and Set mantains the original ordering.

f=s=>[...Set(s.split` `)].join` `
function test() { O.innerHTML=f(I.value) }
test()

#I { width: 70% }

<input id=I value="cat dog cat dog bird dog Snake snake Snake"/><button onclick="test()">-></button>
<pre id=O></pre>

• ETHproductions: Wow wow wow... I am continually amazed by your ability to cut any solution I think up by 25% or more. +1
• Mwr247: Looked at the problem and immediately thought of Sets... only to realize that you'd already done it =P very nice!
• njzk2: how can set maintain the original ordering?
• edc65: @njzk2 ask the developers of the language. It could be: a set is internally an Array, and at each insertion there is a check to reject duplicates. It's an implementation detail anyway
• edc65: @njzk2 while I don't know how, I know that this fact is specified by the language: Set objects are collections of values, you can iterate its elements in insertion order. A value in the Set may only occur once; it is unique in the Set's collection. (developer.mozilla.org/it/docs/Web/JavaScript/Reference/…)

Answer 7

R, 22 bytes

cat(unique(scan(,"")))

This reads a string from STDIN and splits it into a vector on spaces using scan(,""), selects only unique elements, then concatenates them into a string and prints it to STDOUT using cat.

Answer 8

PowerShell, 15 Bytes

$args|select -u

Whoa, an actual entry where PowerShell is somewhat competitive? That's unpossible!

Takes the string as input arguments, pipes to Select-Object with the -Unique flag. Spits out an array of strings, preserving order and capitalization as requested.

Usage:

PS C:\Tools\Scripts\golfing> .\remove-repeated-words-from-string.ps1 cat dog cat dog bird dog Snake snake Snake
cat
dog
bird
Snake
snake

---

If this is too "cheaty" in assuming the input can be as command-line arguments, then go for the following, at (削除) 24 (削除ここまで) 21 Bytes _{(saved some bytes thanks to blabb)}. Interestingly, using the unary operator in this direction happens to also work if the input string is demarcated with quotes or as individual arguments, since the default -split is by spaces. Bonus.

-split$args|select -u

• manatwork: Relying on the environment's behavior of spoon-feeding the code with readily split up input...?
• AdmBorkBork: @manatwork I've added a clarification if the first usage is considered too "cheaty" -- since it's not clear exactly how the input is specified, we'll leave it up to the OP.
• manatwork: And now is clear how efficients are PowerShell's own features. That 24 really deserves an upvote.
• blabb: @timmyD you can chop off 3 bytes to the uncheaty ?? version by using the unary split and no need for "" '' in the commandline args too :\>ls -l split.ps1 & type split.ps1 & echo.&powershell -nologo -f split.ps1 cat dog cat dog bird dog Snake snake Snake -rw-rw-rw- 1 Admin 0 21 2015年11月02日 19:06 split.ps1 -split$args|select -u cat dog bird Snake snake

Answer 9

Julia, 29 bytes

s->join(unique(split(s))," ")

This creates an unnamed function that splits the string into a vector on spaces, keeps only the unique elements (preserving order), and joins the array back into a string with spaces.

Answer 10

Retina, 22 bytes

 (\w+)\b(?<=\b1円\b.+)

Save the file with a trailing linefeed and run it with the -s flag.

This is fairly straight forward in that it matches a single word, and the lookbehind checks whether that same word has appeared in the string before. The trailing linefeed causes Retina to work in Replace mode with an empty replacement string, removing all matches.

Answer 11

Mathematica, (削除) 43 (削除ここまで) 39 bytes

StringRiffle@*Keys@*Counts@*StringSplit

• Michael Stern: Kudos for using StringRiffle[].
• sanchez: could use Keys@Counts instead of DeleteDuplicates
• LegionMammal978: @branislav Does Keys@Counts preserve order?
• sanchez: @LegionMammal978 Counts[list] gives an association whose keys are in the same order as they first occur as elements of list.

Answer 12

Pyth - 9 bytes

Well this is why we're all waiting for Pyth5, could have been 5 bytes.

jdoxzN{cz

Try it online here.

• lirtosiast: Why isn't Pyth5 valid? It appears to be implemented.
• Alex A.: @ThomasKwa I don't think it's finished. There hasn't been a versioned release yet.

Answer 13

C++11, 291 bytes

#include<iostream>
#include<string>
#include<list>
#include<sstream>
#include<algorithm>
using namespace std;main(){string s;getline(cin,s);list<string>m;stringstream b(s);while(getline(b,s,' '))if(find(m.begin(),m.end(),s)==m.end())m.push_back(s);for(auto a:m)cout<<a<<' ';cout<<endl;}

I don't see a whole lot of C++ answers compared to golfing languages, so why not. Note that this uses C++11 features, and so if your compiler is (削除) stuck in the dark ages (削除ここまで) sufficiently old enough, you may need to pass a special compilation switch to make it use the C++11 standard. For g++, it's -std=c++11 (only needed for versions < 5.2). Try it online

• CroCo: If you compare the number of bytes with other languages, you will see why no one is using C++.
• user45941: @CroCo If you realize the point of this site is to find the shortest solution in each language, you will see why I posted this answer.
• CroCo: sorry I'm not aware of it.
• edmz: Why not use a set? It allows no duplicates by design. Just push into it.
• user45941: @black A set is not guaranteed to have the items in the same order they were added.

Answer 14

K5, 9 bytes

" "/?" "\

FYI, this is a function.

Explanation

 " "\ Split the input on spaces
 ? Find all the unique elements
" "/ Join them back together

Answer 15

Matlab: 18 Bytes

unique(d,'stable')

where d is d = {'cat','dog','cat','dog','bird','dog','Snake','snake','Snake'}.

The result is 'cat' 'dog' 'bird' 'Snake' 'snake'

• Alex A.: Welcome to Programming Puzzles and Code Golf! Submissions here need to either be full programs that read from STDIN and write to STDOUT, or functions which accept input and return output. As it stands, this is merely a snippet; it assumes the variable d is already assigned. You can rectify this by using a function handle: @(d)unique(d,'stable'), at the cost of 4 bytes.

Answer 16

Python 3, 55

l=[]
for x in input().split():l+=[x][x in l:]
print(*l)

Yeesh, this is long. Unfortunately, Python's set doesn't keep the order of the elements, so we have to do the work ourselves. We iterate through the input words, keeping a list l of elements that aren't yet in l. Then, we print the contents of l space-separated.

A string version of l would not work if some words are substrings of other words.

Answer 17

C#, 38 bytes

String.Join(" ",s.Split().Distinct());

• Jacob: I'm not sure you can assume input is already populated in s, I think you should get it as an argument.
• Martin Ender: Welcome to PPCG! Please have a look at our default answer formats. Answers should either be full programs or functions. Unnamed functions (like lambda literals) are fine, but snippets which expect the code to already exist in some variable/on the stack etc. or require a REPL environment are generally disallowed unless the OP explicitly permits them.

Answer 18

Perl 6, 14 bytes

As a whole program the only way you would write it is 21 bytes long

say $*IN.words.unique # 21 bytes

---

As a lambda expression the shortest is 14 bytes

*.words.unique # 14 bytes

say ( *.words.unique ).('cat dog cat dog bird dog Snake snake Snake')
my &foo = *.words.unique;
say foo $*IN;

While the output is a List, if you put it in a stringifying context it will put a space between the elements. If it was a requirement to return a string you could just add a ~ to the front ~*.words.unique.

---

If snippets were allowed, you could shorten it to 13 bytes by removing the *.

$_ = 'cat dog cat dog bird dog Snake snake Snake';
say .words.unique

Answer 19

gs2, 3 bytes

,É-

Encoded in CP437.

STDIN is pushed at the start of the program. , splits it over spaces. É is uniq, which filters duplicates. - joins by spaces.

Answer 20

Python 3, (削除) 87 (削除ここまで) 80 bytes

turns out the full program version is shorter

s=input().split(' ')
print(' '.join(e for i,e in enumerate(s)if e not in s[:i]))

Did it without regex, I am happy

Try it online

Answer 21

Lua, 94 bytes

function c(a)l={}return a:gsub("%S+",function(b)if l[b]then return""else l[b]=true end end)end

• Jonathan Frech: An anonymous user suggested to replace ... return""else l[b]=true end end... with ...return""end l[b]=""end....

Answer 22

awk, 25

BEGIN{RS=ORS=" "}!c[0ドル]++

Output:

$ printf "cat dog cat dog bird dog Snake snake Snake" | awk 'BEGIN{RS=ORS=" "}!c[0ドル]++'
cat dog bird Snake snake $ 
$ 

Answer 23

JavaScript, (削除) 106 (削除ここまで) (削除) 102 (削除ここまで) 100 bytes

function(s){o={};s.split(' ').map(function(w){o[w]=1});a=[];for(w in o)a.push(w);return a.join(' ')}

// way too long for JS :(

• kirbyfan64sos: Try using JS (aka ECMAScript) 6 arrow functions, which should save 6 bytes. Also, I can already see porting this to CoffeeScript will save at least 30 bytes.
• Jacob: This answer is in native JavaScript (ECMA5), there is edc65's one for es6.

Answer 24

Hassium, 91 bytes

func main(){d=[]foreach(w in input().split(' '))if(!(d.contains(w))){d.add(w)print(w+" ")}}

Run online and see expanded here

Answer 25

PHP (削除) 64 (削除ここまで) 59 bytes

function r($i){echo join(" ",array_unique(split(" ",$i)));}

• manatwork: explode() → split(), implode() → join()?
• Jeroen: Thanks! Good suggestions. Seems split is being depricated though, but guess that does not matter for codegolving.

Answer 26

AppleScript, 162 bytes

Interestingly, this is almost identical to the non-repeating characters thing.

set x to(display dialog""default answer"")'s text returned's words
set o to""
repeat with i in x
considering case
if not i is in o then set o to o&i&" "
end
end
o

I didn't actually know the considering keyword before this. the more you know...

Answer 27

Burlesque, 6 bytes

blsq ) "cat dog cat dog bird dog Snake snake Snake"wdNBwD
cat dog bird Snake snake

Rather simple: split words, nub (nub = remove duplicates), convert back to words.

Answer 28

Gema, 21 characters

*\S=${0ドル;0ドル}@set{0ドル;}

(Very similar to the unique character solution, as there are no arrays in Gema, so allowing built-in unique functions not helps us much.)

Sample run:

bash-4.3$ gema '*\S=${0ドル;0ドル}@set{0ドル;}' <<< 'cat dog cat dog bird dog Snake snake Snake'
cat dog bird Snake snake 

Answer 29

Scala, (削除) 44 (削除ここまで) 47 bytes

(s:String)=>s.split(" ").distinct.mkString(" ")

EDIT: using toSet might not preserve order, so I'm now using distinct // that just cost me 3 bytes :(

Answer 30

PHP, 37 Bytes

Assuming $s is the input string.

print_r(array_flip(explode(' ',$s)));