Creating consecutive words from a string

Question

This is a method which creates an array of all possible consecutive substrings from a string:

def get_seperated_tokens(query)
 result = []
 length = query.split.count
 tokens = query.downcase.strip.split(' ')
 length.times do |i|
 length.times do |j|
 result << tokens[i..j].join(' ') if j >= i
 end
 end
 result
end

To get a better idea I have added rspec for it:

describe "#get_seperated_tokens" do
 it "returns an array of seperated tokens" do
 query = 'ruby is awesome'
 result = ['ruby','is', 'awesome', 'ruby is', 'is awesome','ruby is awesome']
 expect(get_seperated_tokens(query)).to include(*result)
 end
 it "returns an array of seperated tokens" do
 query = 'red blue iphones'
 result = ['red','blue', 'iphones', 'red blue', 'blue iphones','red blue iphones']
 expect(get_seperated_tokens(query)).to include(*result)
 end
end

How can this be made more idiomatic?

• Mike Szyndel: ruby-doc.org/core-1.9.3/Array.html#method-i-combination

Answer 1

As @Michael Szyndel mentioned in a comment, Array#combination is the more appropriate method to use.

def get_separated_tokens query
 tokens = query.split
 (0..tokens.size).to_a.combination(2).map{ |i,j| tokens[i...j].join " " }
end

• tokland: In fact Michael wrote an answer with combination (now deleted), the problem is that OP needs contiguous combinations, so this won't do.
• Nakilon: @tokland, I don't get you. Did you run this code? It produces the same as OP's code.
• tokland: oh, you're right, you use combinations, but with a slice later, which is right. +1

Answer 2

Are you familiar with functional programming? (my page on the subject: FP with Ruby). Your code feels clunky because, well, imperative programming is clunky (recommended reading: Can Programming Be Liberated From The Von Neumann Style?).

You just need to re-write the code without times (as it's being used as an each), mutable variables (result = []), inplace operations (<<) and inline conditionals with side-effects (do_something if j >= i). I'd write:

def get_separated_tokens(query)
 tokens = query.split
 (0...tokens.size).flat_map do |istart|
 (istart...tokens.size).map do |iend|
 tokens[istart..iend].join(" ")
 end
 end
end
p get_separated_tokens("ruby is awesome")
#["ruby", "ruby is", "ruby is awesome", "is", "is awesome", "awesome"]

• tokland: You can select Nakilon's answer, I calculated the combinations to build ranges, we have Array#combination.