Infinitely scrolling display of web-scraped search results

Question

My web application is simply supposed to take a search query, run a script on it that returns product information and then post it in an "endless" fashion on a web page, making one page with no pagination that is "infinitely" scrollable.

The web scraper I use, called Fletcher, which helps return usable results is slow to fetch each product matching the query. But, as soon as it returns information, I want to update my web page without refreshing it.

I'm thinking of the best way to do this so I don't have to refresh the page, but I don't know exactly where to start. I've had Ajax recommended and I find Backbone.js intriguing, but, really, I know I can do this with Erb, Ruby, Rails, and not too much else.

Please take a look at what I have. Any kind of code review at all will be helpful.

GitHub

Application Controller:

class ApplicationController < ActionController::Base
 # before_filter :authorize
 protect_from_forgery
 private
 def current_cart
 Cart.find(session[:cart_id])
 rescue ActiveRecord::RecordNotFound
 cart = Cart.create
 session[:cart_id] = cart.id
 cart
 end
 protected
 require 'open-uri'
 def authorize
 unless User.find_by_id(session[:user_id])
 redirect_to login_url, notice: "Please log in"
 end
 end
 def nokoSearch (keywords, vendor)
 # def available (info)
 # if (info!=nil)
 # else
 # info = ""
 # end
 # end
 @product_links = Array.new
 @name = ""
 @description = ""
 @image = ""
 @price = ""
 keywords = keywords.split(' ').join('+')
 noko_links = Array.new
 base_url = "http://www.amazon.com/s/url=search-alias%3Daps"
 keyword_url = base_url + "&field-keywords=" + keywords
 page = Nokogiri::HTML(open(keyword_url))
 @product_links = page.css("a").select{|link| link['class'] == "title"}
 @product_links.compact! # get rid of nils
 @product_links.each do |link|
 fletchedProduct = Fletcher.fetch link['href']
 @name = fletchedProduct.name
 @description = fletchedProduct.description
 if (fletchedProduct.image != nil)
 @image = fletchedProduct.image[:src]
 end
 @price = fletchedProduct.prices
 InfiniteProduct.create(:name => @name, :description => @description, :image => @image, :price => @price)
 end
 end
end

index.html.erb for the view:

<% @infinite_products.each do |infinite_product| %>
 <li class="span3">
 <div class="thumbnail">
 <%= image_tag(infinite_product.image) %>
 <div class="caption">
 <h5><%= sanitize(infinite_product.name) %></h5>
 <p><%= sanitize(infinite_product.description) %></p>
 <p><a class="btn primary details" href="#" rel="ajax/1.html"><i class="icon-zoom-in"></i></a> <a class="btn addto" href="#" rel="1">Add to <i class="icon-shopping-cart"></i></a> <span class="label label-info price"><%= infinite_product.price %></span></p>
 </div>
 </div>
 </li>
<% end %>

Controller:

class InfiniteProductsController < ApplicationController
 # GET /infinite_products
 # GET /infinite_products.json
 def index
 #nokoSearch("batman", "amazon")
 @infinite_products = InfiniteProduct.order(:name)
 respond_to do |format|
 format.html # index.html.erb
 format.json { render json: @infinite_products }
 end
 end
 # GET /infinite_products/1
 # GET /infinite_products/1.json
 def show
 @infinite_product = InfiniteProduct.find(params[:id])
 respond_to do |format|
 format.html # show.html.erb
 format.json { render json: @infinite_product }
 end
 end
 # GET /infinite_products/new
 # GET /infinite_products/new.json
 def new
 @infinite_product = InfiniteProduct.new
 respond_to do |format|
 format.html # new.html.erb
 format.json { render json: @infinite_product }
 end
 end
 # GET /infinite_products/1/edit
 def edit
 @infinite_product = InfiniteProduct.find(params[:id])
 end
 # POST /infinite_products
 # POST /infinite_products.json
 def create
 @infinite_product = InfiniteProduct.new(params[:infinite_product])
 respond_to do |format|
 if @infinite_product.save
 format.html { redirect_to @infinite_product, notice: 'Infinite product was successfully created.' }
 format.json { render json: @infinite_product, status: :created, location: @infinite_product }
 else
 format.html { render action: "new" }
 format.json { render json: @infinite_product.errors, status: :unprocessable_entity }
 end
 end
 end
 # PUT /infinite_products/1
 # PUT /infinite_products/1.json
 def update
 @infinite_product = InfiniteProduct.find(params[:id])
 respond_to do |format|
 if @infinite_product.update_attributes(params[:infinite_product])
 format.html { redirect_to @infinite_product, notice: 'Infinite product was successfully updated.' }
 format.json { head :no_content }
 else
 format.html { render action: "edit" }
 format.json { render json: @infinite_product.errors, status: :unprocessable_entity }
 end
 end
 end
 # DELETE /infinite_products/1
 # DELETE /infinite_products/1.json
 def destroy
 @infinite_product = InfiniteProduct.find(params[:id])
 @infinite_product.destroy
 respond_to do |format|
 format.html { redirect_to infinite_products_url }
 format.json { head :no_content }
 end
 end
end

Answer 1

You can't do this without javascript because Rails is a server side language while you should work client side where the language to be used is Javascript.

Basically you should:

1-)Know when an user is near to the bottom of the page

2-)Send an ajax request to the url that returns informations (in json format so it will have to load a lot less data and will be easier to parse the result)

3-)Manipulate the DOM adding the new result returned by the ajax function.

Here there are some plugins in jQuery (a library written in javascript) http://www.jquery4u.com/tutorials/jquery-infinite-scrolling-demos/#.T-NgN7R1Aak

• David West: Thanks for taking the time to answer Matteo. Do you know how to use the form_tag and text_field_tag helpers in a .html.erb file to send an ajax GET request to a url and use the text_field's contents as params?
• Matteo Pagliazzi: i don't think using rails method is the best way since they'll only create some javascript code, anyway i think that there's an attribute :remote to use ajax in forms while i advise you to use directly jquery
• Matteo Pagliazzi: really it should be quite simple using one of the jquery code i've posted to you