Server side processing or client side processing?

Question

I have developed a comparison website for comparing any product sold online in India. Currently, the site is completely client side :-

1. Accepts user input.
2. Makes 20-30 AJAX requests and fetches results from all the major online shops.
3. Uses some client-side scripting to sort the results and show it in most appropriate way.

Disadvantages :-

1. My client side code is available to everyone. Its javascript.
2. More prone to browser errors.
3. Not robust.

Disadvantages after making it server-side:-

1. Considering the traffic of my website, server load will increase as it will be engaged with a client for longer time.
2. Fetching values from various websites can take as much as 10s(at max).Server engaged for that time. Consider the load if I have 500 visitors/min at peak time.

Advantages:-

1. My codes safe and secure
2. Processing at client side will be minimum. Will work even on mobiles and other devices easily.

I want to analyze about these issues before actually implementing them. Can anyone suggest me about what should I choose for my website ? Which approach will be better for me ?

Please comment if my question is ambiguous.

• iJade: just out of curiosity....is it legal scraping web sites such as ebay?
• IdidntKnewIt: You are web scraping, look at TOS of snapdeal man.

Answer 1

Well first of all thats a very nice question.

it completely depends on the volume and transactions your site handles and if you really want your application to scale, i wish you do it right! and do it right from the beginning.

Stop putting your business logic on the client side ** dont expect to eat end user network bandwidth when he makes a comparison call :) and dont expect he has the best bandwidth.

Load balance your server farm make sure your server farm is properly load balanced and the comparison is done using multi threads instead in a single thread

cache results ** if you are doing this on the server side, if user a and user b asks for the same comparison you can actually pull it from the cache for user b instead doing these requests again.

Usability show the progress of comparison to user instead showing a loading spinning image :)

hope it helps

• Prashant Singh: I am working on converting it into a multi-thread design. Can you elaborate about how to cache. We are not using any database now, We scrape it from target website, process and show. All within 6-7 seconds
• Chandra Sekhar Walajapet: @PrashantSingh whats your server farm ? and the technology behind your application ?
• Prashant Singh: Its simple PHP which uses CURL to fetch the content of the webpages. From the fetched data , we scrap out the results
• Chandra Sekhar Walajapet: i would recommend the processed data to cache instead caching the result of CURL. you can try looking in to this to try cache in php php-html.net/tutorials/creating-a-simple-php-cache-script but i would recommed you go with a cache provider like Memcache so that you scaling etc will also be easy

Answer 2

I can't see why you are taking the "More prone to browser errors". Javascript will run exactly the way you want it if you send to the browser the correct libraries/code.

You can see what browser is doing the request and send it the right piece of code (read as: if you have some code that will work in FF & Opera but not IE, write two versions of that code, one for each group of browsers.) There are some libraries that can help you doing so.

Also, a comparison is not anything that should be done on the server side. SO, if you have a lot of traffic on your website, client-side work should be fine for that kind of stuff.

About the code-protection, you're right. Server-side won't let anybody read your code. So you must decide if server load is more important than people reading your code.

Also keep in mind that you can obfuscate your code. I know that's not the best solution, but it will keep away a lot of people not reading it.

EDIT:

Making it server side will make things work as expected on all devices (as your already said). But there are some things you need to do in order to preserve your server's load from going to 100%.

Chandra Sekhar Walajap already told you about the benefit of using some kind of cache for the results. I'd personally go a little bit further as you're just scraping all those pages.

I'd create a scraper that will run, let's say, every 24 hours and fetch/scrape each product. Then I'd save all those products somewhere (read as in a database or whatever you want). This way, when a user makes a request for a comparison between products A and B you won't need to fetch/scrape all the sites, instead of that you'll just have to look for those products in the place where you saved them and show them to the user.

Note that this will preserve a lot of bandwidth only if you have a lot of users comparing a lot of products.

On the other side, if you have a few users searching only for a few products, making that kind of solution won't benefit you at all, because you know, fetching everything from all the sites every 24 hours is a lot of usage, both in CPU and bandwidth.

About the server load: I can't say. It depends on so much things that it's nearly impossible to say. You need to consider how big are the website that you're scraping , how many products are there on each website? What's the hardware you're using? You'd be better by making some simple example using cURL and fetching everything from one of the sites. Then see how that impacts on the performance and decide.

• Prashant Singh: I am facing several issues in IE. Like for making 20-30 Ajax request simultaneously, I need to use CORS to avoid blocking. Not supported by many of the browsers(old ones). Making sequential call will substantially degrade the performance of my engine
• Alexander Nestorov: Well, if you want to support really old browser (FYI IE8 has some basic CORS support) your only option is server side.
• Prashant Singh: Can you give me an estimate about the server side load and update your answer with some tips to be followed while going for server side implementation ?