Saturday, July 2, 2011

Realtime Hadoop usage at Facebook: The Complete Story

I had earlier blogged about why Facebook is starting to use Apache Hadoop technologies to serve realtime workloads. We presented the paper at the SIGMOD 2011 conference and it was very well received.

Here is a link to the complete paper for those who are interested in understanding the details of why we decided to use Hadoop technologies, the workloads that we have on realtime Hadoop, the enhancements that we did to Hadoop for supporting our workloads and the processes and methodologies we have adopted to deploy these workloads successfully. A shortened version of the first two sections of the paper are also described in the slides that you can find here.
[フレーム]

20 comments:

  1. On the flip side of the coin, Cloudera is also stepping up a notch with the announcement of Enterprise 3.5, the latest version of its Hadoop management offering. Some of the new features include real-time monitoring.

    Reply Delete
  2. Nice blog post. I'm not that much aware of this thing but then, thanks.

    Reply Delete
  3. Thank you..its very helpfull dhruba.. actually i'm planning to work with HBase

    Reply Delete
  4. What a wonderful world. What a good job for this post. Very rich and constructive at the same time. I want to say a thumbs up to the creator for keeping this web site simple. Congratulations finally a web site of top-level. Have a nice day!

    Reply Delete
  5. Finally! I've been waiting for months for Facebook to begin using an efficient technology like Apache Hadoop. I've noticed a real improvement in speed since this change has taken place. Keep up the good work Hadoop!

    Reply Delete
  6. Hi Drubha,
    Thanks for the post and links to the white paper. Intersting read.

    Do you have any additional details you can share on muti data center replication. Do you have that in facebook today, if not what are some of the ideas on accomplishing a minimal downtime in case a data center is down.
    thanks

    Reply Delete
  7. @mridula, I do not yet have much details about multi-data center replication. We are working on such a product, will update you here when we get to some sane design :-)

    Reply Delete
  8. Thanks :)

    Reply Delete
  9. Thanks for sharing the links.
    _________________
    Jobs In Brisbane

    Reply Delete
  10. Hello Sir,

    Thanks for posting these files
    that was good. but I am not aware of these things.

    Can you suggest a project which is easy and done in 3 months time.

    Thankk you sir.

    Reply Delete
  11. Is NFS irrelevant going forward? Can a linux node/cluster run on hdfs alone?

    Reply Delete
  12. Apache HTTP is one of the most widely used Web server software in the world today, with more than 100 million Web sites using it. And because it is open source, it is a very dynamic and has become a robust Web server it is today. What is more, it is free to download and use.

    apache jobs

    Reply Delete
  13. They could also switch your existing application servers to J2EE or Java platforms such as WebSphere, Weblogic or Apache.

    apache jobs

    Reply Delete
  14. Hi nice .. Thanks for this posting..

    Thanks

    Reply Delete
  15. Thanks for sharing with us.This provide more useful information for us. I have one of my Recycling Center Chicago business, i hope you like it.

    Reply Delete
  16. I would be very thankful if you continue with quality what you are serving right now with your blog…I really enjoyed it…and i really appriciate to you for this….its always pleasure to read so….Thanks for sharing!!

    Reply Delete
  17. I am desire associated with examining rejuvenating content. Keep the great run! Excellent web page. My wife and i adored disregarding it?s fundamentally an incredible read for several. We now have included in our favswill lemon juice help acne
    Whenever i must condition, as being a pile while
    does Clear Skin Max work
    i precious researching what you may important to state,
    External Hemorrhoids causes
    Internal Hemorrhoids Symptoms
    When i won't aid nevertheless decline curiosity suitable just before a long time. For the reason that while you obtained any incredible love regarding the matter topic.

    Reply Delete
  18. This discussion is very focused on the topic and I’m satisfied with the researched material as is is authentic and unbiased.

    Reply Delete

Subscribe to: Post Comments (Atom)

AltStyle によって変換されたページ (->オリジナル) /