Pages

Thursday 4 July 2013

Hadoop Servers








Client Server


  • Client is neither a master or a slave. Its role here is to store data on hadoop cluster and submit MR jobs with instructions saying how data has to be processed. Also Client can retrieve & view the result after completion of job.
  • Data Storage:  Client accesses the file system on behalf of users by communicating with NameNode & DataNode.
  • Data Processing: To begin the data processing job, client calls the Job Tracker (Master Daemon), Job Tracker then partitions the work & assigns different Map & Reduce tasks to each Task Tracker in the Hadoop Cluster.




Master Servers (Machines)

  • Key Role of masters is to store large amount of data on HDFS & perform parallel computation on this data using Map Reduce programs (Algorithms). 
  • NameNode, Secondary NameNode & Job Tracker are the masters servers in Hadoop.
  • Master Servers are CPU/Memory intensive.
  • Master Servers always assign & monitor the task, they never work on data directly.
  • In a hadoop cluster there will be only 1 NameNode, 1 Secondary NameNode & 1 Job Tracker.
  • NameNode is also called as Brain of Hadoop as it stores the Metadata.
  • Secondary NameNode is also called as CheckPoint Node & it does all housekeeping job. Secondary NameNode is not the backup for Primary NameNode during failure state but with some manual intervention, cluster can be reconfigured to use Secondary NameNode as a Primary NameNode.
  • Job Tracker overseas & coordinates the parallel processing of data using MapReduce algorithms.




Slave Servers (Machines) 

  • Key role of slaves is to do all dirty job, they are also called as work horses in Hadoop Cluster.
  • Slaves always work on the task assigned by masters, they dont act/work by there own.
  • Slave Servers are Storage intensive. So data node always have lots of storage
  • DataNode & Task Tracker are daemons run on every slave node (Machine).
  • Data will be actually stored on DataNodes in equal size chunks/blocks.
  • Task Tracker monitors task running on individual slave machine and coordinates with JobTRacker.
  • Each slave machine has Task Tracker daemon.
  • DataNode is slave to NameNode & they primarily act while storing data to HDFS.
  • Task Tracker is slave to Job Tracker and they primarily act while running the parallel computation using MapReduce.
  • DataNode to DataNode communication in Hadoop Cluster is removed except during data replication.




Master Slave Relation in Hadoop Cluster - (Data Storage & Data Processing)






























76 comments:

  1. This structure help a lot to understand structure of hadoop server. Appreciable post.

    ReplyDelete
  2. Thank for sharing this great Hadoop tutorials Blog post. I will use your command when upgrade hadoop.I get a lot of great information here and this is what I am searching for. Thank you for your sharing. I have bookmark this page for my future
    reference.
    Hadoop Training in hyderabad

    ReplyDelete
  3. Hi,
    Thanks for nice data provided easy way to learn training with real time on
    hadoop online training
    through the experienced experts

    ReplyDelete
  4. Thanks for your support, i am very interested in learning Hadoop.. If you want more details on HADOOP BIGDATA
    just go through this link.....HADOOP TRAINING

    ReplyDelete
  5. The Hadoop tutorial you have explained is most useful for begineers who are taking Hadoop Administrator Online Training
    Thank you for sharing Such a good tutorials on Hadoop

    ReplyDelete
  6. i want run hadoop system for experimental purpose, for my acadmic reseach. i have core2quad ,4 GB, 320 GB.
    is this sufficient for making <10 cluster per cluster 15 node. Android training in chennai | Oracle dba Training in Chennai | Python Training in chennai

    ReplyDelete
  7. Very interested in IBM's move into this space. Agree with the premise that Hadoop designed to scale out...be very interesting to see how well that premise holds up as the market matures, scale increases and total costs (not just acquistion costs) become better known. How easily the x86 Hadoop ecosytem moves and/or ports to Power8 is huge (so far) unknown.

    ReplyDelete
  8. Excellent article, would been interesting to balance these options versus cloud such as AWS's EMR, Azure, etc.
    Vmware Training in Chennai |
    Pega Training in Chennai

    ReplyDelete
  9. thank you for your valuable content very useful data and recommended one.the way you explain is very immpressive

    Hadoop Online trainings

    ReplyDelete
  10. Nice blog.Thanks for sharing.Madschool providing best NATA, NID, NIFT Coaching Centers in Hyderabad and giving information about NATA important dates, NID syllabus, group discussions and tips, nata sample papers, nata,nid and nift registration and nata study material.

    ReplyDelete
  11. Very useful blog. Thankx for sharing this wonderful information, find Nata Training Institute Hyderabad .

    ReplyDelete
  12. This is my first valuable Hadoop blog. Thanks for sharing information about Hadoop Servers, keep sharing like this...

    This is the one of the best training institute for learning Hadoop Training in Hyderabad.

    ReplyDelete
  13. This comment has been removed by the author.

    ReplyDelete
  14. Nice Article on Hadoop.we are the best providers of Hadoop Training in Hyderabad

    ReplyDelete
  15. it is very informative blog and useful article thank you for sharing with us ,keep posting learn more about bigdata training,Bigdata Hadoop Online training
    Big data hadoop online training

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. Webtrackker Technology is offering Digital marketing course in noida Webtrackker for all learners and students. Here Students are access all types of services for quality results.

    Best Web Design Training Institutes in Noida

    Salesforce Training institute in noida

    Hadoop Training Institute in Noida

    Digital Marketing Training Institute in Noida

    ReplyDelete
  18. The My Genius Mind is a well-known academic portal known for offering the best Australia Assignment Services to the students at reasonable rates.
    Contact Information
    My Genius Mind
    Victoria, Melbourne, Australia
    Live Chat @ https://www.mygeniusmind.com/

    ReplyDelete
  19. Each department of CAD have specific programmes which, while completed could provide you with a recognisable qualification that could assist you get a job in anything design enterprise which you would really like.
    AutoCAD training in Noida

    AutoCAD training institute in Noida


    Best AutoCAD training institute in Noida

    ReplyDelete
  20. Video editing course in Noida
    Video editing training institute in Noida- Webtrackker Technology is and IT Training institute providing the Video editing course in Noida, FCP, Final Cut Pro Training in Noida. For more call us- 8802820025.
    Video editing course in Noida
    Company Address:
    Webtrackker Technology
    C- 67, Sector- 63, Noida
    Phone: 01204330760, 8802820025
    Email: info@webtrackker.com
    Website: http://webtrackker.com/Best-training-institute-Video-editing-FCP-course-in-Noida.php

    ReplyDelete
  21. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.
    nebosh courses in chennai

    ReplyDelete
  22. I’ve bookmarked your site, and I’m adding your RSS feeds to my Google account.
    fire and safety course in chennai

    ReplyDelete
  23. When cooking with oil, you will see the fact that smoke usually receives emitted in case you often uses the identical oil. Typically, these form of eating places have today's hoods as well as exhaust fans.
    Visit here
    Kitchen Chimney Repair Service in Noida
    Kitchen Kitchen Chimney Repair Service in Vaishali
    Kitchen Kitchen Chimney Repair Service in indirapuram
    Kitchen Kitchen Chimney Repair Service in vasundhra
    Kitchen Kitchen Chimney Repair Service in faridabad

    ReplyDelete
  24. A model is a person with a function either to sell, show or market it industrial products (appreciably style garb in style indicates), or to serve as a visual useful resource for individuals who are creating works of artwork or to pose for images +919910294610.
    Whatsapp number - 9910294610
    Arvind Rawat model in Noida
    Visit here
    Arvind Rawat Model in Noida
    Arvind Rawat Model in Noida

    ReplyDelete
  25. nice article
    I Found this article useful same as your. you can visit by clicking java training in Hyderabad

    ReplyDelete
  26. Really awesome
    I Found this article useful same as your. you can visit by clicking java training in Hyderabad

    ReplyDelete
  27. Thanks for sharing your valuable information, We are looking forward for more information.

    ReplyDelete
  28. Sound like very interesting, I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.

    Python Training in Chennai | Python Training Institute in Chennai

    ReplyDelete
  29. Excellent read, Positive site, where did u come up with the information on this posting? I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective workPython Training in Hyderabad

    ReplyDelete
  30. Hach Technologies11 October 2019 at 15:04

    HACH TECHNOLOGIES is providing in-depth RPA online training with LIVE project experience.Click on the below URL for registration and to know more about RPA(UI path & Blueprism) Training program

    ReplyDelete
  31. good blog

    https://www.wikitechy.com/interview-questions/aptitude/numbers/which-of-the-following-numbers-is-not-divisible-by-14
    https://www.wikitechy.com/errors-and-fixes/sql/error-the-specified-value-for-setting-medialayout-is-invalid-while-installing-ssms-from-command-line
    https://www.wikitechy.com/online-videos/company-interview-questions-and-answers/brac-interview-questions-and-answers-part1
    https://www.wikitechy.com/interview-questions/software-testing/write-the-test-cases-on-prime-number-with-result
    https://www.wikitechy.com/interview-questions/verbal-reasoning/blood-relation/pointing-out-to-a-photograph-swathi-said-he-is-the-uncle-of-my-brothers-sister
    https://www.wikitechy.com/tutorials/linux/how-to-merge-two-ar-static-libraries-into-one
    https://www.wikitechy.com/resume/tag/java-sample-resume-7-years-experience/
    https://www.wikitechy.com/interview-questions/aptitude/logarithm/find-the-value-of-log-a2-bc
    https://www.wikitechy.com/interview-outfits/hungarian-clothing
    https://www.wikitechy.com/tutorials/javascript/what-is-javascripts-highest-integer-value-that-a-number-can-go-to-without-losing-precision

    ReplyDelete
  32. Thanks for your valuable information,

    ReplyDelete
  33. I like your post very much. It is very much useful for my research. I hope you to share more info about this. Keep posting
    angular training
    ruby on rails online course
    ai online training
    Qlikview Training
    Spark Training

    ReplyDelete
  34. This comment has been removed by the author.

    ReplyDelete
  35. Thank you for sharing this useful information

    Online Live Trainings

    ReplyDelete
  36. Thanks for your support, i am very interested in learning Software Testing. If you want more details on Software Testing
    just go through this link Software
    Testing

    ReplyDelete
  37. Microsoft Windows Azure Training in Noida





    This is a very well written article. I will be sure to bookmark it and come back to read more of your useful information.

    ReplyDelete
  38. thanks for sharing very useful information about online training

    ReplyDelete
  39. just loved your article on the beginner's guide to starting a blog. If somebody takes this blog article seriously in their life, he/she can earn his living by doing blogging.thank you for this article if you looking for Latest Trending course like DevOps, AWS, Ethical Hacking & Bug BOunty Click here www.techmarshals.com

    ReplyDelete
  40. Very useful information, thanks for sharing with us, keep up the good work!

    ReplyDelete
  41. This article is very useful, who want to learn Data science. Data science is a development methodology with a set of practices aimed at reducing the time between Development and Operations. Now a day’s data Science is best future career for students.

    ReplyDelete
  42. Usually, I never comment on blogs but your article is so convincing that I never stop myself to say something about it. I really like this post and Enjoyed reading the article above, really explains everything in detail, the article is very interesting and effective. I am also providing coaching on angular js Angular training in Hyderabad just go through the link

    MindQ Systems provides Training on Angular js, Angular2, Angular4, Angular5, Angular6, Angular7, Angular8. Mind Q provides Classroom Training, Online Training and video course for Angular Training. Mindq provides Training with an expert Trainer. Enroll now and schedule a demo session with the trainer.
    angularjs

    ReplyDelete
  43. python training in hyderabad i am enjoyed while reading your blog .thanks for sharing your article and keep sharing

    ReplyDelete
  44. Thanks for sharing Valuable information about hadoop. Really helpful.Keep sharing more posts about hadoop.

    ReplyDelete
  45. We offer comprehensive Online Python Training that covers strong basics level to advanced level concepts. As part of this online Python course, you will be working on Real-time Live-Projects with right from your hand and assignments that we have immense implications in the real-world industry scenarios with Flexible timing. Enroll now and boost your career.
    • 10+ years Experienced trainer
    • 100% Placement Assistance
    • Affordable Price
    For Further Details: 7619550011
    Visit: https://bit.ly/37HfVvF

    ReplyDelete
  46. Wonderful Information on AWS Training in Chennai by MNC Expert at TIC Academy.

    ReplyDelete
  47. Hey check this website for more stuffs related to data science

    ReplyDelete
  48. I also have saved you to look at new information in your site.
    360digiTMG machine learning course

    ReplyDelete
  49. I also have saved you to look at new information in your site.
    360digiTMG machine learning course

    ReplyDelete
  50. Wonderful illustrated information. I thank you about that. No doubt it will be very useful for my future projects. Would like to see some other posts on the same subject!
    digital marketing courses in hyderabad with placements

    ReplyDelete
  51. Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts.
    digital marketing courses in hyderabad with placements

    ReplyDelete

  52. Two full thumbs up for this magnificent article of yours. I've really enjoyed reading this article today and I think this might be one of the best articles that I've read yet. Please, keep this work going on in the same quality.

    Data Science Training in Hyderabad

    ReplyDelete

Please enter comments related to post only..!! Thank You..!!