{"id":939,"date":"2011-06-17T10:56:55","date_gmt":"2011-06-17T15:56:55","guid":{"rendered":"http:\/\/www.migrate2cloud.com\/blog\/?p=939"},"modified":"2016-03-21T02:49:15","modified_gmt":"2016-03-21T07:49:15","slug":"hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04","status":"publish","type":"post","link":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/","title":{"rendered":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04"},"content":{"rendered":"<p>Let&#8217;s start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google&#8217;s MapReduce and Google File System (GFS) papers.<\/p>\n<p>Technically, Hadoop consists of two key services: reliable data storage using the Hadoop Distributed File System (HDFS) and high-performance parallel data processing using a technique called MapReduce.<\/p>\n<p>Dealing with big data requires two things:<\/p>\n<ul>\n<li>Inexpensive, reliable storage; and<\/li>\n<li>New tools for analyzing unstructured and structured data.<\/li>\n<\/ul>\n<p>Hadoop creates <em>clusters <\/em>of machines and coordinates work among them. Clusters can be built with inexpensive computers.If one fails, Hadoop continues to operate the cluster without losing data or interrupting work, by shifting work to the remaining machines in the cluster.<\/p>\n<p>HDFS manages storage on the cluster by breaking incoming files into pieces, called \u201cblocks,\u201d and storing each of the blocks redundantly across the pool of servers.<\/p>\n<p>The main services running in a hadoop cluster will be<\/p>\n<p>1)namenode<\/p>\n<p>2)jobtracker<\/p>\n<p>3)secondarynamenode<\/p>\n<p>These three will be running only on a single node(machine) ; that machine is the central machine which controls the cluster.<\/p>\n<p>4)datanode<\/p>\n<p>5)tasktracker<\/p>\n<p>These two services will be running on all other nodes in the cluster.<\/p>\n<p>HDFS has a master\/slave architecture. An HDFS cluster consists of a single NameNode, a master server that manages the file system namespace and regulates access to files by clients. In addition, there are a number of DataNodes, usually one per node in the cluster, which manage storage attached to the nodes that they run on.<\/p>\n<p>Above the file systems comes the MapReduce \u00a0engine, which consists of one <em>Job Tracker<\/em>, to which client applications submit MapReduce jobs. The Job Tracker pushes work out to available <em>Task Tracker<\/em> nodes in the cluster, striving to keep the work as close to the data as possible.<\/p>\n<p>The only purpose of the secondary name-node is to perform periodic checkpoints. The secondary name-node periodically downloads current name-node image and edits log files, joins them into new image and uploads the new image back to the (primary and the only) name-node.<\/p>\n<p><strong>Now Let us have a look at how to build a hadoop cluster using Cloudera hadoop-0.20 on ubuntu-10.04<\/strong><\/p>\n<p>You should install sun \u2013jdk\u00a0 first. Then add the following repositories to the apt sources list.<\/p>\n<p>vim \/etc\/apt\/sources.list.d\/cloudera.list<\/p>\n<p>[bash]<\/p>\n<p>deb http:\/\/archive.cloudera.com\/debian lucid-cdh3u0 contrib<\/p>\n<p>deb-src http:\/\/archive.cloudera.com\/debian lucid-cdh3u0 contrib<\/p>\n<p>[\/bash]<\/p>\n<p>Import key<\/p>\n<p>[bash]curl -s http:\/\/archive.cloudera.com\/debian\/archive.key | apt-key add -[\/bash]<\/p>\n<p>Then run<\/p>\n<p>[bash]apt-get update[\/bash]<\/p>\n<div>\n<p><span style=\"text-decoration: underline;\"><strong>For Namenode\/Jobtracker ( These two services should run only on a single central machine in the cluster)<\/strong><\/span><\/p>\n<p>[bash]<\/p>\n<p>apt-get install hadoop \u2013yes<\/p>\n<p>apt-get install hadoop-0.20-namenode<\/p>\n<p>apt-get install hadoop-0.20-jobtracker<\/p>\n<p>apt-get install hadoop-0.20-secondarynamenode<\/p>\n<p>[\/bash]<\/p>\n<p><span style=\"text-decoration: underline;\"><strong>Configuration<\/strong><\/span><\/p>\n<p><strong><em>vim \/etc\/hadoop\/conf\/hadoop-env.sh<\/em><\/strong><\/p>\n<p><em>Append these<\/em><\/p>\n<p>[bash]<\/p>\n<p>export JAVA_HOME=\/usr\/lib\/jvm\/java-6-sun-1.6.0.24\/\u00a0\u00a0 ( your java home comes here )<\/p>\n<p>export HADOOP_CONF_DIR=\/etc\/hadoop\/conf<\/p>\n<p>export HADOOP_HOME=\/usr\/lib\/hadoop-0.20<\/p>\n<p>export HADOOP_NAMENODE_USER=hdfs<\/p>\n<p>export HADOOP_SECONDARYNAMENODE_USER=hdfs<\/p>\n<p>export HADOOP_DATANODE_USER=hdfs<\/p>\n<p>export HADOOP_JOBTRACKER_USER=mapred<\/p>\n<p>export HADOOP_TASKTRACKER_USER=mapred<\/p>\n<p>export HADOOP_IDENT_STRING=hadoop<\/p>\n<p>[\/bash]<\/p>\n<p><strong><em>vim \/etc\/hadoop\/conf\/core-site.xml<\/em><\/strong><\/p>\n<p>[bash]<\/p>\n<p>&lt;?xml version=&#8221;1.0&#8243;?&gt;<\/p>\n<p>&lt;?xml-stylesheet type=&#8221;text\/xsl&#8221; href=&#8221;configuration.xsl&#8221;?&gt;<\/p>\n<p>&lt;!&#8211; Put site-specific property overrides in this file. &#8211;&gt;<\/p>\n<p>&lt;configuration&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;fs.default.name&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;hdfs:\/\/&lt; ip address of this machine &gt;:8020&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;\/configuration&gt;<\/p>\n<p>[\/bash]<\/p>\n<p><strong><em>vim \/etc\/hadoop\/conf\/hdfs-site.xml<\/em><\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>[bash]<\/p>\n<p>&lt;?xml version=&#8221;1.0&#8243;?&gt;<\/p>\n<p>&lt;?xml-stylesheet type=&#8221;text\/xsl&#8221; href=&#8221;configuration.xsl&#8221;?&gt;<\/p>\n<p>&lt;!&#8211; Put site-specific property overrides in this file. &#8211;&gt;<\/p>\n<p>&lt;configuration&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;dfs.name.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/name&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;dfs.data.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/data&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;dfs.replication&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;2&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;\/configuration&gt;<\/p>\n<p>[\/bash]<\/p>\n<p><strong><em>vim \/etc\/hadoop\/conf\/mapred-site.xml<\/em><\/strong><\/p>\n<p>[bash]<\/p>\n<p>&lt;?xml version=&#8221;1.0&#8243;?&gt;<\/p>\n<p>&lt;?xml-stylesheet type=&#8221;text\/xsl&#8221; href=&#8221;configuration.xsl&#8221;?&gt;<\/p>\n<p>&lt;!&#8211; Put site-specific property overrides in this file. &#8211;&gt;<\/p>\n<p>&lt;configuration&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;mapred.job.tracker&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;&lt; ip address of this machine &gt;:8021&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;mapred.system.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/system&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;mapred.local.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/mapred&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;\/configuration&gt;<\/p>\n<p>[\/bash]<\/p>\n<p><strong>&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;<\/strong><\/p>\n<p>[bash]<\/p>\n<p>mkdir\u00a0 \/ var\/lib\/hadoop-0.20\/name<\/p>\n<p>mkdir\u00a0 \/ var\/lib\/hadoop-0.20\/data<\/p>\n<p>mkdir\u00a0 \/ var\/lib\/hadoop-0.20\/system<\/p>\n<p>mkdir\u00a0 \/ var\/lib\/hadoop-0.20\/mapred<\/p>\n<p>chown -R hdfs \/var\/lib\/hadoop-0.20\/name<\/p>\n<p>chown -R hdfs \/var\/lib\/hadoop-0.20\/data<\/p>\n<p>chown -R mapred \/var\/lib\/hadoop-0.20\/mapred<\/p>\n<p>[\/bash]<\/p>\n<p>Now format NameNode<\/p>\n<p>[bash]yes Y | \/usr\/bin\/hadoop namenode \u2013format[\/bash]<\/p>\n<p>Start namenode<\/p>\n<p>[bash]\/etc\/init.d\/hadoop-0.20-namenode start[\/bash]<\/p>\n<p>Check the log Files for error:<\/p>\n<p>less \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-namenode-&lt;ip&gt;.log<\/p>\n<p>Also you can check whether the Namenode process is up or not using the command<\/p>\n<p>[bash]# jps[\/bash]<\/p>\n<p>Start the SecondaryNamenode<\/p>\n<p>[bash]\/etc\/init.d\/hadoop-0.20-secondarynamenode start[\/bash]<\/p>\n<p>Log: less \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-secondarynamenode-&lt;ip&gt;.log<\/p>\n<p>[bash]<\/p>\n<p>sudo -u hdfs hadoop fs -mkdir \/var\/lib\/hadoop-0.20\/system<\/p>\n<p>sudo -u hdfs hadoop fs -chown mapred \/var\/lib\/hadoop-0.20\/system<\/p>\n<p>[\/bash]<\/p>\n<p>Now Start the JobTracker<\/p>\n<p>[bash]\/etc\/init.d\/hadoop-0.20-jobtracker start[\/bash]<\/p>\n<p>Log : less \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-jobtracker-ip-10-108-39-34.log<\/p>\n<p>Now\u00a0 jps\u00a0 command will show the three processes up<\/p>\n<p># jps<\/p>\n<p>19233 JobTracker<\/p>\n<p>18994 SecondaryNameNode<\/p>\n<p>18871 NameNode<\/p>\n<\/div>\n<div>\n<p><span style=\"text-decoration: underline;\"><strong>For Datanode\/Tasktracker ( These two services should be running on all the other machines in the cluster )<\/strong><\/span><\/p>\n<p>[bash]<\/p>\n<p>apt-get install hadoop-0.20-datanode<\/p>\n<p>apt-get install hadoop-0.20-tasktracker<\/p>\n<p>[\/bash]<\/p>\n<div>\n<p><span style=\"text-decoration: underline;\"><strong>Configuration<\/strong><\/span><\/p>\n<\/div>\n<p><strong><em>vim \/etc\/hadoop\/conf\/core-site.xml<\/em><\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>[bash]<\/p>\n<p>&lt;?xml version=&#8221;1.0&#8243;?&gt;<\/p>\n<p>&lt;?xml-stylesheet type=&#8221;text\/xsl&#8221; href=&#8221;configuration.xsl&#8221;?&gt;<\/p>\n<p>&amp;nbsp;<\/p>\n<p>&lt;!&#8211; Put site-specific property overrides in this file. &#8211;&gt;<\/p>\n<p>&amp;nbsp;<\/p>\n<p>&lt;configuration&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;fs.default.name&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;hdfs:\/\/&lt; ip address of the namenode &gt;:8020&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;\/configuration&gt;<\/p>\n<p>[\/bash]<\/p>\n<div>\n<p><strong><em>vim \/etc\/hadoop\/conf\/hdfs-site.xml<\/em><\/strong><\/p>\n<p>[bash]<\/p>\n<p>&lt;?xml version=&#8221;1.0&#8243;?&gt;<\/p>\n<p>&lt;?xml-stylesheet type=&#8221;text\/xsl&#8221; href=&#8221;configuration.xsl&#8221;?&gt;<\/p>\n<p>&amp;nbsp;<\/p>\n<p>&lt;!&#8211; Put site-specific property overrides in this file. &#8211;&gt;<\/p>\n<p>&amp;nbsp;<\/p>\n<p>&lt;configuration&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;dfs.name.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/name&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;dfs.data.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/data&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;dfs.replication&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;2&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;\/configuration&gt;<\/p>\n<p>[\/bash]<\/p>\n<div>\n<p><strong><em>vim \/etc\/hadoop\/conf\/mapred-site.xml<\/em><\/strong><\/p>\n<p>[bash]<\/p>\n<p>&lt;?xml version=&#8221;1.0&#8243;?&gt;<\/p>\n<p>&lt;?xml-stylesheet type=&#8221;text\/xsl&#8221; href=&#8221;configuration.xsl&#8221;?&gt;<\/p>\n<p>&amp;nbsp;<\/p>\n<p>&lt;!&#8211; Put site-specific property overrides in this file. &#8211;&gt;<\/p>\n<p>&amp;nbsp;<\/p>\n<p>&lt;configuration&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;mapred.job.tracker&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;&lt; ip address of jobtracker\u00a0 &gt;:8021&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;mapred.system.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/system&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;property&gt;<\/p>\n<p>&lt;name&gt;mapred.local.dir&lt;\/name&gt;<\/p>\n<p>&lt;value&gt;\/var\/lib\/hadoop-0.20\/mapred&lt;\/value&gt;<\/p>\n<p>&lt;\/property&gt;<\/p>\n<p>&lt;\/configuration&gt;<\/p>\n<p>[\/bash]<\/p>\n<p><strong>&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;<\/strong><\/p>\n<p>[bash]<\/p>\n<p>mkdir\u00a0 \/var\/lib\/hadoop-0.20\/data\/<\/p>\n<p>chown -R hdfs \/var\/lib\/hadoop-0.20\/data<\/p>\n<p>mkdir \/var\/lib\/hadoop-0.20\/mapred<\/p>\n<p>chown -R mapred \/var\/lib\/hadoop-0.20\/mapred<\/p>\n<p>[\/bash]<\/p>\n<p>Start the DataNode<\/p>\n<p>[bash]\/etc\/init.d\/hadoop-0.20-datanode start[\/bash]<\/p>\n<p>Log : less \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-datanode-&lt;ip&gt;.log<\/p>\n<p>Start the TaskTracker<\/p>\n<p>[bash]\/etc\/init.d\/hadoop-0.20-tasktracker start[\/bash]<\/p>\n<p>Log: less \/usr\/lib\/hadoop-0.20\/logs\/hadoop-hadoop-tasktracker-&lt;ip&gt;.log<\/p>\n<p>You can now check the interface<\/p>\n<p>http:\/\/&lt; namenode-ip &gt;:50070\u00a0\u00a0 &#8211; for HDFS overview<\/p>\n<p>and<\/p>\n<p>http:\/\/&lt; jobtracker \u2013ip&gt;:50030\u00a0 &#8211; for Mapreduce overview<\/p>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Let&#8217;s start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google&#8217;s [&hellip;]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[34,3],"tags":[4,434,88,457,127,215,451,260,456,211,437,453,454,89,90,450,444,455,95,449,198,452,136,138],"class_list":["post-939","post","type-post","status-publish","format-standard","hentry","category-amazon-ec2","category-cloud-computing","tag-amazon","tag-apache-hadoop","tag-aws-ec2","tag-client-applications","tag-cloud-computing","tag-cloud-infrastructure","tag-cloudera","tag-company-technology","tag-company-apache-software","tag-company-google","tag-data-intensive-computing","tag-data-intensive-distributed-applications","tag-file-systems","tag-hadoop","tag-hadoop-cluster","tag-html-element","tag-java","tag-job-tracker","tag-mapreduce","tag-parallel-computing","tag-search-engines","tag-software-framework","tag-technologyinternet","tag-xml"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 4.9.10 - aioseo.com -->\n\t<meta name=\"description\" content=\"Let&#039;s start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google&#039;s\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"admin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 4.9.10\" \/>\n\t\t<meta property=\"og:locale\" content=\"en_US\" \/>\n\t\t<meta property=\"og:site_name\" content=\"Migrate to Cloud - we make the clouds rain\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud\" \/>\n\t\t<meta property=\"og:description\" content=\"Let&#039;s start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google&#039;s\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2011-06-17T15:56:55+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2016-03-21T07:49:15+00:00\" \/>\n\t\t<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n\t\t<meta name=\"twitter:title\" content=\"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud\" \/>\n\t\t<meta name=\"twitter:description\" content=\"Let&#039;s start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google&#039;s\" \/>\n\t\t<script type=\"application\/ld+json\" class=\"aioseo-schema\">\n\t\t\t{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"BlogPosting\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#blogposting\",\"name\":\"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud\",\"headline\":\"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04\",\"author\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/author\\\/admin-2\\\/#author\"},\"publisher\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/#organization\"},\"datePublished\":\"2011-06-17T10:56:55-05:00\",\"dateModified\":\"2016-03-21T02:49:15-05:00\",\"inLanguage\":\"en-US\",\"commentCount\":1,\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#webpage\"},\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#webpage\"},\"articleSection\":\"Amazon EC2, Cloud computing, amazon, Apache Hadoop, AWS EC2, client applications, Cloud computing, Cloud infrastructure, Cloudera, Company Technology, Company: APACHE SOFTWARE, Company: Google, Data-intensive computing, data-intensive distributed applications, file systems, Hadoop, Hadoop Cluster, HTML element, Java, Job Tracker, MapReduce, Parallel computing, Search Engines, software framework, Technology\\\/Internet, XML\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#breadcrumblist\",\"itemListElement\":[{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog#listItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/category\\\/cloud-computing\\\/#listItem\",\"name\":\"Cloud computing\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/category\\\/cloud-computing\\\/#listItem\",\"position\":2,\"name\":\"Cloud computing\",\"item\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/category\\\/cloud-computing\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#listItem\",\"name\":\"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog#listItem\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#listItem\",\"position\":3,\"name\":\"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04\",\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/category\\\/cloud-computing\\\/#listItem\",\"name\":\"Cloud computing\"}}]},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/#organization\",\"name\":\"Migrate to Cloud\",\"description\":\"we make the clouds rain\",\"url\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/author\\\/admin-2\\\/#author\",\"url\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/author\\\/admin-2\\\/\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#authorImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d370c5f82d8ed54ea2a14d7d135705e3add3103f5fa9b7670bd9dc6335a92e9e?s=96&d=mm&r=g\",\"width\":96,\"height\":96,\"caption\":\"admin\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#webpage\",\"url\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/\",\"name\":\"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud\",\"description\":\"Let's start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's\",\"inLanguage\":\"en-US\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/#website\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\\\/#breadcrumblist\"},\"author\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/author\\\/admin-2\\\/#author\"},\"creator\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/author\\\/admin-2\\\/#author\"},\"datePublished\":\"2011-06-17T10:56:55-05:00\",\"dateModified\":\"2016-03-21T02:49:15-05:00\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/\",\"name\":\"Migrate to Cloud\",\"description\":\"we make the clouds rain\",\"inLanguage\":\"en-US\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.migrate2cloud.com\\\/blog\\\/#organization\"}}]}\n\t\t<\/script>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud","description":"Let's start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's","canonical_url":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"BlogPosting","@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#blogposting","name":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud","headline":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04","author":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/author\/admin-2\/#author"},"publisher":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/#organization"},"datePublished":"2011-06-17T10:56:55-05:00","dateModified":"2016-03-21T02:49:15-05:00","inLanguage":"en-US","commentCount":1,"mainEntityOfPage":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#webpage"},"isPartOf":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#webpage"},"articleSection":"Amazon EC2, Cloud computing, amazon, Apache Hadoop, AWS EC2, client applications, Cloud computing, Cloud infrastructure, Cloudera, Company Technology, Company: APACHE SOFTWARE, Company: Google, Data-intensive computing, data-intensive distributed applications, file systems, Hadoop, Hadoop Cluster, HTML element, Java, Job Tracker, MapReduce, Parallel computing, Search Engines, software framework, Technology\/Internet, XML"},{"@type":"BreadcrumbList","@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#breadcrumblist","itemListElement":[{"@type":"ListItem","@id":"https:\/\/www.migrate2cloud.com\/blog#listItem","position":1,"name":"Home","item":"https:\/\/www.migrate2cloud.com\/blog","nextItem":{"@type":"ListItem","@id":"https:\/\/www.migrate2cloud.com\/blog\/category\/cloud-computing\/#listItem","name":"Cloud computing"}},{"@type":"ListItem","@id":"https:\/\/www.migrate2cloud.com\/blog\/category\/cloud-computing\/#listItem","position":2,"name":"Cloud computing","item":"https:\/\/www.migrate2cloud.com\/blog\/category\/cloud-computing\/","nextItem":{"@type":"ListItem","@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#listItem","name":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04"},"previousItem":{"@type":"ListItem","@id":"https:\/\/www.migrate2cloud.com\/blog#listItem","name":"Home"}},{"@type":"ListItem","@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#listItem","position":3,"name":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04","previousItem":{"@type":"ListItem","@id":"https:\/\/www.migrate2cloud.com\/blog\/category\/cloud-computing\/#listItem","name":"Cloud computing"}}]},{"@type":"Organization","@id":"https:\/\/www.migrate2cloud.com\/blog\/#organization","name":"Migrate to Cloud","description":"we make the clouds rain","url":"https:\/\/www.migrate2cloud.com\/blog\/"},{"@type":"Person","@id":"https:\/\/www.migrate2cloud.com\/blog\/author\/admin-2\/#author","url":"https:\/\/www.migrate2cloud.com\/blog\/author\/admin-2\/","name":"admin","image":{"@type":"ImageObject","@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#authorImage","url":"https:\/\/secure.gravatar.com\/avatar\/d370c5f82d8ed54ea2a14d7d135705e3add3103f5fa9b7670bd9dc6335a92e9e?s=96&d=mm&r=g","width":96,"height":96,"caption":"admin"}},{"@type":"WebPage","@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#webpage","url":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/","name":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud","description":"Let's start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's","inLanguage":"en-US","isPartOf":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/#website"},"breadcrumb":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/#breadcrumblist"},"author":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/author\/admin-2\/#author"},"creator":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/author\/admin-2\/#author"},"datePublished":"2011-06-17T10:56:55-05:00","dateModified":"2016-03-21T02:49:15-05:00"},{"@type":"WebSite","@id":"https:\/\/www.migrate2cloud.com\/blog\/#website","url":"https:\/\/www.migrate2cloud.com\/blog\/","name":"Migrate to Cloud","description":"we make the clouds rain","inLanguage":"en-US","publisher":{"@id":"https:\/\/www.migrate2cloud.com\/blog\/#organization"}}]},"og:locale":"en_US","og:site_name":"Migrate to Cloud - we make the clouds rain","og:type":"article","og:title":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud","og:description":"Let's start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's","og:url":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/","article:published_time":"2011-06-17T15:56:55+00:00","article:modified_time":"2016-03-21T07:49:15+00:00","twitter:card":"summary_large_image","twitter:title":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04 - Migrate to Cloud","twitter:description":"Let's start with a small introduction- what is hadoop ?. Hadoop is an open-source project administered by the Apache Software Foundation. Apache Hadoop is a Java software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Hadoop was inspired by Google's"},"aioseo_meta_data":{"post_id":"939","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"limit_modified_date":false,"created":"2022-12-27 09:16:42","updated":"2026-07-21 15:30:24","ai":null,"breadcrumb_settings":null,"seo_analyzer_scan_date":null},"aioseo_breadcrumb":"<div class=\"aioseo-breadcrumbs\"><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/www.migrate2cloud.com\/blog\" title=\"Home\">Home<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/www.migrate2cloud.com\/blog\/category\/cloud-computing\/\" title=\"Cloud computing\">Cloud computing<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">&raquo;<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\tHADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/www.migrate2cloud.com\/blog"},{"label":"Cloud computing","link":"https:\/\/www.migrate2cloud.com\/blog\/category\/cloud-computing\/"},{"label":"HADOOP Cluster on AWS EC2 with hadoop-0.20 and ubuntu-10.04","link":"https:\/\/www.migrate2cloud.com\/blog\/hadoop-cluster-with-hadoop-0-20-and-ubuntu-10-04\/"}],"jetpack_featured_media_url":"","jetpack-related-posts":[],"jetpack_sharing_enabled":true,"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/posts\/939","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/comments?post=939"}],"version-history":[{"count":13,"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/posts\/939\/revisions"}],"predecessor-version":[{"id":2150,"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/posts\/939\/revisions\/2150"}],"wp:attachment":[{"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/media?parent=939"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/categories?post=939"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.migrate2cloud.com\/blog\/wp-json\/wp\/v2\/tags?post=939"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}