Hadoop Installation(Single-Node)-2/3

-

Hadoop 1.0.4 on Ubuntu Linux 12.04 ( Single Node )- Part -2/3

Download Hadoop

  • Download a stable hadoop version from http://hadoop.apache.org/
  • I downloaded the this version - hadoop 1.0.4 and this is the link to it stable hadoop download page.
  • Download the file (hadoop-1.0.4.tar.gz) and copy it under the home directory of hadoop_usr ( ie. /home/hadoop_usr/)

Steps to Install

Login to terminal as hadoop_usr and extract the contents of the gz file
tar -xvf hadoop-1.0.4.tar.gz
We have to edit the following configuration file
  1. $HADOOP_HOME/conf/hadoop-env.sh
  2. $HADOOP_HOME/conf/hdfs-site.xml
  3. $HADOOP_HOME/conf/core-site.xml
  4. $HADOOP_HOME/conf/mapred-site.xml
  5. $HADOOP_HOME/conf/masters
  6. $HADOOP_HOME/conf/slaves

$HADOOP_HOME/conf/hadoop-env.sh

set the Java home

# The java implementation to use.Required.
# export JAVA_HOME=/usr/lib/j2sdk1.5-sun
export JAVA_HOME=/usr/lib/jvm/java-7-oracle

$HADOOP_HOME/conf/hdfs-site.xml

before editing this file create the following folder.
mkdir /home/hadoop_usr/hadoop-1.0.4/data
mkdir /home/hadoop_usr/hadoop-1.0.4/name
now edit the hdfs-site.xml and add the following lines 
<configuration>
  <property>
    <name>dfs.data.dir</name>
    <value>/home/hadoop_usr/hadoop-1.0.4/data</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>/home/hadoop_usr/hadoop-1.0.4/name</value>
  </property>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
</configuration>

$HADOOP_HOME/conf/core-site.xml

before editing this file. Create a tmp directory for hadoop's use. I created mine under the following directory. 
mkdir /home/hadoop_usr/hadoop-1.0.4/tmp
now edit the core-site.xml file and add the following values
<configuration>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>/home/hadoop_usr/hadoop-1.0.4/tmp/tmp-${user.name}</value>
        </property>
        <property>
                <name>fs.default.name</name>
                <value>hdfs://localhost:9000</value>
        </property>
</configuration>

$HADOOP_HOME/conf/mapred-site.xml

   <property>
     <name>fs.default.name< /name>
     <value>hdfs://localhost:9100</value>
   </property>
   <property>
     <name>mapred.job.tracker< /name>
     <value>localhost:9101< /value>
   </property>

$HADOOP_HOME/conf/masters

Edit this file and add the following line 
localhost

$HADOOP_HOME/conf/slaves

Edit this file and add the following line 
localhost

Comments

Post a Comment

Popular posts from this blog

Tableau - Accessing Tableau's DB

-
Image
Click here for other Tableau related Blogs Please note that TABLEAU does not advice anyone to access their DB. So if you are doing it you are on your own. Tableau has 2 DB Postgresql for managing all its reporting repositories FireBird for mapping repositories Tools Required pgAdmin 3 for PostgreSql Flamerobin for FireDB Accessing Tableau's PostgreSQL DB Tableau has 2 buitin userid tableau = with read-only access to some of Tableau's view tblwgadmin = admin user will all privileges Please note tblwgadmin is like a admin user, so if you are using this you have to be very careful. Or if you dont admin privs but just need enough access to read the tables then use just "tableau" user. Get access for "tableau" user.  First set a password for "tableau" userid open command prompt go to <Tableau_Home>\Tableau Server\7.0\bin run -  tabadmin dbpass <give a password> Restart the server Now open pgAdmin3 and from
Read more

react-bootstrap-table | header column alignment fix

-
Image
I am working with react-bootstrap-table. Its a great module with few teething problem. for example whenever I have a lots of columns (in the table) my header column is misaligned with the columns in the dataarea. example: (before the fix) after applying the fix ( note that now you have a scroll bar too) Reason for the problem: In react-bootstrap-table "header" is a table and the data is another table. Since they are 2 different tables sometimes the header column width gets misaligned from the column of data table. Fix-Logic applied: The idea is simple, First I allow the table to be rendered as it is  Then first adjust the width of the data table columns using the width of the respective header column width. But remember only for the columns whose width is lesser than the header respective column width.  Now get the width of all data-columns and set the same to the width of the respective header columns. and tadaaaa the table in perfect alignment now. S
Read more

Tableau : Convert ESRI shapes into Tableau Format

-
Image
Click here for other Tableau related Blogs Requirement is to convert data in ESRI Map shapes into Tableau format. Tools Required Quantum GIS -  http://www.qgis.org/en/site/ Python  Download the below Python Code -  https://github.com/venkat-vs-id/python import csv import re import sys #def convertESRI2TableauFormat( p_inputfile, p_outputfile): l_outfileName = 'c:/ESRI_suburb_tab_format.csv' # make changes here.. l_ESRIFileName ='c:/ESRI_suburb.csv' # make changes here.. rdx=0 csv.field_size_limit(sys.maxsize) outfile = open( l_outfileName, 'w', newline='') csvWriter = csv.writer( outfile, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL) with open( l_ESRIFileName, 'r') as csvfile: csvR = csv.reader( csvfile, delimiter=',', quotechar='"') for row in csvR: rdx += 1 ci=0 rowValues = [] rowValues.append(row[1]);# ssc_code rowValues.append(
Read more