hadoop集群安装

1.下载hadoop-2.6.0.tar.gz，进入/usr,然后解压hadoop的jar包，将解压后的目录名为hadoop,创建临时文件目录tmp

(1)cd /usr

(2)tar zxvf hadoop-2.6.0.tar.gz

(3)mv hadoop-2.6.0 hadoop

(4)mkdir tmp

(5)vi /etc/profile  

     # Set hadoop environment

     export HADOOP_HOME=/usr/hadoop

     export PATH=$PATH:$HADOOP_HOME/bin

(6)配置hosts,设置别名hadoop_master,hadoop_slave1,hadoop_slave2

    192.168.10.1 hadoop-master

    192.168.10.2 hadoop-slave1

    192.168.10.3  hadoop-slave2

 

2.配置core-site.xml

<configuration>

     <property>

         <name>hadoop.tmp.dir</name>

         <value>/usr/hadoop/tmp</value>

         <description>A base for other temporary directories.</description>

     </property>

 

     <property>

         <name>fs.defaultFS</name>

         <value>hdfs://hadoop-master</value>

     </property>

 

     <!-- file system properties -->

     <property>

         <name>fs.default.name</name>

         <value>hdfs://hadoop-master:9000</value>

     </property>

 </configuration>

 

3.配置hdsf-site.xml

 <configuration>

     <property>

         <name>dfs.replication</name>

         <value>2</value>

     </property>

 </configuration>

 

4.编辑/usr/hadoop/etc/hadoop/masters和/usr/hadoop/etc/hadoop/slaves

 (1)/usr/hadoop/etc/hadoop/masters增加一行:

    hadoop-master

 (2)/usr/hadoop/etc/hadoop/slaves增加：

    hadoop-slave1

    hadoop-slave2

 

 5.scp -r /usr/hadoop root@hadoop-slave1:/usr

 

 6.格式化

    hadoop namenode -format

DEPRECATED: Use of this script to execute hdfs command is deprecated.

Instead use the hdfs command for it.

......

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at cairewen-zt8h4.vclound.com/192.168.10.1

************************************************************/

 

 7.启动 start-dfs.sh ，报：localhost:Error: JAVA_HOME is not set.

 （1）cd /usr/hadoop/etc/hadoop，

 （2）然后将hadoop-env.sh中 export JAVA_HOME=$JAVA_HOME修改为：

     export JAVA_HOME=/etc/apps/svr/jdk1.7.0

 

 

 

 8.hadoop “util.NativeCodeLoader: Unable to load native-hadoop library for your platform”

    首先下载hadoop-native-64-2.4.0.tar：http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.4.0.tar

    如果你是hadoop2.6的可以下载下面这个：http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.6.0.tar

下载完以后，解压到hadoop的native目录下，覆盖原有文件即可。操作如下：

tar -x hadoop-native-64-2.4.0.tar -C /usr/hadoop/lib/native/