原始
鏈接：
我所做的更動(dòng)
1： 用的是ubuntu 14.04
2: 用的是server版
3： 文中第二次
$ sudo gedit yarn-site.xml 的時(shí)候，應該是
$ sudo gedit mapred-site.xml
4：這個(gè)命令，存在一個(gè)問(wèn)題， 就是id_dsa.pub是不存在的。應該為id_rsa.pub
sudo ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop （這個(gè)hadoop是我的用戶(hù)名）
5: 當 $sudo ssh hadoopmaster的時(shí)候，被拒絕。 原因如下：
By default, SSH server denies password-based login for root. In /etc/ssh/sshd_config, change:
PermitRootLogin without-password
to
PermitRootLogin yes
And restart SSH:
sudo service ssh restart
為了簡(jiǎn)單省事兒。我這里測試 ssh的時(shí)候，沒(méi)有加sudo.
***********************************下面是正文*******************************
Namenode > hadoopmaster > 192.168.23.132
Datanodes > hadoopslave1 > 192.168.23.133
hadoopslave2 > 192.168.23.134
hadoopslave3 > 192.168.23.135
Clone
Hadoop Single node cluster as hadoopmaster
Hadoopmaster Node $ sudo gedit /etc/hosts
hadoopmaster 192.168.23.132
hadoopslave1 192.168.23.133
hadoopslave2 192.168.23.134
hadoopslave3 192.168.23.135
$ sudo gedit /etc/hostname
hadoopmaster
$ cd /usr/local/hadoop/etc/hadoop
$ sudo gedit core-site.xml
replace localhost as hadoopmaster
$ sudo gedit hdfs-site.xml
replace value 1 as 3
(represents no of datanode)
$ sudo gedit yarn-site.xml
add the following configuration <configuration>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>hadoopmaster:8025</value>
</property>
<property> <name>yarn.resourcemanager.scheduler.address</name>
<value>hadoopmaster:8030</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>hadoopmaster:8050</value>
</property>
</configuration>
$ sudo gedit
mapred-site.xml
replace mapreduce.framework.name
as mapred
.job.tracker
replace yarn
as hadoopmaster:54311
$ sudo rm -rf /usr/local/hadoop/hadoop_data
Shutdown hadoopmaster node
Clone Hadoopmaster Node as hadoopslave1, hadoopslave2, hadoopslave3
Hadoopslave Node (conf should be done on each slavenode)
$ sudo gedit /etc/hostname
hadoopslave<
nodenumberhere>
$ sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/datanode
$ sudo chown -R chaalpritam:chaalpritam /usr/local/hadoop
$ sudo gedit /usr/local/hadoop/etc/hadoop/hdfs-site.xml
remove dfs.namenode.name.dir
property section
reboot all nodesHadoopmaster Node
$ sudo gedit /usr/local/hadoop/etc/hadoop/masters
hadoopmaster
$ sudo gedit /usr/local/hadoop/etc/hadoop/slaves
remove localhost
and add hadoopslave1
hadoopslave2
hadoopslave3
$ sudo gedit /usr/local/hadoop/etc/hadoop/hdfs-site.xml
remove dfs.datanode.data.dir
property section
$ sudo mkdir -p /usr/local/hadoop/hadoop_data/hdfs/namenode
$ sudo chown -R chaalpritam:chaalpritam /usr/local/hadoop
$ sudo ssh-copy-id -i ~/.ssh/id_dsa.pub chaalpritam@hadoopmaster
$ sudo ssh-copy-id -i ~/.ssh/id_dsa.pub chaalpritam@hadoopslave1
$ sudo ssh-copy-id -i ~/.ssh/id_dsa.pub chaalpritam@hadoopslave2
$ sudo ssh-copy-id -i ~/.ssh/id_dsa.pub chaalpritam@hadoopslave3
$ sudo ssh hadoopmaster
$ exit
$ sudo ssh hadoopslave1
$ exit
$ sudo ssh hadoopslave2
$ exit
$ sudo ssh hadoopslave3
$ exit
$ hadoop namenode -format
$ start-all.sh
$ jps
(check in all 3 datanodes)
http://hadoopmaster:8088/
http://hadoopmaster:50070/
http://hadoopmaster:50090/
http://hadoopmaster:50075/
**********************************************************************************************這里遇到了2個(gè)問(wèn)題
1： hadoopmaster上， hstart沒(méi)能啟動(dòng) namenode。 通過(guò)單獨啟動(dòng)namenode，發(fā)現是權限問(wèn)題。來(lái)源于運行了$sudo rm -rf /usr/local/hadoop/hadoop_data 之后。
hadoop_data沒(méi)有了。但是master還是需要的
2: 另外也可以在Master節點(diǎn)上通過(guò)命令bin/hdfs dfsadmin -report查看DataNode是否正常啟動(dòng)。例如我這邊一共有1個(gè)Datanodes。
運行發(fā)現，一個(gè)都沒(méi)有，哎，哈哈
