如何在vmware中搭建hadoop集群 如何在虚拟机中搭建hadoop
\u5982\u4f55\u642d\u5efahadoop\u96c6\u7fa4 \u6865\u63a5\u672c\u6587\u901a\u8fc7\u5728vmware workstation\u4e0a\u5efa\u7acb\u4e09\u53f0\u865a\u62df\u673a\u6765\u642d\u5efahadoop\u96c6\u7fa4\u73af\u5883\uff0c\u5176\u4e2d\u4e00\u53f0\u4f5c\u4e3anamenode\uff0c\u4e24\u53f0\u4f5c\u4e3adatanode\u3002
\u4e00\u3001\u5b89\u88c5vmware workstation\uff0810.0.0\uff09\uff08\u4e00\u8def\u4e0b\u4e00\u6b65\u5373\u53ef\uff09
\u4e8c\u3001\u521b\u5efa3\u4e2a\u865a\u62df\u673a
\u6bcf\u4e2a\u865a\u62df\u673a\u5b89\u88c5Ubuntu \u684c\u9762\u7248\uff08\u6211\u7528\u768411.10\uff0c\u4f46\u662f\u6709\u70b9\u8001\u4e86\uff0c\u5efa\u8bae\u752812.04\u6216\u66f4\u9ad8\u7248\u672c\uff09
namenode\u5efa\u8bae\u591a\u7ed9\u5199\u5185\u5b58\uff0c\u6211\u7684\u662fnamenode 1G\u5185\u5b58\uff0c20G\u786c\u76d8\uff0cdatanode 800M\u5185\u5b58\uff0c20G\u786c\u76d8\uff08\u6211\u7535\u8111\u5185\u5b588G\uff0c\u82e5\u5185\u5b58\u4e0d\u591f\u53ef\u4ee5\u5c11\u5206\u70b9\uff0c\u4f46\u662f\u53ef\u80fd\u6709\u4e9b\u5361\uff09\u4e0a\u7f51\u4e0a\u5f0f\u53ef\u8bbe\u7f6e\u4e3a\u6865\u63a5\u65b9\u5f0f\uff1a\u76f4\u63a5\u8fde\u63a5\u7269\u7406\u7f51\u7edc\u3002
\u4e09\u3001\u5b89\u88c5vmware tools\uff08\u4fbf\u4e8e\u4e3b\u673a\u548c\u865a\u62df\u673a\u76f4\u63a5\u53ef\u4ee5\u76f4\u63a5\u62d6\u62fd\uff0c\u590d\u5236\u7c98\u8d34\uff09
\u70b9\u51fbvmware \u4e0a \u865a\u62df\u673a-\u300b\u5b89\u88c5VMware tools\uff0c\u81ea\u52a8\u6253\u5f00\u4e00\u4e2a\u5305\u542bvmware\u5b89\u88c5\u5305\u7684\u4f4d\u7f6e\uff0c\u53ef\u4ee5\u62f7\u8d1d\u5230\u4f60\u6709\u6743\u9650\u7684\u5730\u65b9\uff0c\u4f8b\u5982/usr/src\u4e0b\u7528 sudo tar zxvf \u8f6f\u4ef6\u5305\u540d\u79f0 \u89e3\u538b\uff0c\u4e4b\u540e\u8fdb\u5165\u89e3\u538b\u540e\u7684\u6587\u4ef6\u5939\uff0c\u6267\u884c
sudo ./vmware-install.sh\u6765\u5b89\u88c5\uff0c\u8fc7\u7a0b\u4e2d\u9047\u89c1yes\u6216no\u4e00\u5f8benter\u952e\uff0c\u5b89\u88c5\u540e\u91cd\u542f\u751f\u6548\u3002
\u56db\u3001\u914d\u7f6e\u5171\u4eab\u76ee\u5f55\uff08\u8ba9\u4e3b\u673a\u548c\u865a\u62df\u673a\u76f4\u63a5\u65b9\u4fbf\u5b9e\u65f6\u5171\u4eab\uff09
\u5bf9\u6bcf\u4e2a\u865a\u62df\u673a\uff0c\u70b9\u51fbvmware\u4e0a\u865a\u62df\u673a-\u300b\u8bbe\u7f6e-\u300b\u9009\u9879\uff0c\u9009\u62e9\u5171\u4eab\u6587\u4ef6\u5939\uff0c\u6c38\u4e45\u5171\u4eab\uff0c\u6dfb\u52a0\u4e00\u4e2a\u672c\u673a\u7684\u76ee\u5f55\u6240\u4e3a\u5171\u4eab\u76ee\u5f55\uff0c\u5e76\u4e3a\u8fd9\u4e2a\u5171\u4eab\u8d77\u4e00\u4e2a\u540d\u5b57\uff0c\u4f8b\u5982\u53ebshare\u3002\u8fd9\u6837\u5728\u865a\u62df\u673a\u4e0a\u7684 /mnt/hgfs\u4e0b\u4f1a\u81ea\u52a8\u751f\u6210share\u6587\u4ef6\u5939\u4fdd\u6301\u4e0e\u4e3b\u673a\u4e0a\u7684\u76ee\u5f55\u5b9e\u65f6\u4e00\u81f4\u3002\u91cd\u542f\u751f\u6548\uff0c\u6709\u65f6\u751f\u6548\u8f83\u6162\uff0c\u591a\u91cd\u542f\u51e0\u6b21\u5c31\u884c\u4e86\uff0c\u4e0d\u77e5\u4f55\u6545\u3002
\u4e94\uff0c\u5b89\u88c5ssh\uff0c\u4e3a\u4e86\u4e00\u4f1a\u7684\u514d\u5bc6\u7801\u767b\u9646
\u5bf9\u6bcf\u53f0\u865a\u62df\u673a\u8fd0\u884c\uff1asudo apt-get install openssh-server
\u82e5\u4e0d\u6210\u529f\uff0c\u4f60\u53ef\u80fd\u9700\u8981\u66f4\u65b0\u4e00\u4e0b\u8d44\u6e90\u5217\u8868\u4e86\uff1asudo apt-get update
\u516d\u3001\u914d\u7f6essh\u65e0\u5bc6\u7801\u767b\u5f55\uff0c\u65b9\u4fbfnamenode\u7ed9datanode\u5206\u914d\u4efb\u52a1\u548c\u6c47\u603b\u4fe1\u606f
\u8bf4\u660e\uff1a\u4ee5\u4e0b\u751f\u6210\u7684\u79d8\u94a5\u9ed8\u8ba4\u4fdd\u5b58\u5728\u7528\u6237\u540d\u7a7a\u95f4\uff0c\u6240\u4ee5\u5efa\u8bae\u4e09\u53f0\u865a\u62df\u673a\u5efa\u7acb\u540c\u6837\u7684\u7528\u6237\u540d\uff0c\u4f8b\u5982hadoop\uff0c\u4e5f\u53ef\u4ee5\u90fd\u7528root\u7528\u6237\u3002
\u5728name node,\u548cdatanode\u90fd\u8fd0\u884c:
ssh-keygen \u2013t dsa -P '' \u2013f ~/.ssh/id_dsa \uff08\u5efa\u8bae\u624b\u6253\u6b64\u547d\u4ee4\uff0c\u590d\u5236\u6709\u65f6\u5019\u4e0d\u884c\uff0c\u6ce8\u610f\u90fd\u662f\u82f1\u6587\u5b57\u7b26\uff0c-P\u7684P\u662f\u5927\u5199\uff09
\u4e0b\u9762\u5c06\u4e09\u53f0\u865a\u62df\u673a\u7684\u516c\u94a5\u62f7\u8d1d\u5230\u5171\u4eab\u76ee\u5f55\u4e2d\u5e76\u91cd\u547d\u540d\u4ee5\u4f9b\u5f7c\u6b64\u914d\u7f6e\u4f7f\u7528
\u5728name node:
scp ~/.ssh/id_dsa.pub /mnt/hgfs/share/ id_dsa0.pub
\u5728data node 1:
scp ~/.ssh/id_dsa.pub /mnt/hgfs/share/ id_dsa1.pub
\u5728data node 2:
scp ~/.ssh/id_dsa.pub /mnt/hgfs/share/ id_dsa2.pub
\u5c06\u5171\u4eab\u76ee\u5f55\u4e2d\u516c\u94a5\u5206\u522b\u6dfb\u52a0\u5230\u4e09\u53f0\u865a\u62df\u673a\u7684\u6388\u6743\u4e0b\uff0c\u5206\u522b\u5728name node\uff0cdata node 1\u548cdata node 2\u6267\u884c:
cat /mnt/hgfs/share/id_dsa0.pub >> ~/.ssh/authorized_keys
cat /mnt/hgfs/share/id_dsa1.pub >> ~/.ssh/authorized_keys
cat /mnt/hgfs/share/id_dsa2.pub >> ~/.ssh/authorized_keys
\u6d4b\u8bd5\uff1a
ssh localhost
ssh \u76ee\u6807\u673a\u5668\u5730\u5740
\u4e03\uff0c\u5b89\u88c5jdk
\u5bf9\u6bcf\u4e2a\u865a\u62df\u673a\uff1a
\u4e0b\u8f7djdk(jdk1.6.0)\u5e76\u89e3\u538b\uff0c\u5047\u5b9a\u76ee\u5f55\u4e3a/usr/src/jdk1.6.0\uff08\u5982\u679c\u4f60\u7684\u4e0d\u662f\uff0c\u4e5f\u53ef\u4ee5\u6539\u6210\u662f\uff0c\u8fd9\u6837\u4e0b\u9762\u7684\u547d\u4ee4\u76f4\u63a5\u590d\u5236\u5c31\u884c\u4e86\uff09
\u8bbe\u7f6e\u73af\u5883\u53d8\u91cf\u3002
#vi /etc/profile
\u5728\u6700\u540e\u9762\u52a0\u5165
#set java environment
export JAVA_HOME=/usr/src/jdk1.6.0
export CLASSPATH=.:$JAVA_HOME/lib.tools.jar
export PATH=$JAVA_HOME/bin:$PATH
\u4fdd\u5b58\u9000\u51fa\u3002
\u8981\u4f7fJDK\u5728\u6240\u6709\u7684\u7528\u6237\u4e2d\u4f7f\u7528\uff0c\u53ef\u4ee5\u8fd9\u6837\uff1a
vi /etc/profile.d/java.sh
\u5728\u65b0\u7684java.sh\u4e2d\u8f93\u5165\u4ee5\u4e0b\u5185\u5bb9\uff1a
#set java environment
export JAVA_HOME=/usr/src/jdk1.6.0
export CLASSPATH=.:$JAVA_HOME/lib/tools.jar
export PATH=$JAVA_HOME/bin:$PATH
\u4fdd\u5b58\u9000\u51fa\uff0c\u7136\u540e\u7ed9java.sh\u5206\u914d\u6743\u9650\uff1achmod 755 /etc/profile.d/java.sh
\u91cd\u542f\u865a\u62df\u673a
\u516b\u3001\u5b89\u88c5hadoop
\u5728usr/src\u76ee\u5f55\u89e3\u538b\uff1a tar zxvf hadoop-0.20.2.tar.gz
\u628ahadoop\u7684\u5b89\u88c5\u8def\u5f84\u6dfb\u52a0\u5230/etc/profile\u4e2d\uff1a
export HADOOP_HOME=/usr/src/hadoop-0.20.2
export PATH=$HADOOP_HOME/bin:$PATH
\u4e5d\u3001\u914d\u7f6ehadoop
\u5728hadoop-0.20.1/conf\u76ee\u5f55\uff1a
\uff081\uff09\u5bf93\u4e2a\u865a\u62df\u673a\uff0c\u5728conf/hadoop-env.sh\u4e2d\u914d\u7f6ejava\u73af\u5883
vi hadoop-env.sh
\u5728hadoop-env.sh\u6587\u4ef6\u4e2d\u6dfb\u52a0
export JAVA_HOME=/usr/src/jdk1.6.0
(2) \u7528ifconfig\u547d\u4ee4\u67e5\u770bip\u5730\u5740\uff0c\u5047\u5b9aname node\u5185\u90e8\u5730\u5740\u4e3a192.168.204.157\uff0cdata node 1\u548cdata node 2\u5206\u522b\u4e3a192.168.204.162\uff0c192.168.204.173
\u5bf9name node\u914d\u7f6econf/masters\u548cconf/slaves:
masters : 192.168.204.157
slaves : 192.168.204.162
192.168.204.173
(3)\u5bf9name node, data node 1\u548cdata node2,\u914d\u7f6econf/core-site.xml, conf/hdfs-site.xml\u53caconf/mapred-site.xml\uff0c\u914d\u7f6e\u6587\u4ef6\u5185\u7684IP\u5730\u5740\u5747\u914d\u7f6e\u4e3aname node\u7684IP\u5730\u5740\uff08\u6bd4\u5982name node IP\u5730\u5740\u4e3a192.168.204.157\u3002
\u8bf4\u660e\uff1a\u56e0\u4e3a\u4e09\u53f0\u4e0a\u9762\u8fd9\u4e09\u4e2a\u914d\u7f6e\u4e00\u6837\uff0c\u6240\u4ee5\u914d\u7f6e\u4e00\u4efd\uff0c\u7136\u540e\u590d\u5236\u7c98\u8d34\u5373\u53ef\u3002
\uff08\u53c8\u8bf4\u660e\uff1a\u4e0b\u9762\u4e4b\u6240\u4ee5\u4e0a\u56fe\u7247\u662f\u56e0\u4e3a\uff0c\u65b0\u6d6a\u4f1a\u628a\u5b57\u7b26\u6309html\u91cd\u65b0\u7f16\u8bd1\uff0c\u65e0\u6cd5\u6b63\u5e38\u663e\u793a\uff0c\u65e0\u5948\u554a\uff09
\u5341\u3001\u8fd0\u884chadoop
\u5148\u628a\u4e09\u53f0\u865a\u62df\u673a\u7684\u9632\u706b\u5899\u5173\u4e86\uff1a sudo ufw disable
\u5bf9name node\uff1a
\u8fdb\u5165hadoop-0.20.2/bin\uff0c\u9996\u5148\u683c\u5f0f\u5316\u6587\u4ef6\u7cfb\u7edf\uff1a hadoop namenode \u2013format
\u5bf9data node1 \u548cdata node 2\uff1a
\u8fdb\u5165hadoop-0.20.2/bin\uff0c\u6267\u884c\uff1a hadoop datanode \u2013format
\u5bf9name node\uff1a\uff0c\u5728bin\u76ee\u5f55\u4e0b\u6267\u884c\uff1a
start-all.sh
hadoop dfsadmin \u2013safemode leave
hadoop dfsadmin \u2013report, \u67e5\u770b\u8282\u70b9\u60c5\u51b5\uff0c\u770b\u5230\u7c7b\u4f3c\u754c\u9762\u663e\u793aavailable\u7684Datanodes \u3002
\u7528jps\u547d\u4ee4\u67e5\u770b\u8fdb\u7a0b\uff0cNameNode\u4e0a\u7684\u7ed3\u679c\u4e3a\uff1a
26745 JobTracker
29398 jps
27664 NameNode
Data Node 1 \u7684\u7ed3\u679c\uff1a
6718 TaskTracker
6042 DataNode
6750 jps
Data Node 2 \u7684\u7ed3\u679c\uff1a
12173 TaskTracker
10760 DataNode
12700 jps
\u81f3\u6b64\uff0c\u5b89\u88c5\u6210\u529f\u3002
\u65b9\u6cd5/\u6b65\u9aa4
1
\u8bbe\u7f6e\u865a\u62df\u673a\u548c\u672c\u673a\u4e0a\u7684ip
2
\u53ef\u4ee5\u6e05\u695a\u7684\u770b\u5230\u591a\u51fa\u73b0\u4e86\u4e24\u4e2a\u7f51\u5361\u9009\u9879\uff0c\u8fd9\u91cc\u6211\u4eec\u4f7f\u7528VMware Network Adapter VMnet1 \u5c31\u884c\u4e86\uff0c\u7136\u540e\u5c06\u5176\u4e0e\u7684\u4e00\u4e2a\u7981\u7528\u3002\u542f\u7528\u7b2c\u4e00\u4e2a\u4e4b\u540e\uff0c\u5c31\u8981\u53bb\u7ed9\u8fd9\u4e2a\u8bbe\u7f6eip\u5730\u5740\u4e86\uff0c\u90a3\u4e48\u8fd9\u4e2aip\u5730\u5740\u5e94\u8be5\u8bbe\u7f6e\u4e3a\u591a\u5c11\u5462\uff1f\u8fd9\u4e2a\u53ef\u4ee5\u6839\u636e\u4f60\u81ea\u5df1\u7684\u559c\u597d\u81ea\u5df1\u53bb\u8bbe\u7f6e\u5c31\u597d\u4e86\uff0c\u53ea\u8981\u662f\u5728\u6709\u6548\u7684\u7f51\u5173\u4e4b\u5185\u90fd\u884c
\u4e0b\u9762\u662f\u6211\u8bbe\u7f6e\u7684ip\u5730\u5740
\u8fd9\u6837\u8bbe\u7f6e\u597d\u4e86\u672c\u673a\u7684ip\u5730\u5740\u4e4b\u540e\u5462\uff0c\u5c31\u5e94\u8be5\u53bb\u8bbe\u7f6e\u865a\u62df\u673a\u7684ip\u5730\u5740\u4e86\uff0c\u9996\u5148\u8981\u4fdd\u8bc1\u865a\u62df\u673a\u88ab\u672c\u673a\u7684\u901a\u8baf\u65b9\u5f0f\u4e3ahost-only\u6a21\u5f0f\uff0c\u5373\uff0c\u4ec5\u4e3b\u673a\u6a21\u5f0f\u3001
\u6253\u5f00VmWare\u4e4b\u540e\u5462\uff0c\u53bb\u8fd9\u91cc\u8bbe\u7f6e\u4ec5\u4e3b\u673a\u6a21\u5f0f
-------------------------------------------------------
\u8fd9\u91cc\u8981\u6ce8\u610f\u4e00\u70b9\uff0c\u9996\u5148\u4e00\u4e2a\u662f\u865a\u62df\u673a\u7684ip\u5730\u5740\u4e3a192.168.80.100\u548c\u4e00\u4e2a\u662f\u6211\u4eec\u81ea\u5df1\u7535\u8111\u7684\u4e0a\u8bbe\u7f6e\u7684Vmvare Network Adapter Vmnet1\u7684ip\u5730\u5740\u4e3a192.3168.80.1\u3002\u8bbe\u7f6e\u597d\u4e86\u4e4b\u540e\u6253\u5f00\u865a\u62df\u673a\u7684\u5c0f\u9ed1\u5c4f\u3002
END
liniux\u7cfb\u7edf\u914d\u7f6e
1
2
\u7136\u540e\u9009\u62e9\u7b2c\u56db\u9879\u6253\u5f00\u5c0f\u9ed1\u5c4f\u3002
\u6253\u5f00\u5c0f\u9ed1\u5c4f\u4e4b\u540e\u4e00\u5b9a\u8981\u8bb0\u5f97\u8ba9\u7f51\u5361\u91cd\u65b0\u542f\u52a8\u4e00\u4e0b\uff0c\u6267\u884c\u547d\u4ee4
Service network restart \u6765\u91cd\u542f\u7f51\u5361
END
\u4fee\u6539\u4e3b\u673a\u540d
1
\u4e0b\u4e00\u6b65\uff0c\u4fee\u6539\u4e3b\u673a\u540d\uff0c\u4e3a\u4ec0\u4e48\u8981\u4fee\u6539linux\u865a\u62df\u673a\u7684\u4e3b\u673a\u540d\u5462\uff1f\u4e3b\u8981\u8fd8\u662f\u65b9\u4fbf\u4ee5\u540e\u6211\u4eec\u8bbf\u95ee\u3002
\u73b0\u5728\u6211\u4eec\u53ef\u4ee5\u5728\u672c\u673a\u4e0a\u9762\u4f7f\u7528PieTTY\u8fd9\u4e2a\u5de5\u5177\u6765\u8fde\u63a5\u6211\u4eec\u7684\u865a\u62df\u673a\u4e86\uff0c\u8fd9\u6837\u6211\u4eec\u5c31\u4e0d\u7528\u53bb\u865a\u62df\u673a\u7684\u5c0f\u9ed1\u5c4f\u4e0b\u9762\u64cd\u4f5c\u4e86\uff0c\u592a\u65b9\u4fbf\u4e86\u3002
2
\u7136\u540e\u8fdb\u8fdb\u5165\u4e86network\u8fd9\u4e2a\u6587\u4ef6\u7684\u66f4\u6539\u64cd\u4f5c\u4e2d\u53bb\u4e86
Vi\u8fdb\u5165\u8be5\u6587\u4ef6\u4e4b\u540e\u9ed8\u8ba4\u662f\u4e0d\u53ef\u4fee\u6539\u7684\uff0c\u53ea\u53ef\u4ee5\u770b\uff0c\u4e0d\u53ef\u4ee5\u4fee\u6539\uff0c\u90a3\u4e48\u6309\u4e0b\u952e\u76d8\u4e0a\u9762\u7684i\u5b57\u6bcd\uff0c\u5c31\u53d8\u6210\u53ef\u4ee5\u4fee\u6539\u7684\u72b6\u6001\u4e86
3
\u7136\u540e\u6dfb\u52a0\u4e00\u884c\uff0cHOSTNAME=hadoop\uff0c\u7136\u540e\u70b9\u51fb\u952e\u76d8\u53f3\u4e0a\u89d2Esc\u952e\uff0c\u7136\u540e\u5728\u82f1\u6587\u8f93\u5165\u6cd5\u4e0b\u9762\u8f93\u5165
4
\u63a5\u7740\u8f93\u5165\uff1awq \u7136\u540e\u56de\u8f66\uff0c\u4fdd\u5b58\u4fee\u6539\u3002\u8fd9\u6837\uff0c\u5c31\u6c38\u8fdc\u7684\u628a\u4e3b\u673a\u540d\u7ed9\u6539\u6210hadoop\u4e86\uff0c\u4f7f\u7528hostname\u6765\u9a8c\u8bc1\u4e3b\u673a\u540d
END
\u4e3b\u673a\u540d\u548cip\u5730\u5740\u8fdb\u884c\u7ed1\u5b9a
1
\u9996\u5148\u4e3a\u4ec0\u4e48\u8981\u8ba9ip\u5730\u5740\u548c\u4e3b\u673a\u540d\u6765\u8fdb\u884c\u7ed1\u5b9a\u5462\uff0c\u72e0\u7b80\u5355\uff0c\u6bcf\u4e00\u53f0\u7535\u8111\u5728\u7f51\u7edc\u4e2d\u7684\u552f\u4e00\u6807\u8bc6\u5c31\u662f\u672c\u7535\u8111\u7684ip\u5730\u5740\uff0c\u9996\u5148\u53ef\u4ee5\u786e\u5b9aip\u5730\u5740\u662f\u6c38\u8fdc\u4e0d\u4f1a\u91cd\u590d\u7684\uff0c\u90a3\u4e48\u6211\u4eec\u6bcf\u6b21\u8bbf\u95ee\u7684\u767e\u5ea6\u9996\u9875\u5176\u5b9e\u5c31\u662f\u5728\u8bbf\u95ee\u767e\u5ea6\u4e3b\u673a\u7684ip\u5730\u5740\uff0c\u5982\u679c\u4f60\u77e5\u9053\u767e\u5ea6\u4e3b\u673a\u7684ip\u5730\u5740\uff0c\u76f4\u63a5\u5728\u7f51\u5740\u680f\u91cc\u8f93\u5165ip\u5730\u5740\uff0c\u4e00\u6837\u7684\u4f1a\u8bbf\u95ee\u767e\u5ea6\u7684\u9996\u9875\u3002
\u65e2\u7136\u5982\u6b64\uff0c\u6211\u4eec\u4ee5\u540e\u76f4\u63a5\u7528ip\u6765\u8bbf\u95ee\u7f51\u9875\u5c31\u597d\u4e86\uff0c\u5e72\u561b\u8fd8\u8981\u7528www.baidu.com\u5462\uff1f\u9996\u5148\uff0c\u7528ip\u6765\u8bbf\u95ee\u4e5f\u592a\u9ebb\u70e6\u4e86\uff0c\u5e76\u4e14\u4e5f\u4e0d\u4fbf\u4e8e\u8bb0\u5fc6\uff0c\u6240\u4ee5\u6211\u4eec\u5c06\u6211\u4eec\u7684\u4e3b\u673a\u540d\u8ddf\u6211\u4eec\u7684ip\u5730\u5740\u7ed1\u5b9a\u5230\u4e00\u8d77\uff0c\u4ee5\u540e\u76f4\u63a5\u8bbf\u95ee\u6211\u4eec\u7684\u4e3b\u673a\u540d\u5c31\u662f\u76f8\u5f53\u4e8e\u8bbf\u95ee\u6211\u4eec\u7684ip\u5730\u5740\u4e86\u3002
\u7136\u540e\u6765\u8fdb\u884c\u4e3b\u673a\u540d\u548cip\u5730\u5740\u8fdb\u884c\u7ed1\u5b9a\u7684\u64cd\u4f5c\u5427\u3002
\u8fd8\u662f
2
\u5173\u95ed\u865a\u62df\u673a\u7684\u9632\u706b\u5899
\u76f4\u63a5\u6267\u884cservice iptables stop\u547d\u4ee4\u6765\u5173\u95ed\u9632\u706b\u5899
3
\u5230\u8fd9\u4e00\u6b65\uff0chadoop\u8bbe\u7f6e\u5df2\u5b8c\u6bd5\uff0c\u4e0b\u4e00\u7bc7\uff0c\u5c06\u4f1a\u8be6\u7ec6\u4ecb\u7ecd\u5982\u4f55\u8bbe\u7f6ehadoop\u6e90\u7801\u3002\u3002\u3002\u656c\u8bf7\u671f\u5f85\u5982\u4f55\u642d\u5efahadoop\u5206\u5e03\u5f0f\u73af\u5883\uff0c\u600e\u6837\u5feb\u901f\u642d\u5efa
hadoop俗称分布式计算，最早作为一个开源项目，最初只是来源于谷歌的两份白皮书。然而正如十年前的Linux一样，虽然Hadoop最初十分简单，但随着近些年来大数据的兴起，其也获得了一个充分体现价值的舞台。这也正是业内普遍将Hadoop看做是下一个Linux的原因。
本文介绍基于多台vmware虚拟机来安装hadoop集群的过程及方法，通过这个小的集群让您在本地电脑上就可以研究hadoop的相关工作过程，有人会有疑问在小的虚拟机集群上研究的结果， 写的程序能否在大集群上工作正常？可以肯定的没问题的。
Hadoop的一个特性是线性增长特性，即当前数量情况下，处理时间是1， 若是数据量加倍， 后处理时间加倍， 若是在这种情况下， 处理能力也加倍则处理时间也是1.
正常情况hadoop需要较多的服务器才能搭建， 但是我们在家里学习如何去找那么服务器， 解决办法可以找几台pc机，在pc机上安装linux系统就可以了。
当然我们还有更简单的办法， 就是找一台高性能的电脑， 在电脑上安装虚拟机软件，里面创建若干台虚拟机， 然后让这些虚拟机构成一个小的内部局域网络，在这个网络上我们安装linux软件， java软件， 安装hadoop程序，我们就可以创建一个简单hadoop的研究系统， 进行软件的开发调试， 在这个小的分布式集群上开发的程序可以无缝移植到相同版本的hadoop（不同hadoop的版本兼容性不是很好， 特别是低版本同高版本， 他们的api也有些许变化）集群中。
绛旓細铏氭嫙鏈洪噷闈㈢殑绯荤粺涓嶈璁剧疆 IP 绛夌瓑鐨勪笢瑗匡紝鐩存帴 DHCP 锛堟垨鑰呭彨鍋氳嚜鍔ㄨ幏鍙栵級鍗冲彲銆俈Mware 铏氭嫙鍑烘潵鐨勫眬鍩熺綉浼氳嚜鍔ㄦ彁渚 DHCP 鏈嶅姟鏉ョ粰铏氭嫙鏈鸿缃紒鑷繁璁剧疆闄ら潪浣犲畬鍏ㄦ槑鐧藉眬鍩熺綉鐨勮法缃戠粶妗ユ帴銆丯AT 璁剧疆锛屼笉鐒朵綘缁濆璁剧疆涓嶅嚭鏉ヨ兘鐢ㄧ殑璁剧疆銆
绛旓細铏氭嫙鏈鸿蒋浠惰嚜宸卞畨瑁呬竴涓猇Mware Workstation锛岀劧鍚庝笅杞藉皬绾㈠附鐨勯暅鍍忔枃浠讹紝鍦╒Mware閲岄潰鏂板缓涓涓皬绾㈠附鐨勮櫄鎷熸満锛屽姞杞藉ソ灏忕孩甯界殑闀滃儚鏂囦欢锛屾墦寮VMware鐨勭數婧愰敭锛岀劧鍚庢牴鎹彁绀哄畨瑁呭ソ灏卞彲浠ャ
绛旓細鍦ㄥぇ闀滃儚鏂囦欢涓壘鍒扳榖oot.iso鈥欓暅鍍忔枃浠跺鍒跺嚭鏉ワ紝鍦╒Mware涓瀹夎鏃跺厛杩愯闀滃儚鏂囦欢鈥榖oot.iso鈥欙紝鍦ㄨ瑷閫夐」鐣岄潰閫夋嫨鈥樼畝浣撲腑鏂団欙紝鍦ㄩ敭鐩橀夐」閫夋嫨缇庡紡閿洏锛坲s锛,鏈鍚庢寜鎺ヤ笅鏉ョ殑姝ラ瀹夎绗簩寮犲ぇ闀滃儚鏂囦欢銆傚鏋滃湪鐪熷疄鏈轰腑灏卞彧瑕佹妸绗簩寮犲ぇ闀滃儚鏂囦欢鍒绘垚鍏夌洏鐩存帴瀹夎灏卞彲浠ヤ簡銆傚弬鑰冭祫鏂欙細銆奟ed Hat Enterpr...
绛旓細寮鏈---Del(鏈夌殑鏄疐2锛孎9锛孎10锛屼笉鐭ラ亾鍙戝笘瀛愶級--- 鐪嬪埌 *** Advance ---enter杩涘幓---鐪嬪埌Virtualization 鎴栬匳M鎴栬匳MT浠涔堢殑銆傛敼鎴恊nabled锛岄鍑築IOS鏄寰楄閫塖ave & Exit.darwin300.iso---25,446k darwin_snow.iso---17,768k Rebel EFI.iso---7,748k darwin_snow_legacy.iso 瑙ｅ帇...
绛旓細vagrant鍙互鍦╒irtualBox, VMware涓婇潰杩愯(docker鏃犳硶鎵ц)銆倂agrant涔熷彲浠ュ湪AWS, OpenStack杩欎簺浜戠幆澧冧腑杩愯銆傚嵆渚夸綘浣跨敤浜哾ocker鐨勫鍣,vagrant鍚屾牱涔熸病闂銆倂agrant鍙互鑷姩install, pull down, build, run Docker containers銆 姣斿鍦╲agrant V1.6鐗堟湰涓,vagrant闆嗘垚浜哾ocker-based development environments,鍥犳Vagrant...
绛旓細鍦╞ois閲岄潰寮鍚櫄鎷熷寲锛屽啀鍘昏繙鏅鍧涚埇妤硷紒锛佸鏈変粈涔堜笉娓呮鐨勶紝鍙互HI鎴戞垨QQ:1847539鎴7240044
绛旓細1 鎵撳紑VM 渚濇鍗曟満FILE - NEW - VIRTUAL MACHINE鑿滃崟 鎴栬呭崟鏈轰富椤甸潰涓婄殑 鏂板缓铏氭嫙鏈 鍥捐〃 寮瑰嚭鏂板缓铏氭嫙鏈虹殑閰嶇疆鍚戝瀵硅瘽妗 2 鍗曟満"涓嬩竴姝" 鎸夐挳 杩涘叆铏氭嫙鏈洪厤缃璇濇 閫夋嫨鍏稿瀷 鎴戞槸璇村湪娌℃湁鐗规畩璁惧鎴栬呴厤缃殑鎯呭喌涓 3 缁х画 涓嬩竴姝 杩涘叆 閫夋嫨瀹㈡満鎿嶄綔绯荤粺 瀵硅瘽妗 浣犳牴鎹疄闄呴渶瑕佸湪 瀹㈡満...
绛旓細浣犵殑rhel鏄64浣嶇殑鍚э紝cpu鏀寔64浣嶆寚鐨勬槸鐢佃剳瀹為檯瀹夎鐨勭郴缁熸槸64浣嶏紝濡傛灉鎯宠vmware涔熸敮鎸佸畨瑁64浣嶇殑璇漜pu涓嶄粎鏀寔64浣嶏紝杩樿鏀寔铏氭嫙鍖栨妧鏈備綘鐢╟pu-z鐪嬩竴涓嬪氨鐭ラ亾浣燾pu鏀笉鏀寔浜嗐傚鍥撅紝閭ｄ釜鎸囦护闆嗛偅閲屾湁涓涓猇T鐨勫氨鏄櫄鎷熷寲鎶鏈傚鏋滀笉鏀寔灏卞彧鏈夎32浣嶇殑锛屼笉杩囧ソ鍍忔渶鏂扮増鐨剉mware鍙互瑁64浣嶏紝...
绛旓細浣跨敤鑺辩敓澹筹紝涓撲笟鍋氬唴缃戠┛閫忋佺鍙ｆ槧灏勭殑宸ュ叿锛屼笉闇瑕佸璺敱鍣ㄥ啀杩涜璁剧疆浠涔堢殑锛屾墍浠ヤ篃涓嶅鏄撳嚭閿欍
绛旓細鍦╒mware鐨勬枃浠惰彍鍗曢噷閫夋墦寮椤癸紝鎵惧埌骞舵墦寮浣犲鍒剁殑鍒汉鐨勯暅鍍忔枃浠讹紝涔熷氨鏄墿灞曞悕鏄痸mx鎴杤mtm鐨勬枃浠跺氨鍙互浜嗐傛垨鑰呭湪home鏍囩閲岄夆滄墦寮宸插瓨鍦ㄧ殑铏氭嫙鏈烘垨缁勨濋夋枃浠讹紝缁撴灉鏄竴鏍风殑銆
