Hadoop安装
创建hadoop用户1、sudo useradd -m hadoop
2、sudo passwd hadoop
3、sudo adduser hadoop sudo
4、虚拟机：最后注销当前用户（点击屏幕右上角的齿轮，选择注销），返回登陆界面。在登陆界面中选择刚创建的hadoop 用户进行登陆。
5、重新登录检查当前用户是谁：whoami 出现hadoop 成功
6、更新aptsudo apt-get update更新apt3.安装SSH，
7、配置免密登陆sudo apt-get install sshsudo apt-get install pdsh 安装ssh
8、安装Java环境sudo apt install openjdk-8-jdk 安装java
9、vim ~/.bashrc 进入之后按I 然后输入export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
10、按esc 然后按:wq! 回车 完成 此步骤改变环境变量source ~/.bashrc输入java -version检验
11、安装Hadoopwget https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-2.9.2/hadoop-2.9.2.tar.gz 安装命令
12、sudo tar -zxf ~/hadoop-2.9.2.tar.gz -C /usr/local 解压命令
13、cd /usr/local/ #进入用户安装目录
14、sudo mv ./hadoop-2.9.2/ ./hadoopsudo chown -R hadoop ./hadoop # 修改文件权限（hadoop-2.9.2默认拥有者是root，这里我们让hadoop也成为拥有者）
15、cd /usr/local/hadoop./bin/hadoop version # 查看hadoop版本信息，成功显示则安装成功
16、cd /usr/local/hadoop进入目录
17、mkdir ./input创建目录
18、ls查看当前目录
19、cp ./etc/hadoop/*.xml ./input
20、./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar grep ./input ./output 'dfs[a-z.]+'
21、cat ./output/*查看
22、rm -r ./output删除output文件
23、 cd /usr
24、 cd local
25、ls
26、 cd hadoop
27、 ls
28、cd /usr/local/hadoop/etc/hadoop/
29、Ls
30、vi hdfs-site.xml
31、cd /usr/local/hadoop
绛旓細瀹夎 Hadoop锛氬湪浣犵殑璁＄畻鏈轰笂瀹夎 Hadoop銆傚畨瑁 Python锛氳纭繚浣犵殑璁＄畻鏈轰笂宸茬粡瀹夎浜 Python銆傞厤缃 Hadoop 鐜锛氱紪杈 Hadoop 鐨勯厤缃枃浠讹紝浠ョ‘淇 Hadoop 鍙互涓 Python 閰嶅悎浣跨敤銆傚畨瑁呯浉鍏虫ā鍧楋細璇峰畨瑁呮墍闇鐨 Python 妯″潡锛屼互渚垮湪 Hadoop 鐜涓嬩娇鐢 Python銆傛祴璇 Python 瀹夎锛氳杩愯涓浜涙祴璇曡剼鏈紝浠...
绛旓細鍏瀹夎hadoop 鍦╱sr/src鐩綍瑙ｅ帇锛 tar zxvf hadoop-0.20.2.tar.gz 鎶奾adoop鐨勫畨瑁呰矾寰勬坊鍔犲埌/etc/profile涓細export HADOOP_HOME=/usr/src/hadoop-0.20.2 export PATH=$HADOOP_HOME/bin:$PATH 涔濄侀厤缃甴adoop 鍦╤adoop-0.20.1/conf鐩綍锛氾紙1锛夊3涓櫄鎷熸満锛屽湪conf/hadoop-env.sh涓厤缃...
绛旓細楠岃瘉Hadoop鏄惁瀹夎鎴愬姛涓昏閫氳繃浠ヤ笅涓や釜缃戝潃銆傚鏋滈兘鑳芥煡鐪嬶紝璇存槑瀹夎鎴愬姛銆備竴锛 鏌ョ湅HDFS鏄惁姝ｅ父鍚姩銆傚湪娴忚鍣ㄤ腑杈撳叆 姝ら〉闈负Hadoop HDFS绠＄悊鐨勭晫闈傜偣鍑籐ive Nodes 杩涘叆涓嬪浘銆備笂鍥捐〃绀猴紝娲诲姩鐨勮妭鐐逛负1涓 锛圠ive Datanodes:1锛 鑷虫鍙互纭畾HDFS宸茬粡姝ｅ父鍚姩浜嗐備簩銆佹煡鐪婱apReduce鐨凧obTracker鏄惁姝ｅ父...
绛旓細鐧婚檰hdfs鐨剋eb鐣岄潰(榛樿绔彛50070)锛屽鏋滅晫闈㈡樉绀簄amenode姝ｅ父锛宒atanode涔熷湪绾夸笂锛屽閲忎篃姝ｅ父锛屽垯璇存槑瀹夎鎴愬姛锛涘線hdfs閲岄潰鍐欏叆鏂囦欢锛屽鏋滃啓鍏ユ垚鍔燂紝鍒欏畨瑁呮垚鍔燂紱鍚姩yarn锛屽鏋滃惎鍔ㄦ垚鍔
绛旓細杞浇 Hdoop鐨勮崘閮ㄧ讲鐜鏄痩inux锛屼絾鏄垜浠兂瑕佸湪windows浣撻獙涓涓嬭繕鏄彲浠ョ殑锛屽挨鍏舵槸瀵规垜杩欐牱涓嶆槸缁忓父浣跨敤Linux绯荤粺鐨勪汉銆俉indows涓瀹夎閰嶇疆Hadoop鐨勬暀绋嬭繕鏄尯澶氱殑锛岃繖閲屽彲浠ヤ笅杞姐備笅闈㈡垜璇磋瀹夎閰嶇疆杩囩▼涓亣鍒扮殑涓浜涢棶棰樺惂锛岀粰鑷繁浠ュ悗鎻愪緵鍊熼壌銆傛垜鐨勬搷浣滅郴缁熸槸Windows 7锛屼娇鐢ㄧ殑Cygwin鐗堟湰鏄1.7.7-1锛...
绛旓細1.瑙ｅ帇hadoop瀹夎鍖鍒版寚瀹氱洰褰(/home/es) tar -zxvf hadoop-2.7.0.tar.gz -C /home/es 2.鍒涘缓绋嬪簭杞繛鎺 cd /home/es/ ln -s hadoop-2.7.0 hadoop 3.鍒涘缓鏁版嵁瀛樻斁鐩綍 mkdir /home/data/hadoop mkdir /home/data/hadoop/tmp mkdir /home/data/hadoop/dfs mkdir /home/data/hadoop/dfs/data mkdir ...
绛旓細鍦╳indows涓嬪缓浜嗕竴涓洰褰旻:/myHadoop 鎶奵entos hadoop瀹夎鏍圭洰褰曚笅鐨刯ar鍖呮嫹杩囨潵銆4銆佹柊寤哄伐绋嬶細Map/Reduce Project 閰嶇疆hadoop location:5銆佸埛鏂癉FS Locations 鍒楀嚭HDFS鏈嶅姟鍣ㄤ笂鐨勪笂鐨勭洰褰曪細6銆侀棶棰橈細鍦‥clipse涓婂彲浠ュ垪鍑篽adoop闆嗙兢涓婄殑鐩綍锛屼絾涓嶈兘鍒涘缓鏂囦欢锛屼笂浼犳枃浠 鍘熷洜锛歸indows涓婨clipse瀹夎浜唄adoop...
绛旓細浣犵湅鐨勬暀绋嬫槸鏃х殑锛屾柊鐗堢殑hadoop鍚姩鑴氭湰鏀惧湪sbin涓嬨俿tart-all.sh宸茬粡閫愭笎琚簾寮冿紝閲囩敤鏂扮殑鍚姩鑴氭湰锛歴bin/hadoop-daemon.sh --script hdfs start datanodesbin/hadoop-daemon.sh --script hdfs start namenodesbin/yarn-daemon.sh start resourcemanagersbin/yarn-daemon.sh start proxyserversbin/...
绛旓細#缂栬緫/etc/profilesudo gedit /etc/profile#浠ヤ笂宸茬粡娣诲姞杩噅ava鐨勭幆澧冨彉閲,鍦ㄥ悗杈规坊鍔犲氨鍙互#hadoop export HADOOP_HOME=/opt/Hadoop/hadoop-2.7.2 export PATH=$PATH:$HADOOP_HOME/sbin export PATH=$PATH:$HADOOP_HOME/bin 鎵ц source /etc/profile 浣挎枃浠剁敓鏁堛 2)鍚姩hadoop,杩涘叆hadoop瀹夎鐩綍 bin/hdfs ...
绛旓細1.鎻掍欢瀹夎 鍦$HADOOP_HOME/contrib/eclipse-plugin/鏂囦欢澶逛腑鏈変釜hadoop-eclipse-plugin-0.20.203.0.jar锛屾妸杩欎釜鏂囦欢澶嶅埗鍒癳clipse/plugins锛堣繖鏄痚clipse鐨勬彃浠剁洰褰曪級涓嬮潰鍗冲彲銆傛敞鎰忓湪鐩存帴澶嶅埗鏃朵細鍑洪敊锛屼富瑕佹槸缂哄皯jar鍖呫傝В鍐虫柟娉曪細锛1锛夊皢commons-httpclient-3.0.1.jar,jackson-core-asl-1.0.1.jar...
