L
L
LearnJava
Search…
Hadoop简介

一、简介

Hadoop是一个开源的分布式计算平台,用于存储大数据,并使用MapReduce来处理。Hadoop擅长于存储各种格式的庞大的数据,任意的格式甚至非结构化的处理。两个核心:
  • HDFS:Hadoop分布式文件系统(Hadoop Distributed File System),具有高容错性和伸缩性,使用java开发
  • MapReduce:Google MapReduce的开源实现,分布式编程模型使用户更方便的开发并行应用
使用Hadoop可以轻松的组织计算机资源,从而搭建自己的分布式计算平台,并且可以充分利用集群的计算 和存储能力,完成海量数据的处理。

二、Hadoop的优势

  1. 1.
    高可靠性:Hadoop按位存储和处理数据的能力具有很高的可靠性
  2. 2.
    高拓展性:Hadoop是在可用的计算机集簇间分配数据完成计算任务的,这些集簇可以拓展到数以千计的节点中
  3. 3.
    高效性:Hadoop能够在节点之间动态地移动数据,以保证各个节点的动态平衡,因此其处理速度非常快
  4. 4.
    高容错性:Hadoop能够自动保存数据的多份副本,并且能够自动将失败的任务重新分配

三、关联项目

  • Common:为Hadoop及其子项目提供支持的常用工具,主要包括FileSystem,RPC和串行化库。
  • Avro:Avro用于数据序列化的系统。提供了丰富的数据结构类型、快速可压缩的二进制格式、存储持久性数据的文件集、远程调用RPC的功能和简单的动态语言集成功能。
  • MapReduce:是一种编程模型,用于大规模数据集(大于1TB)的并行运算。
  • HDFS:分布式文件系统。
  • YRAN:分布式资源管理。
  • Chukwa:开源的数据收集系统,用于监控和分析大型分布式系统的数据。
  • Hive:一个建立在Hadoop基础之上的数据仓库,它提供了一些用于对Hadoop文件中的数据集进行数据整理、特殊查询和分析存储的工具。Hive提供一种结构化数据的机制,支持类似传统RDBMS的SQL语言的查询语言来帮助那些熟悉SQL的用户查询Hadoop中的数据,该查询语言成为Hive SQL。
  • Hbase:一个分布式的、面向列的开源数据库,适合非结构化的数据存储。主要用于需要随机访问、实时读写的大数据。
  • Pig:是一个对大型数据集进行分析、评估的平台。Pig最突出的优势是它的结构能够经受住高度并行化的检验。
  • Zookeeper:为分布式应用设计的协调服务,主要为用户提供同步、配置管理、分组和命令等服务。

四、编译安装Hadoop

因为我是用的是32位系统,官方预编译版本只有64位的,无法使用,所以得编译源代码。
根据编译文件BUILDING.txt内容,安装hadoop之前需要保证有以下工具:
1
Hadoop编译说明书
2
3
需要:
4
Unix 系统
5
JDK1.8
6
maven 3.3或更高
7
ProtoBuffer 2.5.0
8
CMake 3.1或更新(如果需要编译本地代码)
9
Zlib develop(如果需要编译本地代码)
10
openssl devel(如果编译原生hadoop-pipe,并获得最佳的HDFS加密性能)
11
Linux FUSE(用户空间的文件系统) 2.6或更高(如果编译fuse_dfs)
12
第一次编译需要网络保持连接(获取所有的maven和Hadoop需要的依赖)
13
Python(发布文档需要)
14
bats(用于shell代码测试)
15
Node.js / bower / Ember-cli(用于编译前端UI界面)
16
---------------------------------------------------------------------
17
获得具有所有工具的环境的最简单方法是通过Docker提供的配置。
18
这就需要一本最近的docker版本1.4.1或者更高的可以正常工作的版本
19
20
在Linux上,你可以运行下面的命名安装Docker
21
$ ./start-build-env.sh
22
接下来显示的提示是位于源树的已安装版本,并且已安装和配置了所有必需的测试和构建工具。
23
请注意,在此docker环境中,您只能从您开始的位置访问Hadoop源树。因此如果你想运行
24
dev-support/bin/test-patch /path/to/my.patch
25
那么这个patch文件必须放在hadoop源树中。
26
27
在ubuntu中清楚并安装所需的软件包:
28
Oracle JDK 1.8 (首选)
29
$ sudo apt-get purge openjdk*
30
$ sudo apt-get install software-properties-common
31
$ sudo add-apt-repository ppa:webupd8team/java
32
$ sudo apt-get update
33
$ sudo apt-get install oracle-java8-installer
34
Maven
35
$ sudo apt-get -y install maven
36
本地依赖包
37
$ sudo apt-get -y install build-essential autoconf automake libtool cmake zlib1g-dev pkg-config libssl-dev
38
ProtocolBuffer 2.5.0 (必须)
39
$ sudo apt-get -y install protobuf-compiler
Copied!
1
# 1.下载源码
2
wget https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.2/hadoop-3.1.2-src.tar.gz
3
# 2.解压
4
tar -zxcf hadoop-3.1.2-src.tar.gz
5
cd hadoop-3.1.2-src
6
# 3.mvn编译
7
mvn package -Pdist,native -DskipTests -Dtar
Copied!
编译这个玩意儿断断续续用了3天时间,下面是遇到的问题总结记录一下。
问题1:
mvn package -Pdist,native -DskipTests -Dtar的时候编译失败:
1
[ERROR] Failed to execute goal org.codehaus.mojo:native-maven-plugin:1.0-alpha-8:javah (default) on project hadoop-common: Error running javah command: Error executing command line. Exit code:2 -> [Help 1]
Copied!
解决:
vim hadoop-common-project/hadoop-common/pom.xml将javah的执行路径改为绝对路径
1
<javahPath>${env.JAVA_HOME}/bin/javah</javahPath>
2
改为
3
<javahPath>/usr/bin/javah</javahPath>
4
# 具体的路径需要对应你机器上的真实路径
Copied!
问题2:
mvn package -Pdist,native -DskipTests -Dtar的时候编译失败:
1
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.1.2:cmake-compile (cmake-compile) on project hadoop-common: CMake failed with error code 1 -> [Help 1]
2
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.1.2:cmake-compile (cmake-compile) on project hadoop-common: CMake failed with error code 1
Copied!
解决:
cmake版本不对,安装cmake3.0版本:
1
# download
2
wget https://cmake.org/files/v3.0/cmake-3.0.0.tar.gz
3
tar -zxvf cmake-3.0.0.tar.gz
4
cd cmake-3.0.0
5
./configure
6
make
7
sudo apt-get install checkinstall
8
sudo checkinstall
9
sudo make install
10
# 建立软链接
11
sudo ln -s bin/* /usr/bin/
Copied!
还是不行。使用mvn package -Pdist,native -DskipTests -Dtar -e -X打印所有日志,可以找到:
1
[INFO] Running cmake /home/wangjun/software/hadoop-3.1.2-src/hadoop-common-project/hadoop-common/src -DGENERATED_JAVAH=/home/wangjun/software/hadoop-3.1.2-src/hadoop-common-project/hadoop-common/target/native/javah -DJVM_ARCH_DATA_MODEL=32 -DREQUIRE_BZIP2=false -DREQUIRE_ISAL=false -DREQUIRE_OPENSSL=false -DREQUIRE_SNAPPY=false -DREQUIRE_ZSTD=false -G Unix Makefiles
2
[INFO] with extra environment variables {}
3
[WARNING] Soft-float JVM detected
4
[WARNING] CMake Error at /home/wangjun/software/hadoop-3.1.2-src/hadoop-common-project/hadoop-common/HadoopCommon.cmake:182 (message):
5
[WARNING] Soft-float dev libraries required (e.g. 'apt-get install libc6-dev-armel'
6
[WARNING] on Debian/Ubuntu)
7
[WARNING] Call Stack (most recent call first):
8
[WARNING] CMakeLists.txt:26 (include)
9
[WARNING]
10
[WARNING]
11
[WARNING] -- Configuring incomplete, errors occurred!
12
[WARNING] See also "/home/wangjun/software/hadoop-3.1.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeOutput.log".
13
[WARNING] See also "/home/wangjun/software/hadoop-3.1.2-src/hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeError.log".
Copied!
查看hadoop-common-project/hadoop-common/target/native/CMakeFiles/CMakeError.log日志,看到报错:
1
gnu/stubs-soft.h: No such file or directory
Copied!
解决方案:更改hadoop-common-project/hadoop-common/HadoopCommon.cmake,将两处-mfloat-abi=softfp改为-mfloat-abi=hard,参考:https://blog.csdn.net/wuyusheng314/article/details/79428996https://stackoverflow.com/questions/49139125/fatal-error-gnu-stubs-soft-h-no-such-file-or-directory。(最好是重新解压原始包更改完重新编译,要不然可能会出错)
这个改完又有了新问题,编译Apache Hadoop MapReduce NativeTask是报错
1
[WARNING] /usr/bin/ranlib libgtest.a
2
[WARNING] make[2]: Leaving directory '/home/wangjun/software/hadoop-3.1.2-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native'
3
[WARNING] /usr/local/bin/cmake -E cmake_progress_report /home/wangjun/software/hadoop-3.1.2-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/CMakeFiles 1
4
[WARNING] [ 7%] Built target gtest
5
[WARNING] make[1]: Leaving directory '/home/wangjun/software/hadoop-3.1.2-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native'
6
[WARNING] /tmp/ccpXG9td.s: Assembler messages:
7
[WARNING] /tmp/ccpXG9td.s:2040: Error: bad instruction `bswap r5'
8
[WARNING] /tmp/ccpXG9td.s:2063: Error: bad instruction `bswap r1'
9
[WARNING] make[2]: *** [CMakeFiles/nativetask.dir/build.make:79: CMakeFiles/nativetask.dir/main/native/src/codec/BlockCodec.cc.o] Error 1
10
[WARNING] make[2]: *** Waiting for unfinished jobs....
11
[WARNING] make[1]: *** [CMakeFiles/Makefile2:96: CMakeFiles/nativetask.dir/all] Error 2
12
[WARNING] make[1]: *** Waiting for unfinished jobs....
13
[WARNING] /tmp/ccBbS5rL.s: Assembler messages:
14
[WARNING] /tmp/ccBbS5rL.s:1959: Error: bad instruction `bswap r5'
15
[WARNING] /tmp/ccBbS5rL.s:1982: Error: bad instruction `bswap r1'
16
[WARNING] make[2]: *** [CMakeFiles/nativetask_static.dir/build.make:79: CMakeFiles/nativetask_static.dir/main/native/src/codec/BlockCodec.cc.o] Error 1
17
[WARNING] make[2]: *** Waiting for unfinished jobs....
18
[WARNING] /tmp/cc6DHbGO.s: Assembler messages:
19
[WARNING] /tmp/cc6DHbGO.s:979: Error: bad instruction `bswap r2'
20
[WARNING] /tmp/cc6DHbGO.s:1003: Error: bad instruction `bswap r3'
21
[WARNING] make[2]: *** [CMakeFiles/nativetask_static.dir/build.make:125: CMakeFiles/nativetask_static.dir/main/native/src/codec/Lz4Codec.cc.o] Error 1
22
[WARNING] make[1]: *** [CMakeFiles/Makefile2:131: CMakeFiles/nativetask_static.dir/all] Error 2
23
[WARNING] make: *** [Makefile:77: all] Error 2
Copied!
看错误应该是指令问题,google一番后,找到解决方案:https://issues.apache.org/jira/browse/HADOOP-14922https://issues.apache.org/jira/browse/HADOOP-11505
编辑primitives.h文件,根据https://issues.apache.org/jira/secure/attachment/12693989/HADOOP-11505.001.patch里面的git log修改后重新编译。
经历了3天的折磨,终于成功了!来,看看成功后的显示:
1
[INFO] No site descriptor found: nothing to attach.
2
[INFO] ------------------------------------------------------------------------
3
[INFO] Reactor Summary for Apache Hadoop Main 3.1.2:
4
[INFO]
5
[INFO] Apache Hadoop Main ................................. SUCCESS [ 3.532 s]
6
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [ 6.274 s]
7
[INFO] Apache Hadoop Project POM .......................... SUCCESS [ 3.668 s]
8
[INFO] Apache Hadoop Annotations .......................... SUCCESS [ 5.743 s]
9
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 1.739 s]
10
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 4.782 s]
11
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 10.777 s]
12
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 5.156 s]
13
[INFO] Apache Hadoop Auth ................................. SUCCESS [ 18.468 s]
14
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 8.293 s]
15
[INFO] Apache Hadoop Common ............................... SUCCESS [03:15 min]
16
[INFO] Apache Hadoop NFS .................................. SUCCESS [ 14.700 s]
17
[INFO] Apache Hadoop KMS .................................. SUCCESS [ 15.340 s]
18
[INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.876 s]
19
[INFO] Apache Hadoop HDFS Client .......................... SUCCESS [ 46.540 s]
20
[INFO] Apache Hadoop HDFS ................................. SUCCESS [02:34 min]
21
[INFO] Apache Hadoop HDFS Native Client ................... SUCCESS [ 12.125 s]
22
[INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 20.005 s]
23
[INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 8.934 s]
24
[INFO] Apache Hadoop HDFS-RBF ............................. SUCCESS [01:08 min]
25
[INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ 0.892 s]
26
[INFO] Apache Hadoop YARN ................................. SUCCESS [ 0.879 s]
27
[INFO] Apache Hadoop YARN API ............................. SUCCESS [ 25.531 s]
28
[INFO] Apache Hadoop YARN Common .......................... SUCCESS [01:57 min]
29
[INFO] Apache Hadoop YARN Registry ........................ SUCCESS [ 14.521 s]
30
[INFO] Apache Hadoop YARN Server .......................... SUCCESS [ 0.920 s]
31
[INFO] Apache Hadoop YARN Server Common ................... SUCCESS [ 23.432 s]
32
[INFO] Apache Hadoop YARN NodeManager ..................... SUCCESS [ 28.782 s]
33
[INFO] Apache Hadoop YARN Web Proxy ....................... SUCCESS [ 9.515 s]
34
[INFO] Apache Hadoop YARN ApplicationHistoryService ....... SUCCESS [ 14.077 s]
35
[INFO] Apache Hadoop YARN Timeline Service ................ SUCCESS [ 12.728 s]
36
[INFO] Apache Hadoop YARN ResourceManager ................. SUCCESS [ 51.338 s]
37
[INFO] Apache Hadoop YARN Server Tests .................... SUCCESS [ 8.675 s]
38
[INFO] Apache Hadoop YARN Client .......................... SUCCESS [ 13.937 s]
39
[INFO] Apache Hadoop YARN SharedCacheManager .............. SUCCESS [ 10.853 s]
40
[INFO] Apache Hadoop YARN Timeline Plugin Storage ......... SUCCESS [ 12.546 s]
41
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [ 1.069 s]
42
[INFO] Apache Hadoop YARN TimelineService HBase Common .... SUCCESS [ 17.176 s]
43
[INFO] Apache Hadoop YARN TimelineService HBase Client .... SUCCESS [ 15.662 s]
44
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [ 0.901 s]
45
[INFO] Apache Hadoop YARN TimelineService HBase Server 1.2 SUCCESS [ 17.512 s]
46
[INFO] Apache Hadoop YARN TimelineService HBase tests ..... SUCCESS [ 17.327 s]
47
[INFO] Apache Hadoop YARN Router .......................... SUCCESS [ 14.430 s]
48
[INFO] Apache Hadoop YARN Applications .................... SUCCESS [ 1.990 s]
49
[INFO] Apache Hadoop YARN DistributedShell ................ SUCCESS [ 10.400 s]
50
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ........... SUCCESS [ 7.210 s]
51
[INFO] Apache Hadoop MapReduce Client ..................... SUCCESS [ 2.549 s]
52
[INFO] Apache Hadoop MapReduce Core ....................... SUCCESS [ 38.022 s]
53
[INFO] Apache Hadoop MapReduce Common ..................... SUCCESS [ 35.908 s]
54
[INFO] Apache Hadoop MapReduce Shuffle .................... SUCCESS [ 15.180 s]
55
[INFO] Apache Hadoop MapReduce App ........................ SUCCESS [ 18.915 s]
56
[INFO] Apache Hadoop MapReduce HistoryServer .............. SUCCESS [ 15.852 s]
57
[INFO] Apache Hadoop MapReduce JobClient .................. SUCCESS [ 12.987 s]
58
[INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 12.106 s]
59
[INFO] Apache Hadoop YARN Services ........................ SUCCESS [ 1.812 s]
60
[INFO] Apache Hadoop YARN Services Core ................... SUCCESS [ 8.685 s]
61
[INFO] Apache Hadoop YARN Services API .................... SUCCESS [ 9.236 s]
62
[INFO] Apache Hadoop YARN Site ............................ SUCCESS [ 0.859 s]
63
[INFO] Apache Hadoop YARN UI .............................. SUCCESS [ 0.840 s]
64
[INFO] Apache Hadoop YARN Project ......................... SUCCESS [ 34.971 s]
65
[INFO] Apache Hadoop MapReduce HistoryServer Plugins ...... SUCCESS [ 7.376 s]
66
[INFO] Apache Hadoop MapReduce NativeTask ................. SUCCESS [02:07 min]
67
[INFO] Apache Hadoop MapReduce Uploader ................... SUCCESS [ 9.915 s]
68
[INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 14.651 s]
69
[INFO] Apache Hadoop MapReduce ............................ SUCCESS [ 15.959 s]
70
[INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 11.747 s]
71
[INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [ 16.314 s]
72
[INFO] Apache Hadoop Archives ............................. SUCCESS [ 7.115 s]
73
[INFO] Apache Hadoop Archive Logs ......................... SUCCESS [ 8.686 s]
74
[INFO] Apache Hadoop Rumen ................................ SUCCESS [ 12.413 s]
75
[INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 10.490 s]
76
[INFO] Apache Hadoop Data Join ............................ SUCCESS [ 7.894 s]
77
[INFO] Apache Hadoop Extras ............................... SUCCESS [ 7.098 s]
78
[INFO] Apache Hadoop Pipes ................................ SUCCESS [ 19.457 s]
79
[INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 12.452 s]
80
[INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [04:55 min]
81
[INFO] Apache Hadoop Kafka Library support ................ SUCCESS [ 36.248 s]
82
[INFO] Apache Hadoop Azure support ........................ SUCCESS [ 43.752 s]
83
[INFO] Apache Hadoop Aliyun OSS support ................... SUCCESS [ 34.905 s]
84
[INFO] Apache Hadoop Client Aggregator .................... SUCCESS [ 17.099 s]
85
[INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 18.819 s]
86
[INFO] Apache Hadoop Resource Estimator Service ........... SUCCESS [ 29.363 s]
87
[INFO] Apache Hadoop Azure Data Lake support .............. SUCCESS [ 30.145 s]
88
[INFO] Apache Hadoop Image Generation Tool ................ SUCCESS [ 8.970 s]
89
[INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 46.265 s]
90
[INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.883 s]
91
[INFO] Apache Hadoop Client API ........................... SUCCESS [08:41 min]
92
[INFO] Apache Hadoop Client Runtime ....................... SUCCESS [06:39 min]
93
[INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [ 4.040 s]
94
[INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS [13:29 min]
95
[INFO] Apache Hadoop Client Packaging Invariants for Test . SUCCESS [ 1.937 s]
96
[INFO] Apache Hadoop Client Packaging Integration Tests ... SUCCESS [ 1.865 s]
97
[INFO] Apache Hadoop Distribution ......................... SUCCESS [01:56 min]
98
[INFO] Apache Hadoop Client Modules ....................... SUCCESS [ 5.050 s]
99
[INFO] Apache Hadoop Cloud Storage ........................ SUCCESS [ 6.457 s]
100
[INFO] Apache Hadoop Cloud Storage Project ................ SUCCESS [ 0.829 s]
101
[INFO] ------------------------------------------------------------------------
102
[INFO] BUILD SUCCESS
103
[INFO] ------------------------------------------------------------------------
104
[INFO] Total time: 01:06 h
105
[INFO] Finished at: 2019-09-03T14:14:45+08:00
106
[INFO] ------------------------------------------------------------------------
Copied!
编译完成后的内容在hadoop-dist里面。感受一下为了编译这个玩意儿尝试了多少个版本:
1
cmake-3.0.0
2
cmake-3.3.0
3
hadoop-2.7.7-src.tar.gz
4
hadoop-2.9.2-src
5
hadoop-3.1.2-src
6
protobuf-2.5.0
7
cmake-3.1.0
8
hadoop-2.8.5-src
9
hadoop-2.9.2-src.tar.gz
10
hadoop-3.1.2-src.tar.gz
11
cmake-3.1.0.tar.gz
12
hadoop-2.7.7-src
13
hadoop-2.8.5-src.tar.gz
14
hadoop-3.1.2
15
hadoop-3.1.2.tar.gz
Copied!

五、启动运行hadoop

hadoop-dist/target里面的hadoop-3.1.2.tar.gz拷贝到你要安装的位置,解压。
1
# 进入bin目录,启动前先格式化HDFS系统
2
cd hadoop-3.1.2/bin
3
./hdfs namenode -format
4
......
5
......
6
2019-09-03 14:35:53,356 INFO namenode.NameNode: SHUTDOWN_MSG:
7
/************************************************************
8
SHUTDOWN_MSG: Shutting down NameNode at raspberrypi/127.0.1.1
9
************************************************************/
10
# 启动所有服务
11
cd ../sbin/
12
./start-all.sh
13
WARNING: Attempting to start all Apache Hadoop daemons as wangjun in 10 seconds.
14
WARNING: This is not a recommended production deployment configuration.
15
WARNING: Use CTRL-C to abort.
16
Starting namenodes on [raspberrypi]
17
Starting datanodes
18
Starting secondary namenodes [raspberrypi]
19
Starting resourcemanager
20
Starting nodemanagers
Copied!
访问8088端口http://localhost:8088就可以看到hadoop的管理界面了!
hadoop的web界面:
1
# All Applications
2
http://localhost:8088
3
# DataNode Information
4
http://localhost:9864
5
# Namenode Information
6
http://localhost:9870
7
# node
8
http://localhost:8042
9
# SecondaryNamenode information
10
http://localhost:9868
Copied!
问题1:启动时报错:
1
$ ./start-all.sh
2
WARNING: Attempting to start all Apache Hadoop daemons as wangjun in 10 seconds.
3
WARNING: This is not a recommended production deployment configuration.
4
WARNING: Use CTRL-C to abort.
5
Starting namenodes on [raspberrypi]
6
raspberrypi: ERROR: JAVA_HOME is not set and could not be found.
7
Starting datanodes
8
localhost: ERROR: JAVA_HOME is not set and could not be found.
9
Starting secondary namenodes [raspberrypi]
10
raspberrypi: ERROR: JAVA_HOME is not set and could not be found.
11
Starting resourcemanager
12
Starting nodemanagers
13
localhost: ERROR: JAVA_HOME is not set and could not be found.
Copied!
解决方案:
1
vim ./etc/hadoop/hadoop-env.sh
2
# export JAVA_HOME=
3
改为具体的java安装路径,比如
4
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-armhf
Copied!
Last modified 1yr ago