2.6.0安装snappy not ok

3htmauhk  于 2021-05-29  发布在  Hadoop
关注(0)|答案(0)|浏览(235)

我的hadoop版本是“hadoop-2.6.0-cdh5.4.5”我的linux版本是“description:centos release 6.5(final)”我的hadoop home是“/home/hadoop/hadoop-2.6.0-cdh5.4.5”

I have install snappy in /usr/lib64:
libsnappy.so -> libsnappy.so.1.1.4    
libsnappy.so.1 -> libsnappy.so.1.1.4  
libsnappy.so.1.1.4    

I also finish hadoop-snappy by mvn:
1) copy hadoop-snappy-0.0.1-SNAPSHOT.jar /home/hadoop/hadoop-2.6.0-cdh5.4.5/lib/.
2)cp -r $HADOOP-SNAPPY_CODE_HOME/target/hadoop-snappy-0.0.1-SNAPSHOT/lib/native/Linux-amd64-64  /home/hadoop/hadoop-2.6.0-cdh5.4.5/lib/native/

I ln the lib in /home/hadoop/hadoop-2.6.0-cdh5.4.5/lib/native:
lrwxrwxrwx.   1 root   root        23 Jan 17 17:11 libsnappy.so -> /usr/lib64/libsnappy.so
lrwxrwxrwx.   1 root   root        25 Jan 17 17:16 libsnappy.so.1 -> /usr/lib64/libsnappy.so.1   

I also finish the /home/hadoop/hadoop-2.6.0-cdh5.4.5/etc/hadoop/hadoop.env.sh:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native/Linux-amd64-64/
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_HOME/lib/native/Linux-amd64-64/
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native"
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native   

but when I check the snappy lib:
[hadoop@stormmaster hadoop]$ hadoop checknative
16/01/17 17:52:20 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Native library checking:
hadoop:  false 
zlib:    false 

**snappy:  false**

lz4:     false 
bzip2:   false 
openssl: false 
16/01/17 17:52:20 INFO util.ExitUtil: Exiting with status 1      

I try to download a hadoop-native-64-2.6.0,and put in /home/hadoop/hadoop-2.6.0-cdh5.4.5/lib/native:
-rw-r--r--.   1 hadoop hadoop 1119486 Jan 15 15:51 libhadoop.a
-rw-r--r--.   1 hadoop hadoop 1486964 Jan 15 15:51 libhadooppipes.a
lrwxrwxrwx.   1 hadoop hadoop      24 Jan 11 15:11 libhadoopsnappy.so -> libhadoopsnappy.so.0.0.1
lrwxrwxrwx.   1 hadoop hadoop      24 Jan 11 15:11 libhadoopsnappy.so.0 -> libhadoopsnappy.so.0.0.1
-rwxr-xr-x.   1 hadoop hadoop   54437 Jan 15 15:51 libhadoopsnappy.so.0.0.1
-rwxr-xr-x.   1 hadoop hadoop  671189 Jan 15 15:51 libhadoop.so
-rwxr-xr-x.   1 hadoop hadoop  671189 Jan 15 15:51 libhadoop.so.1.0.0
-rw-r--r--.   1 hadoop hadoop  581944 Jan 15 15:51 libhadooputils.a
-rw-r--r--.   1 hadoop hadoop  359458 Jan 15 15:51 libhdfs.a
-rwxr-xr-x.   1 hadoop hadoop  228435 Jan 15 15:51 libhdfs.so
-rwxr-xr-x.   1 hadoop hadoop  228435 Jan 15 15:51 libhdfs.so.0.0.0

also hadoop checknative:
16/01/17 17:55:00 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version
16/01/17 17:55:00 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
Native library checking:

**hadoop:  true /home/hadoop/hadoop-2.6.0-cdh5.4.5/lib/native/libhadoop.so

zlib:    true /lib64/libz.so.1
snappy:  false 
lz4:     true revision:99
bzip2:   false 
openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open shared object file: No such file or directory)!**

如何解决这个问题,并在hadoop中使用snappy,thx!

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题