Sqoop数据导出快速上手

x33g5p2x  于2020-09-08 发布在 Sqoop  
字(1.1k)|赞(0)|评价(0)|浏览(718)

本节主要内容:

  • 介绍如何使用Sqoop通过jdbc,从关系型数据库postgresql导出数据到HDFS中。
  • Sqoop从关系型数据PostgreSQL导出数据到HDFS中。

Sqoop下载

tar -zxf sqoop-1.99.7-bin-hadoop200.tar.gz
sudo mkdir -p /opt/sqoop
sudo mv sqoop

设置Sqoop环境变量

mkdir sqoop-1.99.7-bin-hadoop200/extra-lib

添加jdbc驱动包

wget http://central.maven.org/maven2/org/postgresql/postgresql/42.2.2/postgresql-42.2.2.jar

mv postgresql-42.2.2.jar /opt/sqoop2/sqoop-1.99.7-bin-hadoop200/extra-lib

设置Sqoop所需环境变量

sudo vim /etc/profile.d/sqoop-2.0.sh
#!/bin/sh
# Author:wangxiaolei 王小雷
# Blog: http://blog.csdn.net/dream_an
# Github: https://github.com/lycheeman
# Path: /etc/profile.d/
# Date:20180615

export SQOOP_HOME=/opt/sqoop2/sqoop-1.99.7-bin-hadoop200
export PATH=$SQOOP_HOME/bin:$PATH
export SQOOP_SERVER_EXTRA_LIB=$SQOOP_HOME/extra-lib

设置Hadoop环境变量

sudo vim /etc/profile.d/hadoop-3.1.0.sh
#!/bin/sh
# Author:wangxiaolei 王小雷
# Blog: http://blog.csdn.net/dream_an
# Github: https://github.com/lycheeman
# Path: /etc/profile.d/
# Date:20180615

export HADOOP_HOME="/opt/hadoop/hadoop-3.1.0"
export PATH="$HADOOP_HOME/bin:$PATH"
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop

相关文章

微信公众号

最新文章

更多