Loading... ## And 3.Hive的安装使用 ### 3.1.环境准备 <div class="tip inlineBlock warning simple small"> #### 3.1.1.Hive包准备: 将准备好的包均放置到hadoop01下的 `/opt/software`目录内统一管理 --- 这里需要使用的Hive包版本为 `apache-hive-3.1.2-bin.tar.gz` 官方下载:[Index of /dist/hive (apache.org)](https://archive.apache.org/dist/hive/) ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3694645491.png) mysql包为5.7:`mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar` 官方下载:[MySQL :: MySQL Community Downloads](https://dev.mysql.com/downloads/) ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/4126966633.png) </div> #### 3.1.2.计划安装目标 这里我将会把mysql安装到hadoop03上,然后将hive安装到另外两台主机上(hadoop01,hadoop02) ### 3.2.安装mysql 这里安装mysql可以参考之前文章:<div class="preview"> <div class="post-inser post box-shadow-wrap-normal"> <a href="http://www.zmzaxg.top/index.php/archives/689/" target="_blank" class="post_inser_a no-external-link no-underline-link"> <div class="inner-image bg" style="background-image: url(https://www.zmzaxg.top/usr/uploads/2023/10/1571074702.png);background-size: cover;"></div> <div class="inner-content" > <p class="inser-title">MySQL手动安装rpm包安装</p> <div class="inster-summary text-muted"> And 1.获取RPM包自建服务器暂时拆了,不提供自建备份下载,自行前往mysql.com下载压缩包解压,贴个图:... </div> </div> </a> <!-- .inner-content #####--> </div> <!-- .post-inser ####--> </div> #### 3.2.1.分发mysql包 将hadoop01的mysql压缩包拷到hadoop03主机上 ```shell scp -r /opt/software/mysql/mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar root@hadoop03:/opt/software/mysql/ ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1766991032.png) 到hadoop03主机上检查: ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2185681748.png) #### 3.2.2.解压mysql的rpm包 直接将mysql包解压 ```shell tar -xvf mysql-5.7.28-1.el7.x86_64.rpm-bundle.tar ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3523791577.png) #### 3.2.3.检查环境 在安装之前先检查一下目标环境是否有已经安装的mysql,避免出错 ```shell rpm -qa | grep mysql ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/4060731945.png) #### 3.2.4.安装和基础配置 没有问题就可以直接安装了,在当前目录下(mysql解压目录): ```shell rpm -ivh mysql-community-*.rpm --nodeps --force ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1122263750.png) ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1101665076.png) 启动mysql检查是否正常可用: ```shell systemctl start mysqldl ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2266666158.png) ```shell systemctl status mysqld ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3687470022.png) 查看mysql的默认初始密码: ```shell grep password /var/log/mysqld.log cat /var/log/mysqld.log |grep password ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3884259881.png) ```shell mysql -uroot -p ## 复杂密码有些字符可能会被shell解析,所以密码等弹出输入符后再输入 +.uuiXtm,7xp ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/359990328.png) 修改mysql默认密码,先修改密码策略: ```mysql show variables like "password"; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2034910355.png) 提示需要先修改初始密码: ```mysql set password for 'root'@'localhost' = 'P@ssword1'; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/492977243.png) 重启MySQL: ``` mysql> exit Bye [root@hadoop03 mysql]# systemctl restart mysqld ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3454294234.png) 再次登录mysql,使用已经修改的新密码: ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3010763425.png) 查看密码策略: ``` show variables like "%password%"; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3521963899.png) 将 `validate_password_length` 和 `validate_password_policy` 记下,这里需要让配置一直有效,需要修改mysqld的配置文件: ``` mysql> exit Bye [root@hadoop03 mysql]# vi /etc/my.cnf ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2687874470.png) 重启mysql: ```shell systemctl restart mysqld ``` 进入mysql修改密码: ```mysql set password for root@'localhost' = '123456'; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1771417111.png) 为了后面方便,我将mysql的root用户的host修改为 `%` 允许任何地址连接登录,可以不修改 ```mysql update mysql.user set host="%" where user='root' and host="localhost"; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3526273479.png) #### 3.2.5.为Hive创建元数据存储数据库 ```mysql create database hive character set latin1; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2410952189.png) #### 3.2.6.创建数据库用户hive并分配权限 ```mysql create user "hive"@"%" identified by "123456"; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2777040909.png) ```mysql grant all privileges on hive.* to 'hive'@'%' with grant option; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1726206723.png) ```mysql flush privileges; -- 刷新 show grants for hive; -- 查看用户权限 ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1739122687.png) #### 3.2.7.允许从本地查看数据库用户信息 ```mysql grant all privileges on hive.* to 'hive'@'localhost' identified by '123456'; flush privileges; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1003895059.png) ```mysql select host,user,authentication_string from mysql.user; ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/3005858338.png) ### 3.3.安装Hive #### 3.3.1.分发解压hive安装包 ##### 3.3.1.1.分发 将hive包分发到hadoop02上 ```shell scp /opt/software/apache-hive-3.1.2-bin.tar.gz root@hadoop02:/opt/software/ ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/4210367871.png) ##### 3.3.1.2.解压 这边统一将安装放置到 `/opt/module` 目录下 1.进入到压缩包目录下: ``` cd /opt/software ``` ![请输入图片描述](https://www.zmzaxg.top/usr/uploads/2023/12/148091993.png) 2.解压压缩包: ```shell tar -zxvf apache-hive-3.1.2-bin.tar.gz -C /opt/module/ ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1266673968.png) 3.进入 `module` 检查: ``` cp /opt/module ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1475916015.png) #### 3.3.2.配置安装 ##### 3.3.2.1.配置环境变量 ```shell vi /etc/profile ``` 编辑 `profile` 配置文件,添加下方配置: ``` ##########hive3.1.2 export HIVE_HOME=/opt/module/apache-hive-3.1.2-bin export HIVE_CONF_DIR=$HIVE_HOME/conf export HCAT_HOME=$HIVE_HOME/hcatalog export PATH=$HIVE_HOME/bin:$PATH ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1146322990.png) source命令让profile文件配置生效: ```shell source /etc/profile ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2320267532.png) ##### 3.3.2.2.配置hive ###### 3.3.2.2.1.配置hive-env.sh 进入hive的 `conf` 目录 ```shell cd /opt/module/apache-hive-3.1.2-bin/conf/ ``` cp复制一份 `hive-env.sh.template` 命名为 `hive-env.sh` ,并且编辑它: ```shell cp hive-env.sh.template hive-env.sh vi hive-env.sh ``` 将hadoop_home路径填入: ```shell HADOOP_HOME=/opt/module/hadoop-3.1.4 ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/370528095.png) ###### 3.3.2.2.2.配置hive-site.xml 不要切换目录,直接继续vi编辑 `hive-site.xml` 文件 配置如下(按需修改): ``` <?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <configuration> <property> <!--javax.jdo.option.ConnectionURL: 指定了Hive元数据存储数据库的连接URL,这里使用的是MySQL数据库,并指定了连接的主机和端口号,以及Hive元数据数据库的名称。参数createDatabaseIfNotExist=true表示如果指定的数据库不存在,则会尝试创建该数据库。--> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://hadoop03:3306/hive?createDatabaseIfNotExist=true</value> </property> <property> <!--javax.jdo.option.ConnectionDriverName: 指定了连接Hive元数据存储的数据库所使用的JDBC驱动名,这里是MySQL数据库,因此指定为com.mysql.jdbc.Driver。--> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <!--mysql账号--> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> </property> <property> <!--mysql密码--> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> </property> <property> <!--hive.metastore.warehouse.dir: 指定了Hive数据仓库的默认存储路径。这是Hive用于存储表数据的位置。--> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/data</value> </property> <property> <!--hive.hbase.snapshot.restoredir: 指定了HBase快照恢复的目录。这个属性用于指定在HBase快照恢复时存储快照的位置。--> <name>hive.hbase.snapshot.restoredir</name> <value>/tmp</value> </property> <property> <!--system:java.io.tmpdir: 指定了Java临时目录的位置,这个属性可以让Hive指定用于存储临时文件的位置。--> <name>system:java.io.tmpdir</name> <value>/opt/module/apache-hive-3.1.2-bin/iotmp</value> </property> <property> <!--system:user.name: 指定了Hive运行时的用户名称--> <name>system:user.name</name> <value>hive</value> </property> <!--hive.exec.local.scratchdir和hive.downloaded.resources.dir: 分别指定了Hive执行过程中使用的本地临时目录和下载资源的存储目录。--> <property> <name>hive.exec.local.scratchdir</name> <value>${system:java.io.tmpdir}/${system:user.name}</value> </property> <property> <name>hive.downloaded.resources.dir</name> <value>${system:java.io.tmpdir}/${hive.session.id}_resources</value> </property> </configuration> ``` ###### 3.3.2.2.3.应用hive-log4j2.properties ``` cp hive-log4j2.properties.template hive-log4j2.properties ``` ###### 3.3.2.2.4.应用hive-exec-log4j2.properties ``` cp hive-exec-log4j2.properties.template hive-exec-log4j2.properties ``` ###### 3.3.2.2.5.在HDFS创建hive目录 <span style='color:#ff0000'>保证hadoop集群开启状态,且正常使用</span> ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1899029227.png) 使用hdfs命令为hive创建目录 ```shell hadoop fs -mkdir -p /user/hive/data ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/2014023578.png) 确保路劲已经创建 ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/1154068075.png) ###### 3.3.2.2.6.拷JDBC驱动包 将mysql的JDBC驱动包拷到$HIVE_HOME/lib JDBC包在上方mysql官网中可下载,再贴一个:[MySQL :: 下载 MySQL Connector/J (存档版本)](https://downloads.mysql.com/archives/c-j/) ```shell cd $HIVE_HOME/lib wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-java-5.1.49.tar.gz ``` ![image.png](https://www.zmzaxg.top/usr/uploads/2023/12/453402469.png) ###### 3.3.2.2.7.Hive初始化 ``` schematool -dbType mysql -initSchema ``` ##### 3.3.2.3. ##### 3.3.2.4. 最后修改:2024 年 08 月 30 日 © 允许规范转载 打赏 赞赏作者 支付宝微信 赞 觉得文章有用,可以赞赏请我喝瓶冰露