本安装文档是是伪分布式安装文档,若要安装完全分布式集群,请根据本文自己区分主从机器关系。
一 下载安装依赖包
yum -y install cyrus-sasl-plain lsb ntp
安装impala
二 安装 bigtop-utils (主从机都要安装)
rpm -ivh bigtop-utils-0.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.30.el7.noarch.rpm
安装 impala-kudu-2.7.0 (主从机都要安装)
rpm -ivh impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el6.x86_64.rpm --nodeps
安装 impala-kudu-catalog (主机安装)
rpm -ivh impala-kudu-catalog-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.10.el6.x86_64.rpm --nodeps
安装 impala-kudu-state (主机安装)
rpm -ivh impala-kudu-state-store-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el6.x86_64.rpm --nodeps
主机安装 impala-kudu-server (主从机都要安装)
rpm -ivh impala-kudu-server-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el6.x86_64.rpm --nodeps
安装impala-kudu-shell (主从机都要安装)
rpm -ivh impala-kudu-shell-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.10.el6.x86_64.rpm --nodeps
安装 impala-kudu-udf-devel (主从机都要安装)
rpm -ivh impala-kudu-udf-devel-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el6.x86_64.rpm --nodeps
安装 impala-kudu-debuginfo (主从机都要安装)
rpm -ivh impala-kudu-debuginfo-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.10.el6.x86_64.rpm --nodeps
三 配置impala (主从机都要配置)
3.1. 配置 vim /etc/default/bigtop-utils里的JDK
export JAVA_HOME=/usr/java/jdk1.8.0_77
3.2.配置vim /etc/default/impala (主从机都要配置)
IMPALA_CATALOG_SERVICE_HOST=apollo.dt.com
IMPALA_STATE_STORE_HOST=apollo.dt.com
四 配置三台机器时间同步(主从机都要配置)
五 修改HADOOP的core-site.xml和hdfs-site.xml
添加如下内容:
hdfs-site.xml文件
<!--impala configuration -->
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
<property>
<name>dfs.client.read.shortcircuit.skip.checksum</name>
<value>false</value>
</property>
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.block.local-path-access.user</name>
<value>impala</value>
</property>
<property>
<name>dfs.client.file-block-storage-locations.timeout.millis</name>
<value>60000</value>
</property>
<property>
<name>dfs.domain.socket.path</name>
<value>/var/run/hadoop-hdfs</value>
</property>
在目录/var/run/目录下新建目录hadoop-hdfs (主从机都要配置)。
注意:该文件夹可能已经存在,应当确认用impala是否有权限进行读写。
如果已经存在,将用户impala加入该文件所属的组,并修改该文件组的权限即:
usermod -G root impala
chown -R 775 hadoop-hdfs
-g 为主组 -G 为附加组
hadoop fs -mkdir -p /user/impala
hadoop fs -chown impala /user/impala
cp /usr/local/hadoop/etc/hadoop/core-site.xml /etc/impala/conf.dist
cp /usr/local/hadoop/etc/hadoop/hdfs-site.xml /etc/impala/conf.dist
service impala-state-store restart --kudu_master_hosts=hadoop:7051
service impala-catalog restart --kudu_master_hosts=hadoop:7051
service impala-server restart --kudu_master_hosts=hadoop:7051
service impala-server restart --kudu_master_hosts=hadoop:7051
service impala-server restart --kudu_master_hosts=hadoop:7051
登录
impala-shell
CREATE TABLE dept (
id BIGINT,
name STRING
) distribute by range(name) split rows(('it'),('op'),('hr'))
TBLPROPERTIES(
'storage_handler' = 'com.cloudera.kudu.hive.KuduStorageHandler',
'kudu.table_name' = 'dept',
'kudu.master_addresses' = 'hadoop:7051',
'kudu.key_columns' = 'id,name'
);
如果觉得我的文章对您有用,请点赞。您的支持将鼓励我继续创作!
赞2
添加新评论0 条评论