澳门游戏平台大全 澳门游戏平台大全 澳门游戏平台大全CentOS分布式景况设置HBase

澳门游戏平台大全CentOS分布式景况设置HBase

一、安装zookeeper
在安装好分布式Hadoop的环境下,继续安装zookeeper-3.4.5。

  1. $ wget
  2. $ tar -xf zookeeper-3.4.5.tar.gz
  3. $ cp zookeeper-3.4.5/conf/zoo_sample.cfg
    zookeeper-3.4.5/conf/zoo.cfg

$ wget http://124.202.164.9/download/32878722/41763290/4/gz/165/152/1353566847397_152/zookeeper-3.4.5.tar.gz
$ tar -xf zookeeper-3.4.5.tar.gz
$ cp zookeeper-3.4.5/conf/zoo_sample.cfg zookeeper-3.4.5/conf/zoo.cfg 

修改zookeeper配置文件:

  1. $ vim zookeeper-3.4.5/conf/zoo.cfg

$ vim zookeeper-3.4.5/conf/zoo.cfg

做如下修改:

  1. tickTime=2000
  2. dataDir=/home/ruifeng.shan/zookeeper
  3. clientPort=2181
  4. initLimit=5
  5. syncLimit=2
  6. server.1=master:2888:3888
  7. server.2=node:2888:3888
  8. server.3=slave:2888:3888

tickTime=2000  
dataDir=/home/ruifeng.shan/zookeeper  
clientPort=2181  
initLimit=5  
syncLimit=2  
server.1=master:2888:3888  
server.2=node:2888:3888  
server.3=slave:2888:3888

在你指定的dataDir目录下新建一个文件myid,在里面添加你指定的server编号。例如,你为hostname名为master的指定了如下配置
server.1=master:2888:3888 ,所以它的编号应该是1.
将zookeeper-3.4.5分发到其他机器。然后修改对应的myid文件内容为之前给其指定的编号。如node为2,slave为3.
分别启动zookeeper服务。在每台机器上切换到zookeeper-3.4.5目录,执行
$bin/zkServer.sh start 就可以启动zookeeper服务。 二、安装hbase-0.96.0
为了和hadoop-2.2.0匹配,选择的hbase版本为0.96.0

  1. $ wget
  2. $ tar -xf hbase-0.96.0-hadoop2-bin.tar.gz
  3. $ cd hbase-0.96.0

$ wget http://mirror.bit.edu.cn/apache/hbase/hbase-0.96.0/hbase-0.96.0-hadoop2-bin.tar.gz
$ tar -xf hbase-0.96.0-hadoop2-bin.tar.gz
$ cd  hbase-0.96.0

修改 conf 下的配置:

  1. vim conf/hbase-env.sh

 vim conf/hbase-env.sh

修改 JAVA_HOME:

  1. # The java implementation to use. Java 1.6 required.
  2. export JAVA_HOME=/usr/lib/jvm/java/jdk1.6.0_37

# The java implementation to use.  Java 1.6 required.
export  JAVA_HOME=/usr/lib/jvm/java/jdk1.6.0_37

修改HBase 使用zookeeper的方式:

  1. # Tell HBase whether it should manage it’s own instance of
    Zookeeper or not.
  2. export HBASE_MANAGES_ZK=false

# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=false

这个参数默认是 true。意思是是否让hbase 单独管理 zookeeper
如果安装的机器上已经安装了zookeeper,并且这个zookeeper是多个平台共用的,那么这个参数应该设置为
false 如果集群上没有安装zookeeper,并且只有hbase
自己使用zookeeper,那么这个参数应该设置为true。
修改其conf目录下的hbase-site.xml文件。内容如下:

  1. <configuration>
  2. <property>
  3. <name>hbase.rootdir</name>
  4. <value>hdfs://master:8089/hbase</value>
  5. </property>
    1. <property>
  6. <name>hbase.master</name>
  7. <value>hdfs://master:60000</value>
  8. </property>
    1. <property>
  9. <name>hbase.cluster.distributed</name>
  10. <value>true</value>
  11. </property>
    1. <property>
  12. <name>hbase.zookeeper.property.clientPort</name>
  13. <value>2181</value>
  14. </property>
    1. <property>
  15. <name>hbase.zookeeper.quorum</name>
  16. <value>master,node,slave</value>
  17. </property>
    1. <property>
  18. <name>hbase.zookeeper.property.dataDir</name>
  19. <value>/home/aaron/zookeeper</value>
  20. </property>
    1. <property>
  21. <name>hbase.client.scanner.caching</name>
  22. <value>200</value>
  23. </property>
    1. <property>
  24. <name>hbase.balancer.period</name>
  25. <value>300000</value>
  26. </property>
    1. <property>
  27. <name>hbase.client.write.buffer</name>
  28. <value>10485760</value>
  29. </property>
    1. <property>
  30. <name>hbase.hregion.majorcompaction</name>
  31. <value>7200000</value>
  32. </property>
    1. <property>
  33. <name>hbase.hregion.max.filesize</name>
  34. <value>67108864</value>
  35. <description>
  36. Maximum HStoreFile size. If any one of a column families’
    HStoreFiles has
  37. grown to exceed this value, the hosting HRegion is split in
    two.</description>
  38. </property>
    1. <property>
  39. <name>hbase.hregion.memstore.flush.size</name>
  40. <value>1048576</value>
  41. <description>
  42. Memstore will be flushed to disk if size of the memstore
  43. exceeds this number of bytes. Value is checked by a thread that runs
  44. every hbase.server.thread.wakefrequency.</description>
  45. </property>
    1. <property>
  46. <name>hbase.server.thread.wakefrequency</name>
  47. <value>30000</value>
  48. <description>Time
    to sleep in between searches for work (in milliseconds).
  49. Used as sleep interval by service threads such as log roller.</description>
  50. </property>
    1. </configuration>

<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://master:8089/hbase</value>
</property>

<property>
<name>hbase.master</name>
<value>hdfs://master:60000</value>
</property>

<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>

<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>

<property>
<name>hbase.zookeeper.quorum</name>
<value>master,node,slave</value>
</property>

<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/aaron/zookeeper</value>
</property>

<property>
<name>hbase.client.scanner.caching</name>
<value>200</value>
</property>

<property>
<name>hbase.balancer.period</name>
<value>300000</value>
</property>

<property>
<name>hbase.client.write.buffer</name>
<value>10485760</value>
</property>

<property>
<name>hbase.hregion.majorcompaction</name>
<value>7200000</value>
</property>

<property>
<name>hbase.hregion.max.filesize</name>
<value>67108864</value>
<description>
Maximum HStoreFile size. If any one of a column families' HStoreFiles has
grown to exceed this value, the hosting HRegion is split in two.</description>
</property>

<property>
<name>hbase.hregion.memstore.flush.size</name>
<value>1048576</value>
<description>
Memstore will be flushed to disk if size of the memstore
exceeds this number of bytes. Value is checked by a thread that runs
every hbase.server.thread.wakefrequency.</description>
</property>

<property>
<name>hbase.server.thread.wakefrequency</name>
<value>30000</value>
<description>Time to sleep in between searches for work (in milliseconds).
Used as sleep interval by service threads such as log roller.</description>
</property>

</configuration>

其中hbase.rootdir要保持与hadoop的core-site.xml文件中的fs.default.name中的值一致。
hbase的master 和
regionserver的时钟要保持一致,否则启动regionserver的时候会报如下错误:
org.apache.hadoop.hbase.ClockOutOfSyncException:
org.apache.hadoop.hbase.ClockOutOfSyncException: Server
hadoop02,60020,1374576225508 has been rejected; Reported time is too far
out of sync with master. Time difference of 62062623ms > max allowed
of 30000ms 这是由于master 和 regionserver之间的时间误差太大造成的。
解决方法1 :修改下面的参数

  1. <name>hbase.master.maxclockskew</name>
  2. <value>180000</value>
  3. <description>Time
    difference of regionserver from master</description>

<name>hbase.master.maxclockskew</name>
<value>180000</value>
<description>Time difference of regionserver from master</description>

该参数默认值为 30 000 (30秒) 解决方法2: 使用ntpdate 同步集群的时钟
修改regionservers 文件配置:
这个文件是用来指定regionserver的机器的,一行一个,一般使用datanode作为regionserver:

  1. $ vim conf/regionservers

$ vim conf/regionservers

master node slave 配置 backup-masters:

  1. $ vim conf/backup-masters

$ vim conf/backup-masters

master-backup 配置backup-masters 可以在master 挂掉之后自动启动backup
masters 注意:为了兼容Hadoop-2.2.0,需要将hbase的lib包中的内容lib包中的hadoop-common-2.1.0-beta.jar替换成hadoop-2.2.0/share/hadoop/common目录下的hadoop-common-2.2.0.jar。
启动hbase: 如果集群各个机器都有ssh 无密码登陆,可以直接使用
start-hbase.sh 启动整个集群。
或者到每个机器上使用daemon脚本启动(切换到hbase目录下): 到master
机器上:bin/hbase-daemon.sh start master 到regionserver 机器上 :
bin/hbase-daemon.sh start regionserver 查看集群是否正常:
查看master的监控页面:

HBase 的详细介绍:请点这里
HBase 的下载地址:请点这里

Hadoop+HBase搭建云存储总结 PDF

HBase 结点之间时间不一致造成regionserver启动失败

Hadoop+ZooKeeper+HBase集群配置

Hadoop集群安装&HBase实验环境搭建

基于Hadoop集群的HBase集群的配置 ‘

Hadoop安装部署笔记之-HBase完全分布模式安装

单机版搭建HBase环境图文教程详解

在安装好分布式Hadoop的环境下,继续安装zookeeper-3.4.5。 $ wget

标签:

相关文章

发表评论

电子邮件地址不会被公开。 必填项已用*标注

网站地图xml地图