hmaster和hregionserver 16020 端口沖突問題

問題背景:

最近升級了下hadoop、 hbase、 spark,發現hbase升級后主節點上的hregionsever老是啟動不了。

查看日志發現如下端口沖突錯誤:

java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2487) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2502)Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2485) ... 5 moreCaused by: java.net.BindException: Problem binding to hadoop1/192.168.1.201:16020 : 地址已在使用 at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2371) at org.apache.hadoop.hbase.ipc.RpcServer$Listener.<init>(RpcServer.java:524) at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:1899) at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:790) at org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:575) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:492) ... 10 moreCaused by: java.net.BindException: 地址已在使用 at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:444) at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2369)1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

經過仔細查看官方文檔,問題解決。特記錄一下。

環境:

hadoop版本 : 2.5.1

hbase 版本 : 1.0.0

spark 版本: 1.3.0

其中hbase主節點上也同時啟動了hregionserver服務。

問題原因:

在hbase升級到1.0.0版本后,默認端口做了改動。其中16020端口是hmaster服務和hregionserver服務各自使用的默認端口,導致端口沖突。

官方文檔相關信息如下

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers, which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-master-backup.sh. For each backup master you want to start, add a parameter representing the port offset for that master. Each HMaster uses three ports (16010, 16020, and 16030 by default). The port offset is added to these ports, so using an offset of 2, the backup HMaster would use ports 16012, 16022, and 16032. The following command starts 3 backup servers using ports 16012/16022/16032, 16013/16023/16033, and 16015/16025/16035.

The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same system can be useful for testing in pseudo-distributed mode. The local-regionservers.sh command allows you to run multiple RegionServers. It works in a similar way to the local-master-backup.sh command, in that each parameter you provide represents the port offset for an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030. However, the base ports for additional RegionServers are not the default ports since the default ports are used by the HMaster, which is also a RegionServer since HBase version 1.0.0. The base ports are 16200 and 16300 instead. You can run 99 additional RegionServers that are not a HMaster or backup HMaster, on a server. The following command starts four additional RegionServers, running on sequential ports starting at 16202/16302 (base ports 16200/16300 plus 2).

解決方法:

按理說不使用默認配置,定義自己的端口配置就可以解決該問題。

比如使用如下配置:

<property><name>hbase.master.port</name><value>16000</value></property><property><name>hbase.master.info.port</name><value>16010</value></property><property><name>hbase.regionserver.port</name><value>16201</value></property><property><name>hbase.regionserver.info.port</name><value>16301</value></property></configuration>1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

但是實際上 使用start-hbase.sh 腳本啟動regionserver還是會報端口沖突問題,可能通過這個腳本啟動程序存在問題。沒有深究看源碼。

該問題可以通過 使用單獨的regionserver啟動腳本程序啟動regionserver來規避。

使用方法:

bin/local-regionservers.sh start 11

它使用的端口實際上就是

16201和16301

免責聲明:本文僅代表文章作者的個人觀點,與本站無關。其原創性、真實性以及文中陳述文字和內容未經本站證實,對本文以及其中全部或者部分內容文字的真實性、完整性和原創性本站不作任何保證或承諾,請讀者僅作參考,并自行核實相關內容。

http://www.uswqb.club/style/images/nopic.gif
分享
評論
首頁
高速公路之王电子游艺
重庆福彩欢乐生肖开奖结果 北京11选5开奖查 福建快3走势图 如何网上兼职赚钱 福建36选7开奖规则 成都麻将app下载 玩股票开户怎么开 辉煌棋牌官方版正版下载 山西十一选五前三直选开奖结果 内蒙古十一选五任五 国标麻将单机版1.3 西甲赛程 遇乐棋牌二鬼手机版下载 四川快乐12* 广东好彩1推荐 北单比分sp值开奖澳客