Skip to content

HBASE-18095: Zookeeper-less client connection implementation#781

Closed
bharathv wants to merge 1 commit intoapache:masterfrom
bharathv:HBASE-18095
Closed

HBASE-18095: Zookeeper-less client connection implementation#781
bharathv wants to merge 1 commit intoapache:masterfrom
bharathv:HBASE-18095

Conversation

@bharathv
Copy link
Contributor

@bharathv bharathv commented Nov 1, 2019

Generally when a HBaseClient tries to create a cluster
Connection, it fetches a bunch of metadata from Zookeeper
(like active master address, clusterID, meta locations etc)
before it creates the underlying transport. However exposing
ZK to all the clients is a DDOS risk and ZK connections in
the past have caused issues by not timing out on blocking
RPCs (more context in the JIRA).

This patch attempts to remove this ZK dependency by making
the client fetch all the meta information directly from list
of client configured HMaster endpoints. The patch adds a
a new AsyncRegistry implementation that encapsulates this logic
of fetching this meta information from the provided master
end points. New RPCs are added to the HMasters to help fetch
this information.

Meta HRL caching:

One critical piece of metadata needed by clients to query tables
is meta HRegionLocations. These are fetched from ZK by default.
Since this patch moves away from ZK, it adds an in-memory cache
of these locations on both Active/StandBy HMasters. ZK Listeners
are registered to keep the cache up-to-date.

New client configs:

  • 'hbase.client.asyncregistry.masteraddrs' Should be set to a
    list of comma separated HMaster host:port addresses.
  • Should be used in conjunction with 'hbase.client.registry.impl'
    set to HMasterAsyncRegistry class.
  • Testing is still a WIP. Error paths are missing (ex: a master is
    not accessible etc).

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants