HBASE-18095: Zookeeper-less client connection implementation#781
Closed
bharathv wants to merge 1 commit intoapache:masterfrom
Closed
HBASE-18095: Zookeeper-less client connection implementation#781bharathv wants to merge 1 commit intoapache:masterfrom
bharathv wants to merge 1 commit intoapache:masterfrom
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generally when a HBaseClient tries to create a cluster
Connection, it fetches a bunch of metadata from Zookeeper
(like active master address, clusterID, meta locations etc)
before it creates the underlying transport. However exposing
ZK to all the clients is a DDOS risk and ZK connections in
the past have caused issues by not timing out on blocking
RPCs (more context in the JIRA).
This patch attempts to remove this ZK dependency by making
the client fetch all the meta information directly from list
of client configured HMaster endpoints. The patch adds a
a new AsyncRegistry implementation that encapsulates this logic
of fetching this meta information from the provided master
end points. New RPCs are added to the HMasters to help fetch
this information.
Meta HRL caching:
One critical piece of metadata needed by clients to query tables
is meta HRegionLocations. These are fetched from ZK by default.
Since this patch moves away from ZK, it adds an in-memory cache
of these locations on both Active/StandBy HMasters. ZK Listeners
are registered to keep the cache up-to-date.
New client configs:
list of comma separated HMaster host:port addresses.
set to HMasterAsyncRegistry class.
not accessible etc).