@Override
protected boolean shouldSplit() {
if (region.shouldForceSplit()) return true;
boolean foundABigStore = false;
// Get count of regions that have the same common table as this.region 获取到该regionserver下tablename下所有的region个数
int tableRegionsCount = getCountOfCommonTableRegions();
// Get size to check
long sizeToCheck = getSizeToCheck(tableRegionsCount);
for (Store store : region.getStores()) {
// If any of the stores is unable to split (eg they contain reference files)
// then don't split
if ((!store.canSplit())) {
return false;
}
// Mark if any store is big enough
long size = store.getSize();
if (size > sizeToCheck) {
LOG.debug("ShouldSplit because " + store.getColumnFamilyName() +
" size=" + size + ", sizeToCheck=" + sizeToCheck +
", regionsWithCommonTable=" + tableRegionsCount);
foundABigStore = true;
}
}
return foundABigStore;
}
/**
* @return Count of regions on this server that share the table this.region
* belongs to
*/
private int getCountOfCommonTableRegions() {
RegionServerServices rss = this.region.getRegionServerServices();
// Can be null in tests
if (rss == null) return 0;
TableName tablename = this.region.getTableDesc().getTableName();
int tableRegionsCount = 0;
try {
List<Region> hri = rss.getOnlineRegions(tablename);
tableRegionsCount = hri == null || hri.isEmpty()? 0: hri.size();
} catch (IOException e) {
LOG.debug("Failed getOnlineRegions " + tablename, e);
}
return tableRegionsCount;
}
</pre><pre>
如果tableRegionsCount ==0或者tableRegionsCount 》100,sizeToCheck 取desiredMaxFileSize,否则取desiredMaxFileSize和this.initialSize * tableRegionsCount * tableRegionsCount * tableRegionsCount的最小值,initialSize取this.initialSize = conf.getLong("hbase.increasing.policy.initial.size", -1);
采用默认,一张表刚建立,默认情况只有1个region,那么逻辑上是当这个region的store大小超过 1*1*1*flushsize = 128M(没有自己设置flushSize)时 才会允许split,如果达到这个值切分后,会有两个region,其中一个region中的某个store大小大于 2*2*2*
flushsize =
512M 时,则允许split,如此计算下去,直到这个大小超过了
hbase.hregion.max.filesize+ hbase.hregion.max.filesize*随机小数*
hbase.hregion.max.filesize.jitter才允许split,基本也就固定了,如果粗劣的计算可以把这个
hbase.hregion.max.filesize
的大小作为最后的阀值,默认是10G,也就说当这个阀值变化到10G,这个阀值就基本上不再变化。
/**
* @return Region max size or <code>count of regions squared * flushsize, which ever is
* smaller; guard against there being zero regions on this server.
*/
protected long getSizeToCheck(final int tableRegionsCount) {
// safety check for 100 to avoid numerical overflow in extreme cases
return tableRegionsCount == 0 || tableRegionsCount > 100 ? getDesiredMaxFileSize():
Math.min(getDesiredMaxFileSize(),
this.initialSize * tableRegionsCount * tableRegionsCount * tableRegionsCount);
}
<span style="font-family: Arial, Helvetica, sans-serif;">long getDesiredMaxFileSize() {</span>
return desiredMaxFileSize;
}
@Override
protected void configureForRegion(HRegion region) {
super.configureForRegion(region);
Configuration conf = getConf();
HTableDescriptor desc = region.getTableDesc();
if (desc != null) {
this.desiredMaxFileSize = desc.getMaxFileSize();
}
if (this.desiredMaxFileSize <= 0) {
this.desiredMaxFileSize = conf.getLong(HConstants.HREGION_MAX_FILESIZE,
HConstants.DEFAULT_MAX_FILE_SIZE);
}
double jitter = conf.getDouble("hbase.hregion.max.filesize.jitter", 0.25D);
this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() - 0.5D) * jitter);
}代码逻辑都比较简单,这里最这
转载请注明原文地址: https://ju.6miu.com/read-1309279.html