hbase源码分析-是否split

    xiaoxiao2026-05-02  0

    @Override protected boolean shouldSplit() { if (region.shouldForceSplit()) return true; boolean foundABigStore = false; // Get count of regions that have the same common table as this.region 获取到该regionserver下tablename下所有的region个数 int tableRegionsCount = getCountOfCommonTableRegions(); // Get size to check long sizeToCheck = getSizeToCheck(tableRegionsCount); for (Store store : region.getStores()) { // If any of the stores is unable to split (eg they contain reference files) // then don't split if ((!store.canSplit())) { return false; } // Mark if any store is big enough long size = store.getSize(); if (size > sizeToCheck) { LOG.debug("ShouldSplit because " + store.getColumnFamilyName() + " size=" + size + ", sizeToCheck=" + sizeToCheck + ", regionsWithCommonTable=" + tableRegionsCount); foundABigStore = true; } } return foundABigStore; } /** * @return Count of regions on this server that share the table this.region * belongs to */ private int getCountOfCommonTableRegions() { RegionServerServices rss = this.region.getRegionServerServices(); // Can be null in tests if (rss == null) return 0; TableName tablename = this.region.getTableDesc().getTableName(); int tableRegionsCount = 0; try { List<Region> hri = rss.getOnlineRegions(tablename); tableRegionsCount = hri == null || hri.isEmpty()? 0: hri.size(); } catch (IOException e) { LOG.debug("Failed getOnlineRegions " + tablename, e); } return tableRegionsCount; } </pre><pre> 如果tableRegionsCount ==0或者tableRegionsCount 》100,sizeToCheck 取desiredMaxFileSize,否则取desiredMaxFileSize和this.initialSize * tableRegionsCount * tableRegionsCount * tableRegionsCount的最小值,initialSize取this.initialSize = conf.getLong("hbase.increasing.policy.initial.size", -1); 采用默认,一张表刚建立,默认情况只有1个region,那么逻辑上是当这个region的store大小超过 1*1*1*flushsize = 128M(没有自己设置flushSize)时 才会允许split,如果达到这个值切分后,会有两个region,其中一个region中的某个store大小大于 2*2*2* flushsize =  512M 时,则允许split,如此计算下去,直到这个大小超过了 hbase.hregion.max.filesizehbase.hregion.max.filesize*随机小数* hbase.hregion.max.filesize.jitter才允许split,基本也就固定了,如果粗劣的计算可以把这个 hbase.hregion.max.filesize 的大小作为最后的阀值,默认是10G,也就说当这个阀值变化到10G,这个阀值就基本上不再变化。 /** * @return Region max size or <code>count of regions squared * flushsize, which ever is * smaller; guard against there being zero regions on this server. */ protected long getSizeToCheck(final int tableRegionsCount) { // safety check for 100 to avoid numerical overflow in extreme cases return tableRegionsCount == 0 || tableRegionsCount > 100 ? getDesiredMaxFileSize(): Math.min(getDesiredMaxFileSize(), this.initialSize * tableRegionsCount * tableRegionsCount * tableRegionsCount); } <span style="font-family: Arial, Helvetica, sans-serif;">long getDesiredMaxFileSize() {</span> return desiredMaxFileSize; } @Override protected void configureForRegion(HRegion region) { super.configureForRegion(region); Configuration conf = getConf(); HTableDescriptor desc = region.getTableDesc(); if (desc != null) { this.desiredMaxFileSize = desc.getMaxFileSize(); } if (this.desiredMaxFileSize <= 0) { this.desiredMaxFileSize = conf.getLong(HConstants.HREGION_MAX_FILESIZE, HConstants.DEFAULT_MAX_FILE_SIZE); } double jitter = conf.getDouble("hbase.hregion.max.filesize.jitter", 0.25D); this.desiredMaxFileSize += (long)(desiredMaxFileSize * (RANDOM.nextFloat() - 0.5D) * jitter); }代码逻辑都比较简单,这里最这
    转载请注明原文地址: https://ju.6miu.com/read-1309279.html
    最新回复(0)