org.apache.hadoop.hbase.util.Bytes.split()方法的使用及代码示例

x33g5p2x  于2022-01-16 转载在 其他  
字(10.0k)|赞(0)|评价(0)|浏览(114)

本文整理了Java中org.apache.hadoop.hbase.util.Bytes.split()方法的一些代码示例，展示了Bytes.split()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Bytes.split()方法的具体详情如下：
包路径：org.apache.hadoop.hbase.util.Bytes
类名称：Bytes
方法名：split

Bytes.split介绍

[英]Split passed range. Expensive operation relatively. Uses BigInteger math. Useful splitting ranges for MapReduce jobs.
[中]分割通过范围。操作相对昂贵。使用大整数数学。MapReduce作业的有用分割范围。

代码示例

代码示例来源：origin: apache/hbase

/**
 * Split passed range.  Expensive operation relatively.  Uses BigInteger math.
 * Useful splitting ranges for MapReduce jobs.
 * @param a Beginning of range
 * @param b End of range
 * @param num Number of times to split range.  Pass 1 if you want to split
 * the range in two; i.e. one split.
 * @return Array of dividing values
 */
public static byte [][] split(final byte [] a, final byte [] b, final int num) {
 return split(a, b, false, num);
}

代码示例来源：origin: apache/hbase

@Override
public byte[] split(byte[] start, byte[] end) {
 return Bytes.split(start, end, 1)[1];
}

代码示例来源：origin: apache/hbase

private byte[][] getSplitKeys(byte[] startKey, byte[] endKey, int numRegions) {
 if (numRegions < 3) {
  throw new IllegalArgumentException("Must create at least three regions");
 } else if (Bytes.compareTo(startKey, endKey) >= 0) {
  throw new IllegalArgumentException("Start key must be smaller than end key");
 }
 if (numRegions == 3) {
  return new byte[][] { startKey, endKey };
 }
 byte[][] splitKeys = Bytes.split(startKey, endKey, numRegions - 3);
 if (splitKeys == null || splitKeys.length != numRegions - 1) {
  throw new IllegalArgumentException("Unable to split key range into enough regions");
 }
 return splitKeys;
}

代码示例来源：origin: apache/hbase

@Override
public void createTable(TableDescriptor desc, byte[] startKey, byte[] endKey, int numRegions)
  throws IOException {
 if(numRegions < 3) {
  throw new IllegalArgumentException("Must create at least three regions");
 } else if(Bytes.compareTo(startKey, endKey) >= 0) {
  throw new IllegalArgumentException("Start key must be smaller than end key");
 }
 if (numRegions == 3) {
  createTable(desc, new byte[][]{startKey, endKey});
  return;
 }
 byte [][] splitKeys = Bytes.split(startKey, endKey, numRegions - 3);
 if(splitKeys == null || splitKeys.length != numRegions - 1) {
  throw new IllegalArgumentException("Unable to split key range into enough regions");
 }
 createTable(desc, splitKeys);
}

代码示例来源：origin: apache/hbase

@Override
public void createTable(TableDescriptor desc, byte [] startKey,
  byte [] endKey, int numRegions)
throws IOException {
 if(numRegions < 3) {
  throw new IllegalArgumentException("Must create at least three regions");
 } else if(Bytes.compareTo(startKey, endKey) >= 0) {
  throw new IllegalArgumentException("Start key must be smaller than end key");
 }
 if (numRegions == 3) {
  createTable(desc, new byte[][]{startKey, endKey});
  return;
 }
 byte [][] splitKeys = Bytes.split(startKey, endKey, numRegions - 3);
 if(splitKeys == null || splitKeys.length != numRegions - 1) {
  throw new IllegalArgumentException("Unable to split key range into enough regions");
 }
 createTable(desc, splitKeys);
}

代码示例来源：origin: apache/hbase

/**
 * Create region split keys between startkey and endKey
 *
 * @param startKey
 * @param endKey
 * @param numRegions the number of regions to be created. it has to be greater than 3.
 * @return
 */
public byte[][] getRegionSplitStartKeys(byte[] startKey, byte[] endKey, int numRegions){
 assertTrue(numRegions>3);
 byte [][] tmpSplitKeys = Bytes.split(startKey, endKey, numRegions - 3);
 byte [][] result = new byte[tmpSplitKeys.length+1][];
 System.arraycopy(tmpSplitKeys, 0, result, 1, tmpSplitKeys.length);
 result[0] = HConstants.EMPTY_BYTE_ARRAY;
 return result;
}

代码示例来源：origin: apache/hbase

/**
 * Create a table with multiple regions.
 * @param tableName
 * @param family
 * @param numRegions
 * @return A Table instance for the created table.
 * @throws IOException
 */
public Table createMultiRegionTable(TableName tableName, byte[] family, int numRegions)
  throws IOException {
 if (numRegions < 3) throw new IOException("Must create at least 3 regions");
 byte[] startKey = Bytes.toBytes("aaaaa");
 byte[] endKey = Bytes.toBytes("zzzzz");
 byte[][] splitKeys = Bytes.split(startKey, endKey, numRegions - 3);
 return createTable(tableName, new byte[][] { family }, splitKeys);
}

代码示例来源：origin: apache/hbase

@Override
public byte[][] split(byte[] start, byte[] end, int numSplits, boolean inclusive) {
 if (Arrays.equals(start, HConstants.EMPTY_BYTE_ARRAY)) {
  start = firstRowBytes;
 }
 if (Arrays.equals(end, HConstants.EMPTY_BYTE_ARRAY)) {
  end = lastRowBytes;
 }
 Preconditions.checkArgument(
     Bytes.compareTo(end, start) > 0,
     "last row (%s) is configured less than first row (%s)",
     Bytes.toStringBinary(end),
     Bytes.toStringBinary(start));
 byte[][] splits = Bytes.split(start, end, true,
     numSplits - 1);
 Preconditions.checkState(splits != null,
     "Could not calculate input splits with given user input: " + this);
 if (inclusive) {
  return splits;
 } else {
  // remove endpoints, which are included in the splits list
  return Arrays.copyOfRange(splits, 1, splits.length - 1);
 }
}

代码示例来源：origin: apache/hbase

@Override
public byte[][] split(int numRegions) {
 Preconditions.checkArgument(
   Bytes.compareTo(lastRowBytes, firstRowBytes) > 0,
   "last row (%s) is configured less than first row (%s)",
   Bytes.toStringBinary(lastRowBytes),
   Bytes.toStringBinary(firstRowBytes));
 byte[][] splits = Bytes.split(firstRowBytes, lastRowBytes, true,
   numRegions - 1);
 Preconditions.checkState(splits != null,
   "Could not split region with given user input: " + this);
 // remove endpoints, which are included in the splits list
 return splits == null? null: Arrays.copyOfRange(splits, 1, splits.length - 1);
}

代码示例来源：origin: apache/hbase

public void testSplit2() throws Exception {
 // More split tests.
 byte [] lowest = Bytes.toBytes("http://A");
 byte [] highest = Bytes.toBytes("http://z");
 byte [] middle = Bytes.toBytes("http://]");
 byte [][] parts = Bytes.split(lowest, highest, 1);
 for (int i = 0; i < parts.length; i++) {
  System.out.println(Bytes.toString(parts[i]));
 }
 assertEquals(3, parts.length);
 assertTrue(Bytes.equals(parts[1], middle));
}

代码示例来源：origin: apache/hbase

@Override
public int getPartition(final ImmutableBytesWritable key, final VALUE value,
  final int reduces) {
 if (reduces == 1) return 0;
 if (this.lastReduces != reduces) {
  this.splits = Bytes.split(this.startkey, this.endkey, reduces - 1);
  for (int i = 0; i < splits.length; i++) {
   LOG.info(Bytes.toStringBinary(splits[i]));
  }
  this.lastReduces = reduces;
 }
 int pos = Bytes.binarySearch(this.splits, key.get(), key.getOffset(),
  key.getLength());
 // Below code is from hfile index search.
 if (pos < 0) {
  pos++;
  pos *= -1;
  if (pos == 0) {
   // falls before the beginning of the file.
   throw new RuntimeException("Key outside start/stop range: " +
    key.toString());
  }
  pos--;
 }
 return pos;
}

代码示例来源：origin: apache/hbase

public void testSplit() throws Exception {
 byte [] lowest = Bytes.toBytes("AAA");
 byte [] middle = Bytes.toBytes("CCC");
 byte [] highest = Bytes.toBytes("EEE");
 byte [][] parts = Bytes.split(lowest, highest, 1);
 for (int i = 0; i < parts.length; i++) {
  System.out.println(Bytes.toString(parts[i]));
 }
 assertEquals(3, parts.length);
 assertTrue(Bytes.equals(parts[1], middle));
 // Now divide into three parts.  Change highest so split is even.
 highest = Bytes.toBytes("DDD");
 parts = Bytes.split(lowest, highest, 2);
 for (int i = 0; i < parts.length; i++) {
  System.out.println(Bytes.toString(parts[i]));
 }
 assertEquals(4, parts.length);
 // Assert that 3rd part is 'CCC'.
 assertTrue(Bytes.equals(parts[2], middle));
}

代码示例来源：origin: apache/hbase

public void testSplit3() throws Exception {
 // Test invalid split cases
 byte [] low = { 1, 1, 1 };
 byte [] high = { 1, 1, 3 };
 // If swapped, should throw IAE
 try {
  Bytes.split(high, low, 1);
  assertTrue("Should not be able to split if low > high", false);
 } catch(IllegalArgumentException iae) {
  // Correct
 }
 // Single split should work
 byte [][] parts = Bytes.split(low, high, 1);
 for (int i = 0; i < parts.length; i++) {
  System.out.println("" + i + " -> " + Bytes.toStringBinary(parts[i]));
 }
 assertTrue("Returned split should have 3 parts but has " + parts.length, parts.length == 3);
 // If split more than once, use additional byte to split
 parts = Bytes.split(low, high, 2);
 assertTrue("Split with an additional byte", parts != null);
 assertEquals(parts.length, low.length + 1);
 // Split 0 times should throw IAE
 try {
  parts = Bytes.split(low, high, 0);
  assertTrue("Should not be able to split 0 times", false);
 } catch(IllegalArgumentException iae) {
  // Correct
 }
}

代码示例来源：origin: apache/hbase

byte[][] splitKeys = Bytes.split(startRow, endRow, true, n-1);
for (int i = 0; i < splitKeys.length - 1; i++) {

代码示例来源：origin: apache/hbase

private void splitRegion(final RegionInfo regionInfo) throws IOException {
 byte[][] splitPoints = Bytes.split(regionInfo.getStartKey(), regionInfo.getEndKey(), 1);
 admin.split(regionInfo.getTable(), splitPoints[1]);
}

代码示例来源：origin: apache/hbase

protected final void splitRegion(RegionInfo regionInfo) throws IOException {
 byte[][] splitPoints = Bytes.split(regionInfo.getStartKey(), regionInfo.getEndKey(), 1);
 admin.split(regionInfo.getTable(), splitPoints[1]);
}

代码示例来源：origin: larsgeorge/hbase-book

public static void main(String[] args) {
  // vv BytesSplit
  byte[][] splits = Bytes.split(Bytes.toBytes(0), Bytes.toBytes(100), 9); // co BytesSplit-1-9Splits The number defines the amount of splits performed. Splitting one region nine times results in ten parts.
  int n = 0;
  for (byte[] split : splits) {
   System.out.println("Split key[" + ++n + "]: " +
    Bytes.toInt(split));
  }
  // ^^ BytesSplit
 }
}

代码示例来源：origin: apache/hbase

.setColumnFamily(ColumnFamilyDescriptorBuilder.newBuilder(Bytes.toBytes(cf)).build())
  .setReplicationScope(HConstants.REPLICATION_SCOPE_LOCAL).build();
UTIL.createTable(tableDescriptor, Bytes.split(Bytes.toBytes(1), Bytes.toBytes(256), 123));
assertTrue(UTIL.getAdmin().getRegions(tableName).size() > 0);
for (RegionInfo region : UTIL.getAdmin().getRegions(tableName)) {

代码示例来源：origin: apache/phoenix

@Test
public void testSplitBytes() {
  byte[] startRow = Bytes.toBytes("EA");
  byte[] stopRow = Bytes.toBytes("EZ");
  byte[][] splitPoints = Bytes.split(startRow, stopRow, 10);
  for (byte[] splitPoint : splitPoints) {
    assertTrue(Bytes.toStringBinary(splitPoint), Bytes.compareTo(startRow, splitPoint) <= 0);
    assertTrue(Bytes.toStringBinary(splitPoint), Bytes.compareTo(stopRow, splitPoint) >= 0);
  }
}

代码示例来源：origin: forcedotcom/phoenix

@Test
public void testSplitBytes() {
  byte[] startRow = Bytes.toBytes("EA");
  byte[] stopRow = Bytes.toBytes("EZ");
  byte[][] splitPoints = Bytes.split(startRow, stopRow, 10);
  for (byte[] splitPoint : splitPoints) {
    assertTrue(Bytes.toStringBinary(splitPoint), Bytes.compareTo(startRow, splitPoint) <= 0);
    assertTrue(Bytes.toStringBinary(splitPoint), Bytes.compareTo(stopRow, splitPoint) >= 0);
  }
}

内容来源于网络，如有侵权，请联系作者删除！

Bytes

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多

    Bytes类方法

    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com