com.google.common.hash.Hashing.consistentHash()方法的使用及代码示例

x33g5p2x  于2022-01-19 转载在 其他  
字(8.7k)|赞(0)|评价(0)|浏览(232)

本文整理了Java中com.google.common.hash.Hashing.consistentHash()方法的一些代码示例,展示了Hashing.consistentHash()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。Hashing.consistentHash()方法的具体详情如下:
包路径:com.google.common.hash.Hashing
类名称:Hashing
方法名:consistentHash

Hashing.consistentHash介绍

[英]Assigns to input a "bucket" in the range [0, buckets), in a uniform manner that minimizes the need for remapping as buckets grows. That is, consistentHash(h, equals:

  • n - 1, with approximate probability 1/n
  • consistentHash(h, n - 1), otherwise (probability 1 - 1/n)

This method is suitable for the common use case of dividing work among buckets that meet the following conditions:

  • You want to assign the same fraction of inputs to each bucket.
  • When you reduce the number of buckets, you can accept that the most recently added buckets will be removed first. More concretely, if you are dividing traffic among tasks, you can decrease the number of tasks from 15 and 10, killing off the final 5 tasks, and consistentHash will handle it. If, however, you are dividing traffic among servers alpha, bravo, and charlie and you occasionally need to take each of the servers offline, consistentHash will be a poor fit: It provides no way for you to specify which of the three buckets is disappearing. Thus, if your buckets change from [alpha, bravo, charlie] to [bravo, charlie], it will assign all the old alpha traffic to bravo and all the old bravotraffic to charlie, rather than letting bravo keep its traffic.

See the Wikipedia article on consistent hashing for more information.
[中]在[0,bucket]范围内指定输入一个“bucket”,以统一的方式,最大限度地减少在bucket增长时重新映射的需要。即,consistentHash(h,等于:
*n-1,近似概率为1/n
*一致散列(h,n-1),否则(概率1-1/n)
此方法适用于满足以下条件的铲斗之间分工的常见用例:
*您希望为每个存储桶分配相同比例的输入。
*当您减少存储桶的数量时,您可以接受将首先删除最近添加的存储桶。更具体地说,如果在任务之间划分流量,可以将任务数从15个和10个减少到最后5个任务,consistentHash将处理它。但是,如果在服务器alpha、bravo和charlie之间分配流量,并且偶尔需要使每个服务器脱机,那么consistentHash将不适合:它无法指定三个存储桶中的哪一个正在消失。因此,如果您的存储桶从[alpha,bravo,charlie]更改为[bravo,charlie],它将把所有旧alpha流量分配给bravo,并将所有旧bravo流量分配给charlie,而不是让bravo保留其流量。
有关更多信息,请参见Wikipedia article on consistent hashing

代码示例

代码示例来源:origin: google/guava

return consistentHash(hashCode.padToLong(), buckets);

代码示例来源:origin: google/guava

public void checkSameResult(HashCode hashCode, long equivLong) {
 assertEquals(Hashing.consistentHash(equivLong, 5555), Hashing.consistentHash(hashCode, 5555));
}

代码示例来源:origin: apache/nifi

@Override
public QueuePartition getPartition(final FlowFileRecord flowFile, final QueuePartition[] partitions,  final QueuePartition localPartition) {
  final int hash = hash(flowFile);
  // The consistentHash method appears to always return a bucket of '1' if there are 2 possible buckets,
  // so in this case we will just use modulo division to avoid this. I suspect this is a bug with the Guava
  // implementation, but it's not clear at this point.
  final int index;
  if (partitions.length < 3) {
    index = hash % partitions.length;
  } else {
    index = Hashing.consistentHash(hash, partitions.length);
  }
  return partitions[index];
}

代码示例来源:origin: google/guava

private void checkConsistentHashCorrectness(long hashCode) {
 int last = 0;
 for (int shards = 1; shards <= 100000; shards++) {
  int b = Hashing.consistentHash(hashCode, shards);
  if (b != last) {
   assertEquals(shards - 1, b);
   last = b;
  }
 }
}

代码示例来源:origin: google/guava

public void testConsistentHash_outOfRange() {
 try {
  Hashing.consistentHash(5L, 0);
  fail();
 } catch (IllegalArgumentException expected) {
 }
}

代码示例来源:origin: google/guava

private void countRemaps(long h, AtomicLongMap<Integer> map) {
 int last = 0;
 for (int shards = 2; shards <= MAX_SHARDS; shards++) {
  int chosen = Hashing.consistentHash(h, shards);
  if (chosen != last) {
   map.incrementAndGet(shards);
   last = chosen;
  }
 }
}

代码示例来源:origin: google/j2objc

return consistentHash(hashCode.padToLong(), buckets);

代码示例来源:origin: atomix/atomix

@Override
 public PartitionId partition(String key, List<PartitionId> partitions) {
  int hash = Math.abs(Hashing.murmur3_32().hashUnencodedChars(key).asInt());
  return partitions.get(Hashing.consistentHash(hash, partitions.size()));
 }
}

代码示例来源:origin: google/guava

/**
 * Check a few "golden" values to see that implementations across languages are equivalent.
 *
 */
public void testConsistentHash_linearCongruentialGeneratorCompatibility() {
 int[] golden100 = {
  0, 55, 62, 8, 45, 59, 86, 97, 82, 59,
  73, 37, 17, 56, 86, 21, 90, 37, 38, 83
 };
 for (int i = 0; i < golden100.length; i++) {
  assertEquals(golden100[i], Hashing.consistentHash(i, 100));
 }
 assertEquals(6, Hashing.consistentHash(10863919174838991L, 11));
 assertEquals(3, Hashing.consistentHash(2016238256797177309L, 11));
 assertEquals(5, Hashing.consistentHash(1673758223894951030L, 11));
 assertEquals(80343, Hashing.consistentHash(2, 100001));
 assertEquals(22152, Hashing.consistentHash(2201, 100001));
 assertEquals(15018, Hashing.consistentHash(2202, 100001));
}

代码示例来源:origin: wildfly/wildfly

return consistentHash(hashCode.padToLong(), buckets);

代码示例来源:origin: line/armeria

@Override
  public Endpoint select(ClientRequestContext ctx) {
    final List<Endpoint> endpoints = endpointGroup.endpoints();
    if (endpoints.isEmpty()) {
      throw new EndpointGroupException(endpointGroup + " is empty");
    }
    final long key = requestContextHasher.applyAsLong(ctx);
    final int nearest = Hashing.consistentHash(key, endpoints.size());
    return endpoints.get(nearest);
  }
}

代码示例来源:origin: apache/hive

@VisibleForTesting
public static int determineLocation(
  List<String> locations, String path, long start, String desc) {
 byte[] bytes = getHashInputForSplit(path, start);
 long hash1 = hash1(bytes);
 int index = Hashing.consistentHash(hash1, locations.size());
 String location = locations.get(index);
 if (LOG.isDebugEnabled()) {
  LOG.debug(desc + " mapped to index=" + index + ", location=" + location);
 }
 int iter = 1;
 long hash2 = 0;
 // Since our probing method is totally bogus, give up after some time.
 while (location == null && iter < locations.size() * 2) {
  if (iter == 1) {
   hash2 = hash2(bytes);
  }
  // Note that this is not real double hashing since we have consistent hash on top.
  index = Hashing.consistentHash(hash1 + iter * hash2, locations.size());
  location = locations.get(index);
  if (LOG.isDebugEnabled()) {
   LOG.debug(desc + " remapped to index=" + index + ", location=" + location);
  }
  ++iter;
 }
 return index;
}

代码示例来源:origin: QNJR-GROUP/EasyTransaction

@Override
  public Server choose(Object key) {
    
    //使用一致性哈希进行分发
    //TODO 此处使用了GUAVA中简单的一致性哈希算法选择服务,但这里存在性能缺陷:当reachableServers中间某一个服务节点失效了
    //那么后续节点的一致性哈希结果将会不匹配,后续需要使用更完善的哈希环 加上 虚拟节点 的形式解决本问题
    List<Server> reachableServers = getLoadBalancer().getReachableServers();
    if(reachableServers != null && reachableServers.size() != 0){
      int serverSeq = Hashing.consistentHash(Thread.currentThread().getId(), reachableServers.size());
      return reachableServers.get(serverSeq);
    } else {
      return super.choose(key);
    }
  }
}

代码示例来源:origin: apache/usergrid

/**
   * Locate the bucket number given the value, the funnel and the total buckets.
   *
   * Assigns to {@code hashCode} a "bucket" in the range {@code [0, buckets)}, in a uniform manner that minimizes the
   * need for remapping as {@code buckets} grows. That is, {@code consistentHash(h, n)} equals:
   *
   * <ul> <li>{@code n - 1}, with approximate probability {@code 1/n} <li>{@code consistentHash(h, n - 1)}, otherwise
   * (probability {@code 1 - 1/n}) </ul>
   *
   * <p>See the <a href="http://en.wikipedia.org/wiki/Consistent_hashing">wikipedia article on consistent hashing</a>
   * for more information.
   *
   * <p>See <a href="http://arxiv.org/pdf/1406.2294v1.pdf">this paper</a> for more details on the algorithm</p>
   *
   *
   * Note that after testing, increasing buckets does NOT yield the expected results.  You will need an algorithm
   * that manually walks a tree.  See
   *
   */
  public int getBucket( T value ) {

    final HashCode hashCode = HASHER.hashObject( value, funnel );

    int owningIndex = Hashing.consistentHash( hashCode, totalBuckets );

    return owningIndex;
  }
}

代码示例来源:origin: io.github.luzzu/luzzu-ld-qualitymetrics-commons

public int[] getSetBitLocations(String strItem) {
    // Hash the item through the k hashing functions
    int[] arrItemHashings = new int[this.k];
    
    for(int i = 0; i < this.k; i++) {
      arrItemHashings[i] = Hashing.consistentHash(this.arrHashFunctions[i].hashUnencodedChars(strItem), this.bitSetSize);
    }
    
    return arrItemHashings;
  }
}

代码示例来源:origin: soabase/soabase

public static void main(String[] args)
{
  String id = UUID.randomUUID().toString();
  for ( int i = 1; i <= 100; ++i )
  {
    System.out.println(Hashing.consistentHash(Hashing.sha256().hashString(id, Charsets.UTF_8), i));
  }
}

代码示例来源:origin: pravega/pravega

public int hashToBucket(UUID uuid, int numBuckets) {
  return Hashing.consistentHash(
      Hashing.combineOrdered(Arrays.asList(hash.hashLong(uuid.getMostSignificantBits()), hash.hashLong(uuid.getLeastSignificantBits()))),
      numBuckets);
}

代码示例来源:origin: com.google.guava/guava-tests

private void checkConsistentHashCorrectness(long hashCode) {
 int last = 0;
 for (int shards = 1; shards <= 100000; shards++) {
  int b = Hashing.consistentHash(hashCode, shards);
  if (b != last) {
   assertEquals(shards - 1, b);
   last = b;
  }
 }
}

代码示例来源:origin: com.google.guava/guava-tests

private void countRemaps(long h, AtomicLongMap<Integer> map) {
 int last = 0;
 for (int shards = 2; shards <= MAX_SHARDS; shards++) {
  int chosen = Hashing.consistentHash(h, shards);
  if (chosen != last) {
   map.incrementAndGet(shards);
   last = chosen;
  }
 }
}

代码示例来源:origin: com.google.guava/guava-tests

public void testConsistentHash_outOfRange() {
 try {
  Hashing.consistentHash(5L, 0);
  fail();
 } catch (IllegalArgumentException expected) {
 }
}

相关文章