org.apache.pig.data.BagFactory.newSortedBag()方法的使用及代码示例

x33g5p2x  于2022-01-17 转载在 其他  
字(3.1k)|赞(0)|评价(0)|浏览(113)

本文整理了Java中org.apache.pig.data.BagFactory.newSortedBag()方法的一些代码示例,展示了BagFactory.newSortedBag()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。BagFactory.newSortedBag()方法的具体详情如下:
包路径:org.apache.pig.data.BagFactory
类名称:BagFactory
方法名:newSortedBag

BagFactory.newSortedBag介绍

[英]Get a sorted data bag. Sorted bags guarantee that when an iterator is opened on the bag the tuples will be returned in sorted order.
[中]得到一个分类的数据包。排序包保证当在包上打开迭代器时,元组将按排序顺序返回。

代码示例

代码示例来源:origin: org.apache.pig/pig

thisClone = factory.newSortedBag(null);
Iterator<Tuple> i = iterator();
while (i.hasNext()) thisClone.add(i.next());
  otherClone = bOther;
} else {
  otherClone = factory.newSortedBag(null);
  i = bOther.iterator();
  while (i.hasNext()) otherClone.add(i.next());

代码示例来源:origin: org.apache.pig/pig

sortedBag = useDefaultBag ? mBagFactory.newSortedBag(mComparator)
    : new InternalSortedBag(3, mComparator);

代码示例来源:origin: org.apache.pig/pig

thisClone = this;
} else {
  thisClone = factory.newSortedBag(null);
  Iterator<Tuple> i = iterator();
  while (i.hasNext()) thisClone.add(i.next());
  otherClone = bOther;
} else {
  otherClone = factory.newSortedBag(null);
  Iterator<Tuple> i = bOther.iterator();
  while (i.hasNext()) otherClone.add(i.next());

代码示例来源:origin: com.linkedin.datafu/datafu

@Override
 public DataBag exec(Tuple tuple) throws IOException
 {
  DataBag candidates = bagFactory.newSortedBag(CandidateComparator.get());
  for (Tuple intermediateOutputTuple : (DataBag) tuple.get(0))
  {
   candidates.addAll((DataBag) intermediateOutputTuple.get(0));
  }
  DataBag outputBag = bagFactory.newDefaultBag();
  int i = -1;
  for (Tuple candidate : candidates)
  {
   int pos = (Integer) candidate.get(0);
   if (pos > i)
   {
    outputBag.add((Tuple) candidate.get(2));
    i = pos;
   }
  }
  return outputBag;
 }
}

代码示例来源:origin: com.linkedin.datafu/datafu

@Override
public Tuple exec(Tuple tuple) throws IOException
{
 // sort candidates first by index, then by key
 DataBag candidates = bagFactory.newSortedBag(CandidateComparator.get());
 for (Tuple intermediateOutputTuple : (DataBag) tuple.get(0))
 {
  candidates.addAll((DataBag) intermediateOutputTuple.get(0));
 }
 DataBag outputBag = bagFactory.newDefaultBag();
 int i = -1;
 for (Tuple candidate : candidates)
 {
  int pos = (Integer) candidate.get(0);
  if (pos > i)
  {
   outputBag.add(candidate);
   i = pos;
  }
 }
 return tupleFactory.newTuple(outputBag);
}

代码示例来源:origin: com.linkedin.datafu/datafu

@Override
 public DataBag exec(Tuple input) throws IOException
 {
  DataBag bag = (DataBag) input.get(0);
  long n = 0L;
  DataBag selected = bagFactory.newDefaultBag();
  DataBag waiting = bagFactory.newSortedBag(new ScoredTupleComparator());
  for (Tuple innerTuple : bag)
  {
   n += (Long) innerTuple.get(0);
   selected.addAll((DataBag) innerTuple.get(1));
   waiting.addAll((DataBag) innerTuple.get(2));
  }
  long sampleSize = (long) Math.ceil(_samplingProbability * n);
  long nNeeded = sampleSize - selected.size();
  for (Tuple scored : waiting)
  {
   if (nNeeded <= 0)
   {
    break;
   }
   selected.add(ScoredTuple.fromIntermediateTuple(scored).getTuple());
   nNeeded--;
  }
  return selected;
 }
}

代码示例来源:origin: com.linkedin.datafu/datafu

DataBag aggWaiting = bagFactory.newSortedBag(new ScoredTupleComparator());
DataBag waiting = bagFactory.newSortedBag(new ScoredTupleComparator());
Tuple output = tupleFactory.newTuple();

代码示例来源:origin: com.linkedin.datafu/datafu

DataBag waiting = bagFactory.newSortedBag(new ScoredTupleComparator());

相关文章

微信公众号

最新文章

更多