de.tudarmstadt.ukp.dkpro.core.api.segmentation.type.Token.getPos()方法的使用及代码示例

x33g5p2x  于2022-01-29 转载在 其他  
字(13.0k)|赞(0)|评价(0)|浏览(121)

本文整理了Java中de.tudarmstadt.ukp.dkpro.core.api.segmentation.type.Token.getPos()方法的一些代码示例，展示了Token.getPos()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Token.getPos()方法的具体详情如下：
包路径：de.tudarmstadt.ukp.dkpro.core.api.segmentation.type.Token
类名称：Token
方法名：getPos

Token.getPos介绍

[英]getter for pos - gets
[中]

代码示例

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.core/de.tudarmstadt.ukp.dkpro.core.api.segmentation-asl

/**
 * @return the POS value if there is a {@link POS} annotation linked to this token.
 */
public String getPosValue()
{
  POS pos = getPos();
  return pos != null ? pos.getPosValue() : null;
}

代码示例来源：origin: dkpro/dkpro-core

/**
 * @return the POS value if there is a {@link POS} annotation linked to this token.
 */
public String getPosValue()
{
  POS pos = getPos();
  return pos != null ? pos.getPosValue() : null;
}

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.core/de.tudarmstadt.ukp.dkpro.core.arktools-gpl

@Override
public void process(JCas jCas) throws AnalysisEngineProcessException {
  if (tempData == null) {
    try {
      tempData = File.createTempFile("dkpro-arktweet-pos-trainer", ".tsv");
      out = new PrintWriter(new OutputStreamWriter(new FileOutputStream(tempData),
          StandardCharsets.UTF_8));
    }
    catch (IOException e) {
      throw new AnalysisEngineProcessException(e);
    }
  }
  Map<Sentence, Collection<Token>> index = indexCovered(jCas, Sentence.class, Token.class);
  for (Sentence sentence : select(jCas, Sentence.class)) {
    Collection<Token> tokens = index.get(sentence);
    for (Token token : tokens) {
      out.printf("%s\t%s%n", token.getText(), token.getPos().getPosValue());
    }
    out.println();
  }
}

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.core/de.tudarmstadt.ukp.dkpro.core.stanfordnlp-gpl

@Override
public void process(JCas aJCas)
  throws AnalysisEngineProcessException
{
  if (tempData == null) {
    try {
      tempData = File.createTempFile("dkpro-stanford-pos-trainer", ".tsv");
      out = new PrintWriter(new OutputStreamWriter(new FileOutputStream(tempData),
          StandardCharsets.UTF_8));
    }
    catch (IOException e) {
      throw new AnalysisEngineProcessException(e);
    }
  }
  
  Map<Sentence, Collection<Token>> index = indexCovered(aJCas, Sentence.class, Token.class);
  for (Sentence sentence : select(aJCas, Sentence.class)) {
    Collection<Token> tokens = index.get(sentence);
    for (Token token : tokens) {
      out.printf("%s\t%s%n", token.getText(), token.getPos().getPosValue());
    }
    out.println();
  }
}

代码示例来源：origin: dkpro/dkpro-core

@Override
public boolean check(JCas aJCas, List<Message> aMessages)
{
  List<Token> withoutPOS = select(aJCas, Token.class).stream()
      .filter(t -> t.getPos() == null)
      .collect(Collectors.toList());
  
  for (Token t : withoutPOS) {
    aMessages.add(new Message(this, ERROR, String.format("Token has no POS: %s [%d..%d]", t
        .getType().getName(), t.getBegin(), t.getEnd())));
  }
  List<Token> withoutPOSValue = select(aJCas, Token.class).stream()
      .filter(t -> t.getPos() != null && t.getPos().getPosValue() == null)
      .collect(Collectors.toList());
  
  for (Token t : withoutPOSValue) {
    aMessages.add(new Message(this, ERROR, String.format(
        "Token has no POS value: %s [%d..%d]", t.getType().getName(), t.getBegin(),
        t.getEnd())));
  }
  return aMessages.stream().anyMatch(m -> m.level == ERROR);
}

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.core/de.tudarmstadt.ukp.dkpro.core.testing-asl

@Override
public boolean check(JCas aJCas, List<Message> aMessages)
{
  List<Token> withoutPOS = select(aJCas, Token.class).stream()
      .filter(t -> t.getPos() == null)
      .collect(Collectors.toList());
  
  for (Token t : withoutPOS) {
    aMessages.add(new Message(this, ERROR, String.format("Token has no POS: %s [%d..%d]", t
        .getType().getName(), t.getBegin(), t.getEnd())));
  }
  List<Token> withoutPOSValue = select(aJCas, Token.class).stream()
      .filter(t -> t.getPos() != null && t.getPos().getPosValue() == null)
      .collect(Collectors.toList());
  
  for (Token t : withoutPOSValue) {
    aMessages.add(new Message(this, ERROR, String.format(
        "Token has no POS value: %s [%d..%d]", t.getType().getName(), t.getBegin(),
        t.getEnd())));
  }
  return aMessages.stream().anyMatch(m -> m.level == ERROR);
}

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.core/de.tudarmstadt.ukp.dkpro.core.opennlp-asl

@Override
  public POSSample produce(JCas aJCas)
  {
    // Process present sentences
    Sentence sentence = sentences.next();
    
    // Block on next call to read
    if (!sentences.hasNext()) {
      documentComplete();
    }
    
    List<String> words = new ArrayList<>();
    List<String> tags = new ArrayList<>();
    
    for (Token t : selectCovered(Token.class, sentence)) {
      words.add(t.getText());
      if (t.getPos() == null) {
        throw new IllegalStateException("Token [" + t.getText() + "] has no POS");
      }
      tags.add(t.getPos().getPosValue());
    }
    
    return new POSSample(words, tags);
  }
}

代码示例来源：origin: hltfbk/Excitement-Open-Platform

protected Map<String, Integer> countTokenPoses(JCas text) {
  Map<String, Integer> tokenNumMap = new HashMap<String, Integer>();
  Iterator<Annotation> tokenIter = text.getAnnotationIndex(Token.type)
      .iterator();
  while (tokenIter.hasNext()) {
    Token curr = (Token) tokenIter.next();
    String tokenText = curr.getLemma().getValue().replace("#", "\\#")
        + " ### " + curr.getPos().getPosValue();
    Integer num = tokenNumMap.get(tokenText);
    if (null == num) {
      tokenNumMap.put(tokenText, 1);
    } else {
      tokenNumMap.put(tokenText, num + 1);
    }
  }
  return tokenNumMap;
}

代码示例来源：origin: dkpro/dkpro-similarity

public List<String> getSubstitutions(JCas jcas)
{
  List<String> tokens = new ArrayList<String>();
  List<String> postags = new ArrayList<String>();;
  
  for (Token t : JCasUtil.select(jcas, Token.class))
  {
    try
    {
      tokens.add(t.getLemma().getValue().toLowerCase());
      postags.add(t.getPos().getPosValue());
    }
    catch (NullPointerException e) {
      System.err.println("Couldn't read lemma value for token \"" + t.getCoveredText() + "\"");
    }
  }
  
  return getSubstitutions(tokens, postags);
}

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.core/de.tudarmstadt.ukp.dkpro.core.opennlp-asl

@Override
  public ChunkSample produce(JCas aJCas)
  {
    // Process present sentences
    Sentence sentence = sentences.next();
    
    // Block on next call to read
    if (!sentences.hasNext()) {
      documentComplete();
    }
    
    List<String> words = new ArrayList<>();
    List<String> tags = new ArrayList<>();
    List<String> preds = new ArrayList<>();
    
    for (Token t : selectCovered(Token.class, sentence)) {
      words.add(t.getText());
      if (t.getPos() == null) {
        throw new IllegalStateException("Token [" + t.getText() + "] has no POS");
      }
      tags.add(t.getPos().getPosValue());
      preds.add(chunkEncoder.encode(t));
    }
    
    return new ChunkSample(words, tags, preds);
  }
}

代码示例来源：origin: dkpro/dkpro-similarity

public List<String> getSubstitutions(JCas jcas, Annotation coveringAnnotation)
{
  List<String> tokens = new ArrayList<String>();
  List<String> postags = new ArrayList<String>();;
  
  for (Token t : JCasUtil.selectCovered(jcas, Token.class, coveringAnnotation))
  {
    try
    {
      tokens.add(t.getLemma().getValue().toLowerCase());
      postags.add(t.getPos().getPosValue());
    }
    catch (NullPointerException e) {
      System.err.println("Couldn't read lemma value for token \"" + t.getCoveredText() + "\"");
    }
  }
  
  return getSubstitutions(tokens, postags);
}

代码示例来源：origin: hltfbk/Excitement-Open-Platform

private void updateEntitiesAndRelations(Collection<AnnotationFS> col, int Identication) {
  for (AnnotationFS annotationFS : col) {
    Type type = annotationFS.getType();
    String typeShortName = type.getShortName();
    
    if(typeShortName.equals("Dependency"))
    {
      
      Token governor = ((Dependency)annotationFS).getGovernor();
      Token dependent = ((Dependency)annotationFS).getDependent();
      String dependencyType = ((Dependency)annotationFS).getDependencyType();
                
      //tmp
      //System.out.println(governor.getLemma().getValue() + "--" + dependencyType + "--> " + dependent.getLemma().getValue());
      
      String strGovernorId="T"+(governor.getBegin()+Identication)+"S"+(governor.getEnd()+Identication);
      String strDependentId="T"+(dependent.getBegin()+Identication)+"S"+(dependent.getEnd()+Identication);
      
      hashPOS.put(strGovernorId, governor.getPos().getPosValue());
      hashPOS.put(strDependentId, dependent.getPos().getPosValue());
      if(!hashRel.keySet().contains(strGovernorId))
        hashRel.put(strGovernorId, new HashMap<String, String>());
      
      hashRel.get(strGovernorId).put(strDependentId, dependencyType);
      
    }
  }
  
}

代码示例来源：origin: dkpro/dkpro-tc

@Override
public void process(JCas aJCas) throws AnalysisEngineProcessException
{
  if (embedding == null) {
    return;
  }
  Collection<Token> select = JCasUtil.select(aJCas, Token.class);
  for (Token t : select) {
    if (vocab.contains(t.getCoveredText())) {
      continue;
    }
    POS pos = t.getPos();
    if (pos != null) {
      pos.removeFromIndexes();
      t.setPos(null);
    }
    t.removeFromIndexes();
    droppedVocabulary++;
  }
}

代码示例来源：origin: hltfbk/Excitement-Open-Platform

protected Map<String, String> indexLemmaDepTree(JCas text) {
    Map<String, String> depTree = new HashMap<String, String>();

    for (Dependency dep : JCasUtil.select(text, Dependency.class)) {
      Token child = dep.getDependent();
      Token parent = dep.getGovernor();
      depTree.put(child.getBegin() + " ### "
          + child.getLemma().getValue().replace("#", "\\#") + " ### "
          + child.getPos().getPosValue(), dep.getDependencyType()
          + " ## " + parent.getBegin() + " ### "
          + parent.getLemma().getValue().replace("#", "\\#")
          + " ### " + parent.getPos().getPosValue());
    }

    return depTree;
  }
}

代码示例来源：origin: org.dkpro.tc/dkpro-tc-core

@Override
public void process(JCas aJCas) throws AnalysisEngineProcessException
{
  if (embedding == null) {
    return;
  }
  Collection<Token> select = JCasUtil.select(aJCas, Token.class);
  for (Token t : select) {
    if (vocab.contains(t.getCoveredText())) {
      continue;
    }
    POS pos = t.getPos();
    if (pos != null) {
      pos.removeFromIndexes();
      t.setPos(null);
    }
    t.removeFromIndexes();
    droppedVocabulary++;
  }
}

代码示例来源：origin: hltfbk/Excitement-Open-Platform

protected Map<String, String> indexDepTree(JCas text) {
  Map<String, String> depTree = new HashMap<String, String>();
  // format: key: 1 ### word ### pos; value: dep_rel ## 2 ### word ### pos
  // escape: .replace("#", "\\#")
  // depTree.put("1 ### The ### Det", "DET ## 2 ### dog ### N");
  // depTree.put("2 ### dog ### N", "SUBJ ## 3 ### chases ### V");
  // depTree.put("3 ### chases ### V", "ROOT ## 0 ### NULL ### NULL");
  // depTree.put("4 ### The ### Det", "DET ## 5 ### cat ### N");
  // depTree.put("5 ### cat ### N", "OBJ ## 3 ### chases ### V");
  for (Dependency dep : JCasUtil.select(text, Dependency.class)) {
    Token child = dep.getDependent();
    Token parent = dep.getGovernor();
    depTree.put(child.getBegin() + " ### "
        + child.getCoveredText().replace("#", "\\#") + " ### "
        + child.getPos().getPosValue(), dep.getDependencyType()
        + " ## " + parent.getBegin() + " ### "
        + parent.getCoveredText().replace("#", "\\#") + " ### "
        + parent.getPos().getPosValue());
  }
  return depTree;
}

代码示例来源：origin: webanno/webanno

@SuppressWarnings("unused")
private boolean hasPos(FeatureStructure fs, String posValue)
{
  if (fs instanceof POS) {
    POS pos = (POS) fs;
    if (pos.getPosValue().equals(posValue)) {
      return true;
    }
  }
  else if (fs instanceof Token) {
    Token token = (Token) fs;
    if (token.getPos().getPosValue().equals(posValue)) {
      return true;
    }
  }
  return false;
}

代码示例来源：origin: hltfbk/Excitement-Open-Platform

private static NodeInfo buildNodeInfo(JCas jcas, Token tokenAnno, int serial) throws CasTreeConverterException, UnsupportedPosTagStringException {
  String word = tokenAnno.getCoveredText();
  String lemma = tokenAnno.getLemma().getValue();
  String pos = tokenAnno.getPos().getPosValue();
  
  // We rely on the fact the NamedEntity enum values have the same names as the ones
  // specified in the DKPro mapping (e.g. PERSON, ORGANIZATION)
  eu.excitementproject.eop.common.representation.parse.representation.basic.NamedEntity namedEntity=null;
  List<NamedEntity> namedEntities = JCasUtil.selectCovered(NamedEntity.class, tokenAnno);
  switch (namedEntities.size()) {
  case 0: break; // if no NER - ignore and move on
  case 1: namedEntity = eu.excitementproject.eop.common.representation.parse.representation.basic.NamedEntity.valueOf(namedEntities.get(0).getValue());
      break;
  default: throw new CasTreeConverterException(String.format("Got %d NamedEntity annotations for token %s", namedEntities.size(), tokenAnno));
  }
      
  return new DefaultNodeInfo(word, lemma, serial, namedEntity, new DefaultSyntacticInfo(new PennPartOfSpeech(pos)));
}

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.keyphrases/de.tudarmstadt.ukp.dkpro.keyphrases.decompounding-asl

@Override
public void process(final JCas aJCas)
  throws AnalysisEngineProcessException
{
  Token token;
  for (Compound compound : JCasUtil.select(aJCas, Compound.class)) {
    final Token compoundToken = JCasUtil.selectCovered(aJCas, Token.class,
        compound.getBegin(), compound.getEnd()).get(0);
    for (Split compoundPart : compound.getSplitsWithoutMorpheme(compoundSplitLevel)) {
      token = new Token(aJCas);
      token.setBegin(compoundPart.getBegin());
      token.setEnd(compoundPart.getEnd());
      token.setPos(compoundToken.getPos());
      token.addToIndexes();
    }
  }
}

代码示例来源：origin: de.tudarmstadt.ukp.dkpro.core/de.tudarmstadt.ukp.dkpro.core.stanfordnlp-gpl

public static CoreLabel tokenToWord(Token aToken)
{
  CoreLabel t = new CoreLabel();
  
  t.setOriginalText(aToken.getCoveredText());
  t.setWord(aToken.getText());
  t.setBeginPosition(aToken.getBegin());
  t.setEndPosition(aToken.getEnd());
  
  if (aToken.getLemma() != null) {
    t.setLemma(aToken.getLemma().getValue());
  }
  else {
    t.setLemma(aToken.getText());
  }
  
  if (aToken.getPos() != null) {
    t.setTag(aToken.getPos().getPosValue());
  }
  
  return t;
}

内容来源于网络，如有侵权，请联系作者删除！

token

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com