org.apache.kylin.common.util.Dictionary.write()方法的使用及代码示例

x33g5p2x  于2022-01-18 转载在 其他  
字(8.9k)|赞(0)|评价(0)|浏览(106)

本文整理了Java中org.apache.kylin.common.util.Dictionary.write()方法的一些代码示例，展示了Dictionary.write()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Dictionary.write()方法的具体详情如下：
包路径：org.apache.kylin.common.util.Dictionary
类名称：Dictionary
方法名：write

Dictionary.write介绍

[英]Serialize the fields of this object to out.
[中]将此对象的字段序列化为out。

代码示例

代码示例来源：origin: apache/kylin

public static void serialize(Dictionary<?> dict, OutputStream outputStream) {
  try {
    DataOutputStream out = new DataOutputStream(outputStream);
    out.writeUTF(dict.getClass().getName());
    dict.write(out);
    out.flush();
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
}

代码示例来源：origin: apache/kylin

public static ByteArray serialize(Dictionary<?> dict) {
  try {
    final ByteArrayOutputStream baos = new ByteArrayOutputStream();
    DataOutputStream out = new DataOutputStream(baos);
    out.writeUTF(dict.getClass().getName());
    dict.write(out);
    return new ByteArray(baos.toByteArray());
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
}

代码示例来源：origin: apache/kylin

private void outputDict(TblColRef col, Dictionary<String> dict) throws IOException, InterruptedException {
  // output written to baseDir/colName/colName.rldict-r-00000 (etc)
  String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;
  try (ByteArrayOutputStream baos = new ByteArrayOutputStream();
      DataOutputStream outputStream = new DataOutputStream(baos);) {
    outputStream.writeUTF(dict.getClass().getName());
    dict.write(outputStream);
    mos.write(BatchConstants.CFG_OUTPUT_DICT, NullWritable.get(),
        new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName);
  }
}

代码示例来源：origin: apache/kylin

@Override
public void serialize(DictionaryInfo obj, DataOutputStream out) throws IOException {
  String json = JsonUtil.writeValueAsIndentString(obj);
  out.writeUTF(json);
  if (infoOnly == false)
    obj.getDictionaryObject().write(out);
}

代码示例来源：origin: apache/kylin

void writeData(DataOutput out) throws IOException {
  out.writeInt(rowIndices.size());
  if (rowIndices.size() > 0) {
    int n = rowIndices.get(0).length;
    out.writeInt(n);
    if (this.useDictionary == true) {
      dict.write(out);
      for (int i = 0; i < rowIndices.size(); i++) {
        int[] row = rowIndices.get(i);
        for (int j = 0; j < n; j++) {
          out.writeInt(row[j]);
        }
      }
    } else {
      for (int i = 0; i < rowIndices.size(); i++) {
        int[] row = rowIndices.get(i);
        for (int j = 0; j < n; j++) {
          // NULL_STR is tricky, but we don't want to break the current snapshots
          out.writeUTF(dict.getValueFromId(row[j]) == null ? NULL_STR : dict.getValueFromId(row[j]));
        }
      }
    }
  }
}

代码示例来源：origin: apache/kylin

private void outputDict(TblColRef col, Dictionary<String> dict) throws IOException, InterruptedException {
    // output written to baseDir/colName/colName.rldict-r-00000 (etc)
    String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;

    try (ByteArrayOutputStream baos = new ByteArrayOutputStream(); DataOutputStream outputStream = new DataOutputStream(baos);) {
      outputStream.writeUTF(dict.getClass().getName());
      dict.write(outputStream);

      mos.write(BatchConstants.CFG_OUTPUT_DICT, NullWritable.get(), new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName);
    }
    mos.close();
  }
}

代码示例来源：origin: apache/kylin

private void outputDict(TblColRef col, Dictionary<String> dict,
    List<Tuple2<String, Tuple3<Writable, Writable, String>>> result)
    throws IOException {
  // output written to baseDir/colName/colName.rldict-r-00000 (etc)
  String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;
  try (ByteArrayOutputStream baos = new ByteArrayOutputStream();
      DataOutputStream outputStream = new DataOutputStream(baos)) {
    outputStream.writeUTF(dict.getClass().getName());
    dict.write(outputStream);
    result.add(new Tuple2<String, Tuple3<Writable, Writable, String>>(BatchConstants.CFG_OUTPUT_DICT,
        new Tuple3<Writable, Writable, String>(NullWritable.get(),
            new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName)));
  }
}

代码示例来源：origin: apache/kylin

shrunkenDict.write(dos);

代码示例来源：origin: apache/kylin

private void readWriteTest(Dictionary<String> dict) throws Exception {
    final String path = "src/test/resources/dict/tmp_dict";
    File f = new File(path);
    f.deleteOnExit();
    f.createNewFile();
    String dictClassName = dict.getClass().getName();
    DataOutputStream out = new DataOutputStream(new FileOutputStream(f));
    out.writeUTF(dictClassName);
    dict.write(out);
    out.close();
    //read dict
    DataInputStream in = null;
    Dictionary<String> dict2 = null;
    try {
      File f2 = new File(path);
      in = new DataInputStream(new FileInputStream(f2));
      String dictClassName2 = in.readUTF();
      dict2 = (Dictionary<String>) ClassUtil.newInstance(dictClassName2);
      dict2.readFields(in);
    } finally {
      if (in != null) {
        in.close();
      }
    }
    assertTrue(dict.equals(dict2));
  }
}

代码示例来源：origin: apache/kylin

DataOutputStream dos = new DataOutputStream(bos);
shrunkenDict.write(dos);

代码示例来源：origin: org.apache.kylin/kylin-core-dictionary

public static void serialize(Dictionary<?> dict, OutputStream outputStream) {
  try {
    DataOutputStream out = new DataOutputStream(outputStream);
    out.writeUTF(dict.getClass().getName());
    dict.write(out);
    out.flush();
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
}

代码示例来源：origin: org.apache.kylin/kylin-core-dictionary

public static ByteArray serialize(Dictionary<?> dict) {
  try {
    final ByteArrayOutputStream baos = new ByteArrayOutputStream();
    DataOutputStream out = new DataOutputStream(baos);
    out.writeUTF(dict.getClass().getName());
    dict.write(out);
    return new ByteArray(baos.toByteArray());
  } catch (IOException e) {
    throw new RuntimeException(e);
  }
}

代码示例来源：origin: org.apache.kylin/kylin-dictionary

@Override
public void serialize(DictionaryInfo obj, DataOutputStream out) throws IOException {
  String json = JsonUtil.writeValueAsIndentString(obj);
  out.writeUTF(json);
  if (infoOnly == false)
    obj.getDictionaryObject().write(out);
}

代码示例来源：origin: org.apache.kylin/kylin-core-dictionary

@Override
public void serialize(DictionaryInfo obj, DataOutputStream out) throws IOException {
  String json = JsonUtil.writeValueAsIndentString(obj);
  out.writeUTF(json);
  if (infoOnly == false)
    obj.getDictionaryObject().write(out);
}

代码示例来源：origin: org.apache.kylin/kylin-engine-mr

private void outputDict(TblColRef col, Dictionary<String> dict) throws IOException, InterruptedException {
  // output written to baseDir/colName/colName.rldict-r-00000 (etc)
  String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;
  try (ByteArrayOutputStream baos = new ByteArrayOutputStream();
      DataOutputStream outputStream = new DataOutputStream(baos);) {
    outputStream.writeUTF(dict.getClass().getName());
    dict.write(outputStream);
    mos.write(BatchConstants.CFG_OUTPUT_DICT, NullWritable.get(),
        new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName);
  }
}

代码示例来源：origin: org.apache.kylin/kylin-dictionary

void writeData(DataOutput out) throws IOException {
  out.writeInt(rowIndices.size());
  if (rowIndices.size() > 0) {
    int n = rowIndices.get(0).length;
    out.writeInt(n);
    if (this.useDictionary == true) {
      dict.write(out);
      for (int i = 0; i < rowIndices.size(); i++) {
        int[] row = rowIndices.get(i);
        for (int j = 0; j < n; j++) {
          out.writeInt(row[j]);
        }
      }
    } else {
      for (int i = 0; i < rowIndices.size(); i++) {
        int[] row = rowIndices.get(i);
        for (int j = 0; j < n; j++) {
          out.writeUTF(dict.getValueFromId(row[j]));
        }
      }
    }
  }
}

代码示例来源：origin: org.apache.kylin/kylin-core-dictionary

void writeData(DataOutput out) throws IOException {
  out.writeInt(rowIndices.size());
  if (rowIndices.size() > 0) {
    int n = rowIndices.get(0).length;
    out.writeInt(n);
    if (this.useDictionary == true) {
      dict.write(out);
      for (int i = 0; i < rowIndices.size(); i++) {
        int[] row = rowIndices.get(i);
        for (int j = 0; j < n; j++) {
          out.writeInt(row[j]);
        }
      }
    } else {
      for (int i = 0; i < rowIndices.size(); i++) {
        int[] row = rowIndices.get(i);
        for (int j = 0; j < n; j++) {
          // NULL_STR is tricky, but we don't want to break the current snapshots
          out.writeUTF(dict.getValueFromId(row[j]) == null ? NULL_STR : dict.getValueFromId(row[j]));
        }
      }
    }
  }
}

代码示例来源：origin: org.apache.kylin/kylin-engine-mr

private void outputDict(TblColRef col, Dictionary<String> dict) throws IOException, InterruptedException {
    // output written to baseDir/colName/colName.rldict-r-00000 (etc)
    String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;

    try (ByteArrayOutputStream baos = new ByteArrayOutputStream(); DataOutputStream outputStream = new DataOutputStream(baos);) {
      outputStream.writeUTF(dict.getClass().getName());
      dict.write(outputStream);

      mos.write(BatchConstants.CFG_OUTPUT_DICT, NullWritable.get(), new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName);
    }
    mos.close();
  }
}

代码示例来源：origin: org.apache.kylin/kylin-engine-spark

private void outputDict(TblColRef col, Dictionary<String> dict,
    List<Tuple2<String, Tuple3<Writable, Writable, String>>> result)
    throws IOException {
  // output written to baseDir/colName/colName.rldict-r-00000 (etc)
  String dictFileName = col.getIdentity() + "/" + col.getName() + DICT_FILE_POSTFIX;
  try (ByteArrayOutputStream baos = new ByteArrayOutputStream();
      DataOutputStream outputStream = new DataOutputStream(baos)) {
    outputStream.writeUTF(dict.getClass().getName());
    dict.write(outputStream);
    result.add(new Tuple2<String, Tuple3<Writable, Writable, String>>(BatchConstants.CFG_OUTPUT_DICT,
        new Tuple3<Writable, Writable, String>(NullWritable.get(),
            new ArrayPrimitiveWritable(baos.toByteArray()), dictFileName)));
  }
}

代码示例来源：origin: org.apache.kylin/kylin-engine-mr

shrunkenDict.write(dos);

内容来源于网络，如有侵权，请联系作者删除！

Dictionary

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com