org.apache.kafka.common.utils.Utils.utf8()方法的使用及代码示例

x33g5p2x  于2022-02-01 转载在 其他  
字(9.1k)|赞(0)|评价(0)|浏览(101)

本文整理了Java中org.apache.kafka.common.utils.Utils.utf8()方法的一些代码示例，展示了Utils.utf8()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台，是从一些精选项目中提取出来的代码，具有较强的参考意义，能在一定程度帮忙到你。Utils.utf8()方法的具体详情如下：
包路径：org.apache.kafka.common.utils.Utils
类名称：Utils
方法名：utf8

Utils.utf8介绍

[英]Turn a string into a utf8 byte[]
[中]将字符串转换为utf8字节[]

代码示例

代码示例来源：origin: apache/kafka

/**
 * Read a UTF8 string from a byte buffer. Note that the position of the byte buffer is not affected
 * by this method.
 *
 * @param buffer The buffer to read from
 * @param length The length of the string in bytes
 * @return The UTF8 string
 */
public static String utf8(ByteBuffer buffer, int length) {
  return utf8(buffer, 0, length);
}

代码示例来源：origin: apache/kafka

@Test
public void utf8ByteArraySerde() {
  String utf8String = "A\u00ea\u00f1\u00fcC";
  byte[] utf8Bytes = utf8String.getBytes(StandardCharsets.UTF_8);
  assertArrayEquals(utf8Bytes, Utils.utf8(utf8String));
  assertEquals(utf8Bytes.length, Utils.utf8Length(utf8String));
  assertEquals(utf8String, Utils.utf8(utf8Bytes));
}

代码示例来源：origin: apache/kafka

/**
 * Read a UTF8 string from the current position till the end of a byte buffer. The position of the byte buffer is
 * not affected by this method.
 *
 * @param buffer The buffer to read from
 * @return The UTF8 string
 */
public static String utf8(ByteBuffer buffer) {
  return utf8(buffer, buffer.remaining());
}

代码示例来源：origin: apache/kafka

@Override
public void write(ByteBuffer buffer, Object o) {
  if (o == null) {
    buffer.putShort((short) -1);
    return;
  }
  byte[] bytes = Utils.utf8((String) o);
  if (bytes.length > Short.MAX_VALUE)
    throw new SchemaException("String length " + bytes.length + " is larger than the maximum string length.");
  buffer.putShort((short) bytes.length);
  buffer.put(bytes);
}

代码示例来源：origin: apache/kafka

@Override
public String read(ByteBuffer buffer) {
  short length = buffer.getShort();
  if (length < 0)
    return null;
  if (length > buffer.remaining())
    throw new SchemaException("Error reading string of length " + length + ", only " + buffer.remaining() + " bytes available");
  String result = Utils.utf8(buffer, length);
  buffer.position(buffer.position() + length);
  return result;
}

代码示例来源：origin: apache/kafka

@Override
public void write(ByteBuffer buffer, Object o) {
  byte[] bytes = Utils.utf8((String) o);
  if (bytes.length > Short.MAX_VALUE)
    throw new SchemaException("String length " + bytes.length + " is larger than the maximum string length.");
  buffer.putShort((short) bytes.length);
  buffer.put(bytes);
}

代码示例来源：origin: apache/kafka

/**
 * Read a UTF8 string from a byte buffer at a given offset. Note that the position of the byte buffer
 * is not affected by this method.
 *
 * @param buffer The buffer to read from
 * @param offset The offset relative to the current position in the buffer
 * @param length The length of the string in bytes
 * @return The UTF8 string
 */
public static String utf8(ByteBuffer buffer, int offset, int length) {
  if (buffer.hasArray())
    return new String(buffer.array(), buffer.arrayOffset() + buffer.position() + offset, length, StandardCharsets.UTF_8);
  else
    return utf8(toArray(buffer, offset, length));
}

代码示例来源：origin: apache/kafka

@Override
public String read(ByteBuffer buffer) {
  short length = buffer.getShort();
  if (length < 0)
    throw new SchemaException("String length " + length + " cannot be negative");
  if (length > buffer.remaining())
    throw new SchemaException("Error reading string of length " + length + ", only " + buffer.remaining() + " bytes available");
  String result = Utils.utf8(buffer, length);
  buffer.position(buffer.position() + length);
  return result;
}

代码示例来源：origin: apache/kafka

private static Header[] readHeaders(ByteBuffer buffer, int numHeaders) {
  Header[] headers = new Header[numHeaders];
  for (int i = 0; i < numHeaders; i++) {
    int headerKeySize = ByteUtils.readVarint(buffer);
    if (headerKeySize < 0)
      throw new InvalidRecordException("Invalid negative header key size " + headerKeySize);
    String headerKey = Utils.utf8(buffer, headerKeySize);
    buffer.position(buffer.position() + headerKeySize);
    ByteBuffer headerValue = null;
    int headerValueSize = ByteUtils.readVarint(buffer);
    if (headerValueSize >= 0) {
      headerValue = buffer.slice();
      headerValue.limit(headerValueSize);
      buffer.position(buffer.position() + headerValueSize);
    }
    headers[i] = new RecordHeader(headerKey, headerValue);
  }
  return headers;
}

代码示例来源：origin: apache/kafka

private void doTestUtf8ByteBuffer(ByteBuffer utf8Buffer) {
  String utf8String = "A\u00ea\u00f1\u00fcC";
  byte[] utf8Bytes = utf8String.getBytes(StandardCharsets.UTF_8);
  utf8Buffer.position(4);
  utf8Buffer.put(utf8Bytes);
  utf8Buffer.position(4);
  assertEquals(utf8String, Utils.utf8(utf8Buffer, utf8Bytes.length));
  assertEquals(4, utf8Buffer.position());
  utf8Buffer.position(0);
  assertEquals(utf8String, Utils.utf8(utf8Buffer, 4, utf8Bytes.length));
  assertEquals(0, utf8Buffer.position());
}

代码示例来源：origin: apache/kafka

throw new IllegalArgumentException("Invalid null header key found in headers");
byte[] utf8Bytes = Utils.utf8(headerKey);
ByteUtils.writeVarint(utf8Bytes.length, out);
out.write(utf8Bytes);

代码示例来源：origin: apache/kafka

int expectedLeaderEpoch = Integer.parseInt(Utils.utf8(record.value()));
assertEquals(Optional.of(expectedLeaderEpoch), record.leaderEpoch());

代码示例来源：origin: apache/kafka

assertEquals("1", utf8(logRecords.get(0).key()));
assertEquals("2", utf8(logRecords.get(1).key()));
assertEquals("3", utf8(logRecords.get(2).key()));
  assertEquals(RecordBatch.MAGIC_VALUE_V1, batches.get(1).magic());
  assertEquals(1, batches.get(1).baseOffset());
  assertEquals("1", utf8(logRecords.get(0).key()));
  assertEquals("2", utf8(logRecords.get(1).key()));
  assertEquals("3", utf8(logRecords.get(2).key()));
  verifyRecordsProcessingStats(convertedRecords.recordConversionStats(), 3, 2,
      records.sizeInBytes(), buffer.limit());
  assertEquals(RecordBatch.MAGIC_VALUE_V1, batches.get(1).magic());
  assertEquals(2, batches.get(1).baseOffset());
  assertEquals("1", utf8(logRecords.get(0).key()));
  assertEquals("3", utf8(logRecords.get(1).key()));
  verifyRecordsProcessingStats(convertedRecords.recordConversionStats(), 3, 1,
      records.sizeInBytes(), buffer.limit());

代码示例来源：origin: apache/kafka

if (magic > RecordBatch.MAGIC_VALUE_V0)
  assertEquals(11L, first.timestamp());
assertEquals("1", Utils.utf8(first.key(), first.keySize()));
assertEquals("b", Utils.utf8(first.value(), first.valueSize()));
if (magic > RecordBatch.MAGIC_VALUE_V0)
  assertEquals(20L, second.timestamp());
assertEquals("4", Utils.utf8(second.key(), second.keySize()));
assertEquals("e", Utils.utf8(second.value(), second.valueSize()));
if (magic > RecordBatch.MAGIC_VALUE_V0)
  assertEquals(15L, third.timestamp());
assertEquals("5", Utils.utf8(third.key(), third.keySize()));
assertEquals("f", Utils.utf8(third.value(), third.valueSize()));
if (magic > RecordBatch.MAGIC_VALUE_V0)
  assertEquals(16L, fourth.timestamp());
assertEquals("6", Utils.utf8(fourth.key(), fourth.keySize()));
assertEquals("g", Utils.utf8(fourth.value(), fourth.valueSize()));

代码示例来源：origin: apache/kafka

@Test
public void testWriteTo() throws IOException {
  if (compression == CompressionType.ZSTD && magic < MAGIC_VALUE_V2)
    return;
  try (FileRecords fileRecords = FileRecords.open(tempFile())) {
    fileRecords.append(MemoryRecords.withRecords(magic, compression, new SimpleRecord("foo".getBytes())));
    fileRecords.flush();
    FileLogInputStream logInputStream = new FileLogInputStream(fileRecords, 0, fileRecords.sizeInBytes());
    FileChannelRecordBatch batch = logInputStream.nextBatch();
    assertNotNull(batch);
    assertEquals(magic, batch.magic());
    ByteBuffer buffer = ByteBuffer.allocate(128);
    batch.writeTo(buffer);
    buffer.flip();
    MemoryRecords memRecords = MemoryRecords.readableRecords(buffer);
    List<Record> records = Utils.toList(memRecords.records().iterator());
    assertEquals(1, records.size());
    Record record0 = records.get(0);
    assertTrue(record0.hasMagic(magic));
    assertEquals("foo", Utils.utf8(record0.value(), record0.valueSize()));
  }
}

代码示例来源：origin: apache/kafka

assertTrue("Inner record should have magic " + magicByte, record.hasMagic(batch.magic()));
assertEquals("Offset should not change", initialOffsets.get(i).longValue(), record.offset());
assertEquals("Key should not change", utf8(initialRecords.get(i).key()), utf8(record.key()));
assertEquals("Value should not change", utf8(initialRecords.get(i).value()), utf8(record.value()));
assertFalse(record.hasTimestampType(TimestampType.LOG_APPEND_TIME));
if (batch.magic() == RecordBatch.MAGIC_VALUE_V0) {

代码示例来源：origin: apache/kafka

assertTrue("Inner record should have magic " + toMagic, record.hasMagic(batch.magic()));
assertEquals("Offset should not change", initialOffsets.get(i).longValue(), record.offset());
assertEquals("Key should not change", utf8(initialRecords.get(i).key()), utf8(record.key()));
assertEquals("Value should not change", utf8(initialRecords.get(i).value()), utf8(record.value()));
assertFalse(record.hasTimestampType(TimestampType.LOG_APPEND_TIME));
if (batch.magic() == RecordBatch.MAGIC_VALUE_V0) {

代码示例来源：origin: me.jeffshaw.kafka/kafka-clients

@Override
public Object read(ByteBuffer buffer) {
  int length = buffer.getShort();
  byte[] bytes = new byte[length];
  buffer.get(bytes);
  return Utils.utf8(bytes);
}

代码示例来源：origin: me.jeffshaw.kafka/kafka-clients

@Override
public void write(ByteBuffer buffer, Object o) {
  byte[] bytes = Utils.utf8((String) o);
  if (bytes.length > Short.MAX_VALUE)
    throw new SchemaException("String is longer than the maximum string length.");
  buffer.putShort((short) bytes.length);
  buffer.put(bytes);
}

内容来源于网络，如有侵权，请联系作者删除！

Utils

相关文章

    查看更多
    微信公众号

    热门标签

    更多
    JavaquerypythonNode开发语言requestUtil数据库Table后端算法LoggerMessageElementParser

    最新文章

    更多
    • 蜀ICP备13028337号-1 大数据知识库 https://www.saoniuhuo.com © All rights reserved
    • 本站内容来源互联网,如果侵犯您的权益请联系我们删除, 联系方式：448109455@qq.com