org.apache.spark.api.java.JavaSparkContext.binaryRecords()方法的使用及代码示例

x33g5p2x  于2022-01-21 转载在 其他  
字(2.6k)|赞(0)|评价(0)|浏览(169)

本文整理了Java中org.apache.spark.api.java.JavaSparkContext.binaryRecords()方法的一些代码示例,展示了JavaSparkContext.binaryRecords()的具体用法。这些代码示例主要来源于Github/Stackoverflow/Maven等平台,是从一些精选项目中提取出来的代码,具有较强的参考意义,能在一定程度帮忙到你。JavaSparkContext.binaryRecords()方法的具体详情如下:
包路径:org.apache.spark.api.java.JavaSparkContext
类名称:JavaSparkContext
方法名:binaryRecords

JavaSparkContext.binaryRecords介绍

暂无

代码示例

代码示例来源:origin: org.apache.spark/spark-core_2.10

@Test
public void binaryRecords() throws Exception {
 // Reusing the wholeText files example
 byte[] content1 = "spark isn't always easy to use.\n".getBytes(StandardCharsets.UTF_8);
 int numOfCopies = 10;
 String tempDirName = tempDir.getAbsolutePath();
 File file1 = new File(tempDirName + "/part-00000");
 FileOutputStream fos1 = new FileOutputStream(file1);
 FileChannel channel1 = fos1.getChannel();
 for (int i = 0; i < numOfCopies; i++) {
  ByteBuffer bbuf = ByteBuffer.wrap(content1);
  channel1.write(bbuf);
 }
 channel1.close();
 JavaRDD<byte[]> readRDD = sc.binaryRecords(tempDirName, content1.length);
 assertEquals(numOfCopies,readRDD.count());
 List<byte[]> result = readRDD.collect();
 for (byte[] res : result) {
  assertArrayEquals(content1, res);
 }
}

代码示例来源:origin: org.apache.spark/spark-core_2.11

@Test
public void binaryRecords() throws Exception {
 // Reusing the wholeText files example
 byte[] content1 = "spark isn't always easy to use.\n".getBytes(StandardCharsets.UTF_8);
 int numOfCopies = 10;
 String tempDirName = tempDir.getAbsolutePath();
 File file1 = new File(tempDirName + "/part-00000");
 FileOutputStream fos1 = new FileOutputStream(file1);
 FileChannel channel1 = fos1.getChannel();
 for (int i = 0; i < numOfCopies; i++) {
  ByteBuffer bbuf = ByteBuffer.wrap(content1);
  channel1.write(bbuf);
 }
 channel1.close();
 JavaRDD<byte[]> readRDD = sc.binaryRecords(tempDirName, content1.length);
 assertEquals(numOfCopies,readRDD.count());
 List<byte[]> result = readRDD.collect();
 for (byte[] res : result) {
  assertArrayEquals(content1, res);
 }
}

代码示例来源:origin: org.apache.spark/spark-core

@Test
public void binaryRecords() throws Exception {
 // Reusing the wholeText files example
 byte[] content1 = "spark isn't always easy to use.\n".getBytes(StandardCharsets.UTF_8);
 int numOfCopies = 10;
 String tempDirName = tempDir.getAbsolutePath();
 File file1 = new File(tempDirName + "/part-00000");
 FileOutputStream fos1 = new FileOutputStream(file1);
 FileChannel channel1 = fos1.getChannel();
 for (int i = 0; i < numOfCopies; i++) {
  ByteBuffer bbuf = ByteBuffer.wrap(content1);
  channel1.write(bbuf);
 }
 channel1.close();
 JavaRDD<byte[]> readRDD = sc.binaryRecords(tempDirName, content1.length);
 assertEquals(numOfCopies,readRDD.count());
 List<byte[]> result = readRDD.collect();
 for (byte[] res : result) {
  assertArrayEquals(content1, res);
 }
}

相关文章

微信公众号

最新文章

更多