用javahdfs读取文件

brtdzjyr  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(297)

我在集群上运行程序时遇到了一个问题,决定在函数map和reduce中读取hdfs文件。如何逐行读取hdfs文件并烧录以读取arraylist中的行?

to94eoyn

to94eoyn1#

只是一段代码片段进行演示:

Path path = new Path(filePath);
FileSystem fs = path.getFileSystem(context.getConfiguration()); // context of mapper or reducer
FSDataInputStream fdsis = fs.open(path);
BufferedReader br = new BufferedReader(new InputStreamReader(fdsis));
String line = "";
ArrayList<String> lines = new ArrayList<String>();
while ((line = br.readLine()) != null) {
    lines.add(line);
}
br.close();

相关问题