hadoop 2.2.0-recoveryinprogressexception将内容附加到现有文件时

f87krz0w  于 2021-06-03  发布在  Hadoop
关注(0)|答案(1)|浏览(381)

我无法将内容附加到hdfs中的现有文件中。下面一行将抛出异常。

outputStream = hdfs.append(dirPath);

“dirpath所在的位置”hdfs://master:54310/test/readme.txt”。
请注意,我在单节点上运行hadoop进行开发。
异常日志如下所示。

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.RecoveryInProgressException): Failed to close file /test/Readme.txt. Lease recovery is in progress. Try again later.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:2310)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInternal(FSNamesystem.java:2153)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFileInt(FSNamesystem.java:2386)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.appendFile(FSNamesystem.java:2347)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.append(NameNodeRpcServer.java:508)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.append(ClientNamenodeProtocolServerSideTranslatorPB.java:320)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59572)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042)

还要注意的是,在同一行,我有时会得到这个例外

failed to create file /test/Readme.txt for DFSClient_NONMAPREDUCE_187487992_15 on client 127.0.0.1 because current leaseholder is trying to recreate file

有人能详细说明一下为什么会抛出这些异常吗?

xzv2uavs

xzv2uavs1#

即使在hdfs site.xml中将dfs.replication设置为1,复制值仍将设置为3(请说明发生这种情况的原因,重新启动了几次),

hdfs.setReplication(dirPath, (short) 1);

相关问题