我的工作环境如下： spark = 2.0.0 , hdp = 2.5.3.0 , python = 2.7 , yarn-client 我的pyspark代码大部分时间都很好用。
然而，有时我得到以下例外上 df.count() 功能
适用于我的代码：

df= spark.read.orc("${path}")
df.count()

获取异常的代码：

df= spark.read.orc("${path}")
df = df.cache()
df.count()

堆栈跟踪：

Job aborted due to stage failure: Task 0 in stage 4.0 failed 4 times, most recent failure: Lost task 0.3 in stage 4.0 (TID 9, apii): java.nio.BufferOverflowException +details
    Job aborted due to stage failure: Task 0 in stage 4.0 failed 4 times, most recent failure: Lost task 0.3 in stage 4.0 (TID 9, apii): java.nio.BufferOverflowException
        at java.nio.Buffer.nextPutIndex(Buffer.java:521)
        at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:169)
        at org.apache.spark.sql.execution.columnar.BOOLEAN$.append(ColumnType.scala:286)
        at org.apache.spark.sql.execution.columnar.compression.RunLengthEncoding$Encoder.compress(compressionSchemes.scala:143)
        at org.apache.spark.sql.execution.columnar.compression.CompressibleColumnBuilder$class.build(CompressibleColumnBuilder.scala:103)
        at org.apache.spark.sql.execution.columnar.NativeColumnBuilder.build(ColumnBuilder.scala:97)

感谢您的帮助：）