将sparkDataframe写入awss3时出现问题

tv6aics1 于 2021-07-12 发布在 Spark

关注(0)|答案(0)|浏览(191)

你好，我有困难写SparkDataframe到s3。这些是我拥有的库aws-java-sdk-1.7.4.jar hadoop-aws-2.7.3.jar
我做了以下工作 dataframe.write.csv('s3a://this/is/path/', header=True) 我看到的错误是 com.amazonaws.AmazonClientException: Unable to verify integrity of data upload. Client calculated content hash didn't match hash calculated by Amazon S3.

apache-spark pyspark apache-spark-sql amazon-s3

来源：https://stackoverflow.com/questions/66377957/problem-writing-spark-dataframe-to-aws-s3