如何修复运行import pypsark时出现的“typeerror:an integer is required(get type bytes)”错误

gev0vcfq  于 2021-05-27  发布在  Spark
关注(0)|答案(1)|浏览(933)

spark版本2.4.5,python版本3.8.2
出现以下错误:

VirtualBox:~/spark-2.4.5-bin-hadoop2.7/python$ python3  
Python 3.8.2 (default, Apr 27 2020, 15:53:34)   
[GCC 9.3.0] on linux  
Type "help", "copyright", "credits" or "license" for more information.    
>>> import pyspark  
Traceback (most recent call last):  
  File "<stdin>", line 1, in <module>  
  File "/home/prasanth/spark-2.4.5-bin-hadoop2.7/python/pyspark/__init__.py", line 51, in <module>
    from pyspark.context import SparkContext   
  File "/home/prasanth/spark-2.4.5-bin-hadoop2.7/python/pyspark/context.py", line 31, in <module>
    from pyspark import accumulators  
  File "/home/prasanth/spark-2.4.5-bin-hadoop2.7/python/pyspark/accumulators.py", line 97, in <module>
    from pyspark.serializers import read_int, PickleSerializer  
  File "/home/prasanth/spark-2.4.5-bin-hadoop2.7/python/pyspark/serializers.py", line 72, in <module>
    from pyspark import cloudpickle    
  File "/home/prasanth/spark-2.4.5-bin-hadoop2.7/python/pyspark/cloudpickle.py", line 145, in <module>
    _cell_set_template_code = _make_cell_set_template_code()  
  File "/home/prasanth/spark-2.4.5-bin-hadoop2.7/python/pyspark/cloudpickle.py", line 126, in _make_cell_set_template_code
    return types.CodeType(  
TypeError: an integer is required (got type bytes)
i1icjdpr

i1icjdpr1#

出现此问题是因为spark与python不兼容 3.8.x ! 请使用 python3.7 . 您可以使用以下命令:

PYSPARK_PYTHON=python3.7 pyspark

相关问题