windowserror:[错误2]系统找不到指定的文件-pyspark

pieyvz9o  于 2021-06-01  发布在  Hadoop
关注(0)|答案(0)|浏览(304)

盖亚!运行此代码时,我在pycharm上遇到filenotfound错误:

import findspark
findspark.init("C:\\Users\\user\\spark-2.3.0-bin-hadoop2.7")

from pyspark import SparkConf 
from pyspark.sql import SparkSession

conf = SparkConf().setAppName('Fresh-Fish')  
spark = SparkSession.builder.config(conf=conf).getOrCreate()

我已经尝试了好几种方法来解决这个问题,但是运气不好。我使用的是windows 8.1 pro。

Traceback (most recent call last):
  File "C:/Users/user/PycharmProjects/spark-project/spark1.py", line 8, in <module>
    spark = SparkSession.builder.config(conf=conf).getOrCreate()
  File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\sql\session.py", line 173, in getOrCreate
    sc = SparkContext.getOrCreate(sparkConf)
  File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\context.py", line 331, in getOrCreate
    SparkContext(conf=conf or SparkConf())
  File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\context.py", line 115, in __init__
    SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
  File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\context.py", line 280, in _ensure_initialized
    SparkContext._gateway = gateway or launch_gateway(conf)
  File "C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\pyspark\java_gateway.py", line 80, in launch_gateway
    proc = Popen(command, stdin=PIPE, env=env)
  File "C:\Python27\lib\subprocess.py", line 390, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 640, in _execute_child
    startupinfo)

WindowsError: [Error 2] The system cannot find the file specified

Process finished with exit code 1

我的系统变量是:

SPARK_HOME = C:\Users\user\spark-2.3.0-bin-hadoop2.7
HADOOP_HOME = C:\hadoop  #hadoop folder contains bin folder and bin folder contains winutils.exe
PATH = C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\Program Files\Java\jre1.8.0_171\bin;C:\Python27;%SPARK_HOME%\bin;%JAVA_HOME%\bin;%HADOOP_HOME%\bin
JAVA_HOME = C:\Program Files\Java\jdk1.8.0_161

我还尝试从一个新变量自身指向python.exe,例如:

PYSPARK_HOME = C:\Python27

甚至向pycharm项目解释器添加特定路径,指向:

C:\Users\user\spark-2.3.0-bin-hadoop2.7\python
"C:\Users\user\spark-2.3.0-bin-hadoop2.7\python\lib\py4j-0.10.6-src.zip"

但没用
pycharm的两个截图:
项目结构pycharm
项目翻译pycharm
如果你需要更多的细节,我很乐意提供。我已经挣扎了好几天了,任何新的想法都是非常受欢迎的!

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题