你可以通过求和函数来实现

import pyspark.sql.functions as f
df.select(*[f.sum(cols).alias(cols) for cols in df.columns]).show()

+----+---+---+
|val1|  x|  y|
+----+---+---+
|  36| 29|159|
+----+---+---+

要将所有列求和为一个新列，可以使用python的sum函数来理解列表

import pyspark.sql.functions as F
from pyspark.sql.functions import udf
from pyspark.sql.types import *
tst= sqlContext.createDataFrame([(10,7,14),(5,1,4),(9,8,10),(2,6,90),(7,2,30),(3,5,11)],schema=['val1','x','y'])
tst_sum= tst.withColumn("sum_col",sum([tst[coln] for coln in tst.columns]))

结果：

tst_sum.show()
+----+---+---+-------+
|val1|  x|  y|sum_col|
+----+---+---+-------+
|  10|  7| 14|     31|
|   5|  1|  4|     10|
|   9|  8| 10|     27|
|   2|  6| 90|     98|
|   7|  2| 30|     39|
|   3|  5| 11|     19|
+----+---+---+-------+

注意：如果您从pyspark函数导入sum函数 from import pyspark.sql.functions import sum 然后你得把名字改成别的名字，比如 from import pyspark.sql.functions import sum_pyspark