这可以是一个工作的解决方案，供您使用 spark 在构建函数中使用udf，这最终会使应用程序变慢。功能
explode() groupBy() 与 collect_set() 会帮助你达到预期的效果。

在这里创建df

df = spark.createDataFrame([(1, 'book bike car'), (18, 'car TV bike')], ['id', 'val'])
df = df.withColumn("dummy_col", F.lit(1))
df.show()
+---+-------------+---------+
| id|          val|dummy_col|
+---+-------------+---------+
|  1|book bike car|        1|
| 18|  car TV bike|        1|
+---+-------------+---------+

逻辑在这里

# Add a dummy column to groupBy & in a single line

df = df.withColumn("array_col", F.split("val", " "))

# Collect_set will return you an array without duplicates

df_grp = df.groupBy("dummy_col").agg(F.collect_set("array_col").alias("array_col"))

# explode to transpoe the column

df_grp = df_grp.withColumn("explode_col", F.explode("array_col"))
df_grp = df_grp.withColumn("explode_col", F.explode("explode_col"))

# Distince to remove the duplicates

df_grp = df_grp.select("explode_col").distinct()

# another dummy column to create the row number

df_grp = df_grp.withColumn("dummy_col", F.lit("A"))
_w = W.partitionBy("dummy_col").orderBy("dummy_col")
df_grp = df_grp.withColumn("rnk", F.row_number().over(_w))
df_grp.show(truncate=False)

最终输出

+-----------+---------+---+
|explode_col|dummy_col|rnk|
+-----------+---------+---+
|TV         |A        |1  |
|car        |A        |2  |
|bike       |A        |3  |
|book       |A        |4  |
+-----------+---------+---+