我有一个由json文件创建的Dataframe，其模式如下

root
 |-- content: string (nullable = true)
 |-- conversationId: long (nullable = true)
 |-- date: string (nullable = true)
 |-- id: long (nullable = true)
 |-- lang: string (nullable = true)
 |-- likeCount: long (nullable = true)
 |-- media: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- duration: double (nullable = true)
 |    |    |-- fullUrl: string (nullable = true)
 |    |    |-- previewUrl: string (nullable = true)
 |    |    |-- thumbnailUrl: string (nullable = true)
 |    |    |-- type: string (nullable = true)
 |    |    |-- variants: array (nullable = true)
 |    |    |    |-- element: struct (containsNull = true)
 |    |    |    |    |-- bitrate: long (nullable = true)
 |    |    |    |    |-- contentType: string (nullable = true)
 |    |    |    |    |-- url: string (nullable = true)

我想提取单独的Dataframe
不带媒体列的根Dataframe
不带variants列的媒体Dataframe
变体Dataframe
问题是我不能放弃 variants 列自 media 新建Dataframe。
而且在 media 新的Dataframe我不想要所有的数据 media 专栏，我要 media 对象是其Dataframe中的新根，字段是列，对于 variants 新建Dataframe
这是我的密码：

tweets_df = spark.read.json(jsonFile)
media_df = tweets_df.select(col('media'))
variants_df = media_df.select(col('media.variants'))

# i tried also media_df = media_df.drop('media.variants')

media_df = media_df.drop('variants')
tweets_df = tweets_df.drop('media')

结果是：

root
 |-- variants: array (nullable = true)
 |    |-- element: array (containsNull = true)
 |    |    |-- element: struct (containsNull = true)
 |    |    |    |-- bitrate: long (nullable = true)
 |    |    |    |-- contentType: string (nullable = true)
 |    |    |    |-- url: string (nullable = true)

root
 |-- media: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- duration: double (nullable = true)
 |    |    |-- fullUrl: string (nullable = true)
 |    |    |-- previewUrl: string (nullable = true)
 |    |    |-- thumbnailUrl: string (nullable = true)
 |    |    |-- type: string (nullable = true) **IT'S NOT DELETED**
 |    |    |-- variants: array (nullable = true)
 |    |    |    |-- element: struct (containsNull = true)
 |    |    |    |    |-- bitrate: long (nullable = true)
 |    |    |    |    |-- contentType: string (nullable = true)
 |    |    |    |    |-- url: string (nullable = true)

root
 |-- content: string (nullable = true)
 |-- conversationId: long (nullable = true)
 |-- date: string (nullable = true)
 |-- idOnSite: long (nullable = true)
 |-- lang: string (nullable = true)
 |-- likeCount: long (nullable = true)