我已经写了一个小程序,它正在工作,但它是添加参数值到列中,我不需要。
输入:
预期:
一个大写字母上方的图像变为大写字母
获得:
代码:
# !/usr/bin/env python
import sys
import logging
from pyspark.sql.types import StringType
from pyspark.sql import SparkSession
from pyspark.sql.functions import udf, lit
from functools import partial
def main():
process_function = {'TOLOWER' : process_lower_function }
spark = SparkSession.builder.master("local[1]").appName('intelligent').getOrCreate()
data = [(0, "cards"), (1, "upper")]
deptColumns = ["index", "data"]
df = spark.createDataFrame(data=data, schema = deptColumns)
print (df.show())
process_rule = "pyspark,testing,upper"
lower_function = partial(process_lower_function, process_rule)
udf_parser = udf(lower_function)
df = df.withColumn("ndata",udf_parser("data"))
#df.write.csv("udf.csv")
print (df.show())
def process_lower_function(text, lower_rule):
print (lower_rule)
for rule in lower_rule.split(","):
if rule in text:
text = text.replace(rule, rule.upper())
logging.info(f"LOWER :: {text} -- {rule}")
print (text)
return text
if __name__ == "__main__":
sys.exit(main())
暂无答案!
目前还没有任何答案,快来回答吧!