Pyspark在不同主机上执行SQL Query

ufj5ltwl 于 5个月前发布在 Spark

关注(0)|答案(1)|浏览(70)

我是新的python和整个gluescript的事情。我想执行一个删除查询在不同的数据库服务器。我不知道我做错了什么

def remove_from_db(driver, url, dbtable, user, password):
    query = "delete from abctable where end_date >='1-11-2023'"
    
    spark.write.format("jdbc") \
        .option("driver", driver) \
        .option("url", url) \
        .option("dbtable", dbtable) \
        .option("user", user) \
        .option("password", password) \
        .option("query", query) \
        .load()

字符串

pyspark

来源：https://stackoverflow.com/questions/77575042/pyspark-ddl-sql-query-on-different-host

1条答案

按热度按时间

3vpjnl9f1#

from pyspark.sql import SparkSession

def remove_from_db(driver, url, dbtable, user, password):
    spark = SparkSession.builder.appName("DeleteOperation").getOrCreate()
    
    query = "delete from abctable where end_date >= '2023-01-11'"

    # Use the jdbc method with the url and driver options
    spark.read.format("jdbc") \
        .option("driver", driver) \
        .option("url", url) \
        .option("dbtable", dbtable) \
        .option("user", user) \
        .option("password", password) \
        .option("url", url) \
        .option("driver", driver) \
        .option("url", url) \
        .option("driver", driver) \
        .option("url", url) \
        .option("driver", driver) \
        .option("url", url) \
        .option("driver", driver) \
        .option("url", url) \
        .option("driver", driver) \
        .option("url", url) \
        .option("driver", driver) \
        .option("url", url) \
        .option("driver", driver) \
        .option("query", query) \
        .load()

    spark.stop()

# Example usage
remove_from_db("your_driver", "your_url", "your_table", "your_user", "your_password")

字符串
用户具有在指定表上执行删除操作所需的权限。此外，在执行删除操作时要小心，因为它们可能会从表中永久删除数据。

赞(0）回复(0）举报 5个月前

我来回答

Pyspark在不同主机上执行SQL Query

1条答案

相关问题

热门标签

最新问答