sql—在检查现有行时提高插入性能

vecaoik1  于 2021-07-26  发布在  Java
关注(0)|答案(1)|浏览(240)

我有一个简单的查询,它从一个表中插入行( sn_users_main )变成另一个( sn_users_history ).
确保 sn_users_history 只有唯一的行它检查列 query_time 已经存在,如果存在则不插入。 query_time 是一种会话标识符,对于中的每一行都是相同的 sn_users_main .
这很好,但自从 sn_users_history 达到5万行运行此查询需要2分钟以上,这太多了。我能做些什么来提高绩效并得到同样的结果吗?

INSERT INTO sn_users_history(query_time,user_id,sn_name,sn_email,sn_manager,sn_active,sn_updated_on,sn_last_Login_time,sn_is_vip,sn_created_on,sn_is_team_lead,sn_company,sn_department,sn_division,sn_role,sn_employee_profile,sn_location,sn_employee_type,sn_workstation) --- Columns of history table
    SELECT snm.query_time,
            snm.user_id,
            snm.sn_name,
            snm.sn_email,
            snm.sn_manager,
            snm.sn_active,
            snm.sn_updated_on,
            snm.sn_last_Login_time,
            snm.sn_is_vip,
            snm.sn_created_on,
            snm.sn_is_team_lead,
            snm.sn_company,
            snm.sn_department,
            snm.sn_division,
            snm.sn_role,
            snm.sn_employee_profile,
            snm.sn_location,
            snm.sn_employee_type,
            snm.sn_workstation
    ---Columns of main table
    FROM sn_users_main snm
    WHERE NOT EXISTS(SELECT snh.query_time
    FROM sn_users_history snh
    WHERE snh.query_time = snm.query_time) --Dont insert items into history table if they already exist
qyzbxkaa

qyzbxkaa1#

我认为,当您插入到历史表中时,您缺少关于用户标识的额外条件。你必须检查userid和querytime的组合。
对于你的问题,我认为你是在试图重新发明方向盘。sql server已具有时态表,以支持此历史数据存储。了解sql server时态表
如果你还想继续这种方法,我建议你分批进行:
创建一个配置表来保存最后处理的查询时间

CREATE TABLE HistoryConfig(HistoryConfigId int, HistoryTableName SYSNAME,     
lastProcessedQueryTime DATETIME)

您可以执行增量历史插入

DECLARE @lastProcessedQueryTime DATETIME = (SELECT MAX(lastProcessedQueryTime) FROM HistoryConfig)
INSERT INTO sn_users_history(query_time,user_id,sn_name,sn_email,sn_manager,sn_active,sn_updated_on,sn_last_Login_time,sn_is_vip,sn_created_on,sn_is_team_lead,sn_company,sn_department,sn_division,sn_role,sn_employee_profile,sn_location,sn_employee_type,sn_workstation) --- Columns of history table
    SELECT snm.query_time,
            snm.user_id,
            snm.sn_name,
            snm.sn_email,
            snm.sn_manager,
            snm.sn_active,
            snm.sn_updated_on,
            snm.sn_last_Login_time,
            snm.sn_is_vip,
            snm.sn_created_on,
            snm.sn_is_team_lead,
            snm.sn_company,
            snm.sn_department,
            snm.sn_division,
            snm.sn_role,
            snm.sn_employee_profile,
            snm.sn_location,
            snm.sn_employee_type,
            snm.sn_workstation
    ---Columns of main table
    FROM sn_users_main snm
    WHERE query_time > @lastProcessedQueryTime

现在,您可以再次更新配置

UPDATE HistoryConfig SET lastProcessedQueryTime = (SELECT MAX(lastProcessedQueryTime) FROM HistoryConfig)
HistoryTableName = 'sn_users_history'

我建议您在userid上创建聚集索引,查询时间(如果可能,否则创建非聚集索引),这将提高性能。
您可以想到的其他方法:
在历史表中对userid、querytime创建聚集索引,并将userid、querytime作为主表的聚集索引,然后执行合并操作。

相关问题