重写此查询时需要帮助吗

mbskvtky  于 2021-06-24  发布在  Hive
关注(0)|答案(1)|浏览(251)

我们在生产环境中有一个查询,它每天都运行,它做了很多连接,并且在配置单元中使用了窗口函数
我们试着添加一些设置选项,但没有多大帮助
结构是这样的-

SELECT
        C.f1, C.f2, A.f2 ...
FROM (
    SELECT * FROM (
        SELECT T1.*, B.atid, B.a_id,
        ROW_NUMBER() OVER (PARTITION BY T1.wtid, B.atid ORDER BY T1.b_ts DESC) AS RANK_
        FROM T1 AS T1
        JOIN T5 ON T1.t_dt = T5.t_dt
        JOIN T2 B ON T1.wtid = B.wtid and T1.b_ts = B.b_ts
        LEFT OUTER JOIN (SELECT p_cd FROM T3 WHERE PV_TY_CD = 'ORIG_CD') PV
        ON T1.TYP = PV.p_cd
        WHERE T1.state not in ("INVALID")
        AND T1.evt_name NOT IN ('INACTIVE','DORMANT')
        AND ISNULL(PV.p_cd)
    ) T
    WHERE T.rank_ = 1
) A

JOIN (SELECT *, row_number() over (partition by ac_id order by b_ts desc) rank_  
      FROM T4
      WHERE event not in ('CT','UPD')
     ) AS C
  ON A.a_id = C.a_id
AND A.atid = C.ac_id
AND C.rank_ = 1
JOIN T6 ON C.t_dt = T6.t_dt

由于我不能忽略任何表(和联接),我的方法是用另一个使用聚合函数max的联接来替换window函数,但我无法重写它。
我也不确定这是否一定会有助于提高绩效,所以任何指导都会对我们有所帮助。

wswtfjt7

wswtfjt71#

解析函数的性能通常比使用select max的join更好,因为在解析函数的情况下,您只读取同一个表一次,而且行数计算是按分区并行的。
尝试重新组合联接和筛选。
加入

LEFT OUTER JOIN (SELECT p_cd FROM T3 WHERE PV_TY_CD = 'ORIG_CD') PV
        ON T1.TYP = PV.p_cd

带where条件 ISNULL(PV.p_cd) 正在减少t1中的一些行。这些条件也一样:

WHERE T1.state not in ("INVALID")
        AND T1.evt_name NOT IN ('INACTIVE','DORMANT')

将此联接移到子查询中,如果它过滤lo,这可能有助于在所有其他联接和行数()之前减少t1中的数据集:

(select T1.* from T1 
             left join (SELECT p_cd FROM T3 WHERE PV_TY_CD = 'ORIG_CD') PV 
                       ON T1.TYP = PV.p_cd 
 where T1.state not in ("INVALID")
        AND T1.evt_name NOT IN ('INACTIVE','DORMANT')
        AND ISNULL(PV.p_cd)
) as T1

此外,第一行\u数仅在t1和b表上计算:

PARTITION BY T1.wtid, B.atid ORDER BY T1.b_ts DESC

考虑在行数筛选器之后连接t5表,如果此连接很重,并且行数筛选器正在减少数据集,则再次在子查询中用筛选器 Package 行数,并连接用t5筛选的子查询。

(--filtered by row_number
select * from
(
 SELECT T1.*, B.atid, B.a_id,
        ROW_NUMBER() OVER (PARTITION BY T1.wtid, B.atid ORDER BY T1.b_ts DESC) AS RANK_
  from
    (select T1.* from T1 
                 left join (SELECT p_cd FROM T3 WHERE PV_TY_CD = 'ORIG_CD') PV 
                           ON T1.TYP = PV.p_cd 
     where T1.state not in ("INVALID")
            AND T1.evt_name NOT IN ('INACTIVE','DORMANT')
            AND ISNULL(PV.p_cd)
    ) as T1 JOIN T2 B ON T1.wtid = B.wtid and T1.b_ts = B.b_ts
) T WHERE T.rank_ = 1
) T --filtered
JOIN T5 ON T1.t_dt = T5.t_d

根据您的数据,这可能会有所帮助。
另请阅读:https://stackoverflow.com/a/51061613/2700344 还有这个:https://stackoverflow.com/a/51061613/2700344

相关问题