snowflake环境下的sql查询数据验证

mkshixfv  于 2021-07-26  发布在  Java
关注(0)|答案(1)|浏览(296)

我有两个数据库说在雪花开发和质量。目前我正在两个数据库表上分别运行sql查询。并分别存储输出。然后使用excel对两个文件进行手动数据验证。是通过任何方式实现任何脚本或sql查询自动化之类的事情。进行验证。
谢谢

cmssoen2

cmssoen21#

根据您的需求,有许多方法可以进行验证。假设您想验证两个不同数据库中employee表的列数据,您可以创建一个sql脚本并运行它来验证是否存在任何差异。下面是一个示例,您可以进一步扩展它,或者在它的基础上构建您的用例。

// Comparing data of tables in different databases
use role accountadmin;

create or replace database exampledb1;

use database exampledb1;
create or replace table employee

(   id int,
    FirstName  varchar(20),
    LastName   varchar(20),
    email      varchar(50) 
);

insert into employee values 
(1,'Sarah', 'Janes','sarah.janes@hotmail.com'),
(2,'Richard', 'Hays','richard.hays@gmail.com'),
(3,'Maria', 'Khan','maria.khan@wow.com');

create or replace database exampledb2;
use database exampledb2;
create or replace table employee

(   id  int,
    FirstName  varchar(20),
    LastName   varchar(20),
    email      varchar(50) 
);

insert into employee values 
(1,'Sarah', 'Janes','sarah.janes@hotmail.com'),
(2,'Richard1', 'Hays','richard.hays@gmail.com'),
(3,'Maria', 'Khan','maria.khan@wow.com');

 // Check the variance by comparing hash of each row
 // You should have correct access to both databases, scheam and underlying tables

 with qadata as
 (
 select id,hash(FirstName||LastName||email) as r1, FirstName, LastName,Email
 from exampledb2.public.employee as r1
 ),
 proddata as
 (
 select id,hash(t.FirstName||t.LastName||t.email) as r1, FirstName, LastName,Email
 from exampledb1.public.employee as t
  )
  select prod.id,
         case when prod.r1 = qa.r1 then 'Matched' else 'Variance' end as CheckStatus,
         prod.FirstName, 
         prod.LastName,
         prod.Email, 
         qa.FirstName, 
         qa.LastName,
         qa.Email
  from qadata qa 
  inner join proddata prod on qa.id=prod.id;

下面是输出列checkstatus,它将标识行数据是否匹配,或者是否存在如第2行所示的差异。比较结果

相关问题