使用bash脚本构建sql update语句

wdebmtf2  于 2021-07-26  发布在  Java
关注(0)|答案(2)|浏览(249)

对于最近的一个项目,我需要循环浏览一个csv(ignore.csv)的行,其中有3列: acctnum, errcode, date ← 按顺序…
(列命名在这里并不重要,它只是用于上下文)并使用这些变量来构建sql语句。
csv可能有500行,也可能有20k行。我的sql语句应该如下所示:

UPDATE O.ACCT_ERR SET REC_ACTV_IND='T'
  WHERE BUS_DT='20200603' and ERR_CD='R4442' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM O.ACCT
      WHERE ACCT_SRCH_NBR='10100000011'
  );

目前我在bash中的草率示例如下:

while IFS=, read -r field1 field2 field3;
do
   echo "UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T' WHERE BUS_DT='$field3' and ETL_ERR_CD='$field2' AND ACCT_KEY in (SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO WHERE ACCT_SRCH_NBR='$field1');" > sqlfile.txt
done < ignore.csv

它只对一行起作用,正如在的输出中验证的那样 sqlfile.txt .
但是我如何迭代它来打印csv的所有行呢?
我仍然在循环逻辑可怕-幸运的是,在这方面是一个刺激支持的家伙。
非常感谢您的帮助。

q35jwt9p

q35jwt9p1#

正如巴尔马指出的:
每次在循环中回显时都会覆盖文件。
您可以这样做,在csv的字段分隔符中包含空格。
需要注意的是:
shell无法可靠地解析可能包含引号的csv字段,引号转义。您应该考虑使用能够正确解析csv并将值作为参数或参数返回的命令 null 分隔字段,如 csvtool .
如果您的sql数据库引擎知道 PREPARE 声明,更安全的做法是 PREPARE sql UPDATE 请求,并提供参数。您应该在循环之前准备请求,然后在循环中输入参数并执行准备好的语句。


# !/usr/bin/env bash

while IFS=', ' read -r acctnum errcode date; do

  cat <<SQL
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
  WHERE BUS_DT='$date' and ETL_ERR_CD='$errcode' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
      WHERE ACCT_SRCH_NBR='$acctnum');
SQL
done < ignore.csv > sqlfile.txt

使用csvtool将exclude.csv解析并转换为sql查询: excludeCSV2sql :


# !/usr/bin/env bash

# This script uses csvtool to parse exclude.csv CSV data from stdin

# and output SQL queries to stdout

# Convert arguments from csvtool call, into an SQL query

to_sql ()
{
  # Double single-quote for SQL string values if any
  local -- \
    acct_num="${1//\'/\'\'}" \
    err_code="${2//\'/\'\'}" \
    date="${3//\'/\'\'}"

  cat <<SQL
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
  WHERE BUS_DT='$date' AND ETL_ERR_CD='$err_code' AND ACCT_KEY
  IN (
    SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
      WHERE ACCT_SRCH_NBR='$acct_num'
  );
SQL
}

# Export for use in csvtool call

export -f to_sql

# Process CSV from stdin

csvtool call to_sql -

测试:
使上述脚本可执行:

chmod +x excludeCSV2sql

创建示例 test_exclude.csv :

cat >test_exclude.csv <<CSV
foo,bar,baz
here,it's using a single quote, string
this, "has a double-quoted string
with a newline", in it
10100000012, "R4242, has comma", 20200524
10100000042, R1337, 20200525
CSV

运行测试:

./excludeCSV2sql <test_exclude.csv >test.sql

检查结果: test.sql ```
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
WHERE BUS_DT='baz' AND ETL_ERR_CD='bar' AND ACCT_KEY
IN (
SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
WHERE ACCT_SRCH_NBR='foo'
);
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
WHERE BUS_DT='string' AND ETL_ERR_CD='it''s using a single quote' AND ACCT_KEY
IN (
SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
WHERE ACCT_SRCH_NBR='here'
);
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
WHERE BUS_DT='in it' AND ETL_ERR_CD='has a double-quoted string
with a newline' AND ACCT_KEY
IN (
SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
WHERE ACCT_SRCH_NBR='this'
);
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
WHERE BUS_DT='20200524' AND ETL_ERR_CD='R4242, has comma' AND ACCT_KEY
IN (
SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
WHERE ACCT_SRCH_NBR='10100000012'
);
UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T'
WHERE BUS_DT='20200525' AND ETL_ERR_CD='R1337' AND ACCT_KEY
IN (
SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO
WHERE ACCT_SRCH_NBR='10100000042'
);

ukqbszuj

ukqbszuj2#

每次在循环中回显时都会覆盖文件。将输出重定向移到循环的末尾。

while IFS=, read -r field1 field2 field3;
do
   echo "UPDATE ODS.PERF_ACCT_ERR_DTL SET REC_ACTV_IND='T' WHERE BUS_DT='$field3' and ETL_ERR_CD='$field2' AND ACCT_KEY in (SELECT ACCT_KEY FROM ODS.ACCT_PORTFOLIO WHERE ACCT_SRCH_NBR='$field1');"
done < ignore.csv > sqlfile.txt

相关问题