当用作边界查询时,有效的mysql查询中断

4bbkushb  于 2021-06-03  发布在  Sqoop
关注(0)|答案(1)|浏览(434)

注意:这不是sqoop-syntaxt error-boundary query-“sql语法中的错误”的副本
为了限制仅从过去8天获取数据,我使用以下方法 boundary-querySqoop ```
SELECT min(created_at),
max(created_at)
FROM billing_db.billing_ledger
WHERE created_at >= timestamp(date(convert_tz(now(), IF(@@global.time_zone = 'SYSTEM', @@system_time_zone, @@global.time_zone),'Asia/Kolkata')) + interval -2 DAY)"

为了可读性,我在这里将查询分为多行,实际上我只在单行中将其传递给sqoop
对边界查询的不同部分进行了解释 `IF(@@global.time_zone = 'SYSTEM', @@system_time_zone, @@global.time_zone)` 确定服务器时区
适用于mysql和tidb `convert_tz(now(), <server-timezone>,'Asia/Kolkata')` 从ist中的服务器时区转换时间 `timestamp(date(<ist-timestamp> + interval -{num_days} DAY)` 返回比今天早{num\u days}的日期在00:00时的ist时间戳(当前时间->tz特定)
而查询在mysql上运行良好

mysql> SELECT min(created_at),
-> max(created_at)
-> FROM billing_db.billing_ledger
-> WHERE created_at >= timestamp(date(convert_tz(now(), IF(@@global.time_zone = 'SYSTEM', @@system_time_zone, @@global.time_zone),'Asia/Kolkata')) + interval -2 DAY);
+---------------------+---------------------+
| min(created_at) | max(created_at) |
+---------------------+---------------------+
| 2020-05-08 00:00:00 | 2020-05-10 20:12:32 |
+---------------------+---------------------+
1 row in set (0.02 sec)

它与sqoop上的stacktrace一起中断

INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT min(), max() FROM . WHERE >= timestamp(date(convert_tz(now(), IF(@@global.time_zone = 'SYSTEM', @@system_time_zone, @@global.time_zone),'Asia/Kolkata')) + interval -2 DAY)
[2020-05-10 12:45:34,968] {ssh_utils.py:130} WARNING - 20/05/10 18:15:34 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/hadoop/.staging/job_1589114450995_0001
[2020-05-10 12:45:34,971] {ssh_utils.py:130} WARNING - 20/05/10 18:15:34 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@6ab7a896
[2020-05-10 12:45:34,973] {ssh_utils.py:130} WARNING - 20/05/10 18:15:34 ERROR tool.ImportTool: Import failed: java.io.IOException: java.sql.SQLSyntaxErrorException: (conn=313686) You have an error in your SQL syntax; check the manual that corresponds to your TiDB version for the right syntax to use line 1 column 12 near "), max() FROM . WHERE >= timestamp(date(convert_tz(now(), IF(@@global.time_zone = 'SYSTEM', @@system_time_zone, @@global.time_zone),'Asia/Kolkata')) + interval -2 DAY)"
at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:207)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:303)

记录在案
使用 `WHERE $CONDITIONS` 中需要 `--query` (自由形式查询导入)但是 `--boundary-query` 这不是强制性的。没有它,sqoop只会生成这个警告 `WARN db.DataDrivenDBInputFormat: Could not find $CONDITIONS token in query: SELECT min(), max() FROM . WHERE >= timestamp(date(convert_tz(now(), IF(@@global.time_zone = 'SYSTEM', @@system_time_zone, @@global.time_zone),'Asia/Kolkata')) + interval -2 DAY); splits may not partition data.` 我用过类似的复合物 `boundary-query` 但在这个特殊的情况下,它正在崩溃
我试过什么
我试着把化名加进去 `SELECT` 像这样的疑问句 `SELECT min(`created_at`) AS min_created_at,...` 
vcudknz3

vcudknz31#

回纹 ```` 是罪魁祸首
从边界查询中删除反记号解决了错误
讨论中的一些评论指出,反勾号可能会导致 sqoop 但是文件中没有提到它,一些讨论甚至鼓励使用它

相关问题