invalidatemetadata给出tablenotfoundexception,尽管我可以在表列表中看到该表

p5cysglq  于 2021-06-26  发布在  Impala
关注(0)|答案(0)|浏览(163)

我正在尝试运行一个脚本,该脚本验证应该作为部署的一部分创建的所有表。这是我的剧本:

set -x

ENV=$1

. /user/setenv.sh

ticket=""
wlf_kinit ticket

echo "Checking if Kerberos Ticket is available.."
    klist $ticket

    if [ $? -eq 1 ]; then
        echo "Kerberos Ticket not found..Exiting"
        exit 1
    fi

echo "Validating Hadoop tables" > /tmp/psk1/db_validation_log.txt

db_dir=/user/db

cd $db_dir

for current_directory in `find . -maxdepth 1 -type d`
do
    #current_directory=`echo $current_directory | awk -F '/' '{print $2}'`
    echo $current_directory

    if [ "$current_directory" != "." ]; then
        current_directory=`echo $current_directory | awk -F '/' '{print $2}'`
        if [ "${current_directory:0:1}" = "v" ]; then
            dir=$db_dir/$current_directory/views
        else    
            dir=$db_dir/$current_directory/tables
        fi  
        cd $dir
        pwd
        #echo "Validating tables in "$current_directory >> /tmp/psk1/db_validation_log.txt
        find . -name '*.hql' | while read rec; do
        echo $rec
        tbl_name=`echo $rec | awk -F '/' '{print $2}' | awk -F '.' '{print $1}'`
        result=$(impala-shell --quiet --delimited --ssl -i ${impala_host} -ku ${user_id}${impala_realm} -q "set request_pool = ${request_pool}; use $current_directory$ENV; invalidate metadata $tbl_name; show tables like '$tbl_name';") 2>> /tmp/psk1/db_validation_log.txt
        #echo $result

        if [ $? -eq 0 ]; then

            if [ ${result} == ${tbl_name} ]; then
                echo "$tbl_name exists" #>> /tmp/psk1/db_validation_log.txt
            else
                echo "$tbl_name does not exist"  #>> /tmp/psk1/db_validation_log.txt
            fi
        else
            echo $current_directory$ENV"."$tbl_name" Query error" >> /tmp/psk1/db_validation_log.txt
            impala-shell --quiet --delimited --ssl -i ${impala_host} -ku ${user_id}${impala_realm} -q "set request_pool = ${request_pool}; use $current_directory$ENV; invalidate metadata $tbl_name; show tables like '$tbl_name';" 2>> /tmp/psk1/db_validation_log.txt

        fi
        done
    fi      
done

cat /tmp/psk1/db_validation_log.txt | mail -a  /tmp/psk1/db_validation_log.txt -s 'Hadoop DB validation completed. Check the attached log.'  -r ${from_email} ${to_email} 2>> /tmp/psk1/db_validation_log.txt

kdestroy -c $ticket

此脚本将失败的查询打印到一个.txt文件,并通过电子邮件发送。在文本文件中,我看到一些查询因tablenotfoundexception而失败。但是当我打开 Impala 壳列出表时,我可以看到列表中的表。我不确定是什么导致这种不一致。
任何帮助都将不胜感激。谢谢您。

暂无答案!

目前还没有任何答案,快来回答吧!

相关问题