我已经在c#中创建了一个API来使用ODBC连接访问Databricks,在安装Simba驱动程序后,它在本地工作正常。通过Azure管道,创建Docker镜像以部署在AWS ECS和Linux上。下面的命令创建Docker镜像并成功推送到ECS。
RUN apt-get install -y apt-utils
RUN apt-get install wget -y && wget https://databricks.com/wp-content/uploads/drivers-2020/SimbaSparkODBC-2.6.16.1019-Debian-64bit.zip
RUN apt-get install zip -y
RUN apt-get install -y libsasl2-modules-gssapi-mit
RUN unzip SimbaSparkODBC-2.6.16.1019-Debian-64bit.zip && rm SimbaSparkODBC-2.6.16.1019-Debian-64bit.zip
RUN dpkg -i SimbaSparkODBC-2.6.16.1019-Debian-64bit/simbaspark_2.6.16.1019-2_amd64.deb
RUN apt-get install -y unixodbc unixodbc-dev
RUN export ODBCINI=/root/odbc.ini ODBCSYSINI=/root/odbcinst.ini SIMBASPARKINI=/opt/simba/spark/lib/64/simba.sparkodbc.ini
docker镜像创建成功,但在尝试调用API时,出现以下错误:
Error: System.Data.Odbc.OdbcException (0x80131937): ERROR [01000] [unixODBC][Driver Manager]Can't open lib 'Simba Spark ODBC Driver' : file not found
at System.Data.Odbc.OdbcConnection.HandleError(OdbcHandle hrHandle, SQLRETURN retcode)
at System.Data.Odbc.OdbcConnectionHandle..ctor(OdbcConnection connection, OdbcConnectionString constr, OdbcEnvironmentHandle environmentHandle)
请帮助我优化上述命令并解决问题。谢谢C#代码,如果需要:
using Microsoft.Identity.Client;
using System.Data.Odbc;
using Microsoft.Azure.Databricks.Client;
using Microsoft.Graph.Core;
using System;
using System.Data;
using System.Data.Common;
using System.Net.Http;
using System.Text;
class Program
{
private static async Task<String> auth()
{
var authorityUri = new Uri("https://login.microsoftonline.com/ea80952e-a476-42d4-aaf4-5457852b0f7e");
IConfidentialClientApplication app;
app = ConfidentialClientApplicationBuilder.Create("AppID")
.WithClientSecret("Client-secret")
.WithAuthority(authorityUri)
.Build();
string[] scopes = new string[] { "2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default" };
AuthenticationResult result = null;
try
{
result = await app.AcquireTokenForClient(scopes)
.ExecuteAsync();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
return result.AccessToken;
}
static void Main(string[] args)
{
var authToken = auth().Result;
Console.WriteLine(authToken);
OdbcConnectionStringBuilder odbcConnectionStringBuilder =
new OdbcConnectionStringBuilder()
{
Driver = "Simba Spark ODBC Driver"
};
odbcConnectionStringBuilder.Add("Host", "adb-host.azuredatabricks.net");
odbcConnectionStringBuilder.Add("Port", "443");
odbcConnectionStringBuilder.Add("SSL", "1");
odbcConnectionStringBuilder.Add("ThriftTransport", "2");
odbcConnectionStringBuilder.Add("AuthMech", "11");
odbcConnectionStringBuilder.Add("Auth_Flow", "0");
odbcConnectionStringBuilder.Add("Auth_AccessToken", authToken);
odbcConnectionStringBuilder.Add("httpPath", "/sql/1.0/warehouses/id");
using (OdbcConnection connection = new OdbcConnection(odbcConnectionStringBuilder.ConnectionString))
{
string sqlQuery = "SELECT * FROM table LIMIT 2";
OdbcCommand command = new OdbcCommand(sqlQuery, connection);
connection.Open();
OdbcDataReader reader = command.ExecuteReader();
for (int i = 0; i < reader.FieldCount; i++)
{
Console.Write(reader.GetName(i) + "\t");
}
if (reader.HasRows)
{
while (reader.Read())
{
Console.WriteLine("{0}\t{1}\t{2}", reader.GetString(0), reader.GetString(1), reader.GetString(2));
}
}
Console.Write("\n");
reader.Close();
command.Dispose();
}
}
}
1条答案
按热度按时间wvmv3b1j1#
在我的情况下,我是不匹配的DSN.但这里是完整的解决方案:
当您通过Azure Active Directory连接时,在其他情况下,您必须指定UID、密码或PAT(个人访问令牌)。
要获取主机、httpPath和配置详细信息,请执行以下操作:https://learn.microsoft.com/en-us/azure/databricks/integrations/jdbc-odbc-bi#--install-and-configure-the-odbc-driver-for-linux
Docker命令:
在C#中:DSN名称应与docker命令中的名称相同: