最新消息:雨落星辰是一个专注网站SEO优化、网站SEO诊断、搜索引擎研究、网络营销推广、网站策划运营及站长类的自媒体原创博客

Unable to mount ADLS Gen 2 abfss Storage account with Databricks: IllegalArgumentException: Unsupported Azure Scheme: abfss - St

programmeradmin0浏览0评论

When I attempt to mount an ADLS Gen Storage account with the below code, I get the error:

IllegalArgumentException: Unsupported Azure Scheme: abfss

container_name = "mycontainer"
storage_account = "MyStorageAccount"
key = "xxxxxxxxxx=="

url = "abfss://" + container_name + "@" + storage_account + ".dfs.core.windows/"
config = "fs.azure.account.key." + storage_account + ".dfs.core.windows"

mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()

mounted_exist = False
for item in mounted_list:
  if mount_folder in item[0]:
    mounted_exist = True
    break

  if not mounted_exist:
    dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})

I have successfully mounted an ADLS Gen 2 account from Databricks in the past using this method, so I'm uncertain why I'm getting this error?

I wanted to update this question to mention that our current environment prevents us from creating an App Registration and thus prevents us from creating a Service Principal. Which is why I'm trying to mount the storage account using the 'Account Key'

When I attempt to mount an ADLS Gen Storage account with the below code, I get the error:

IllegalArgumentException: Unsupported Azure Scheme: abfss

container_name = "mycontainer"
storage_account = "MyStorageAccount"
key = "xxxxxxxxxx=="

url = "abfss://" + container_name + "@" + storage_account + ".dfs.core.windows/"
config = "fs.azure.account.key." + storage_account + ".dfs.core.windows"

mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()

mounted_exist = False
for item in mounted_list:
  if mount_folder in item[0]:
    mounted_exist = True
    break

  if not mounted_exist:
    dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})

I have successfully mounted an ADLS Gen 2 account from Databricks in the past using this method, so I'm uncertain why I'm getting this error?

I wanted to update this question to mention that our current environment prevents us from creating an App Registration and thus prevents us from creating a Service Principal. Which is why I'm trying to mount the storage account using the 'Account Key'

Share Improve this question edited Feb 3 at 15:41 Patterson asked Feb 3 at 15:18 PattersonPatterson 2,8238 gold badges56 silver badges146 bronze badges 2
  • Do you have permissions for you AZ keyvault? ADB app? – Dileep Raj Narayan Thumula Commented Feb 4 at 3:30
  • Hi @DileepRajNarayanThumula, thanks for getting in touch I have access AZ Keyvault. I'm not familiar with the ADB app – Patterson Commented Feb 4 at 9:43
Add a comment  | 

1 Answer 1

Reset to default 1

I encountered the same error when attempting to mount Azure Data Lake Storage using an account key.

Since abfss typically requires OAuth authentication, trying to mount it using an account key resulted in an error. However, after modifying the code as shown below, I was able to successfully mount it and list its contents.

container_name = "synpdemocontainer"
storage_account = "allnewsynp"
key = "<YOUR ACCOUNT KEY>"
url = "wasbs://" + container_name + "@" + storage_account + ".blob.core.windows/"
config = "fs.azure.account.key." + storage_account + ".blob.core.windows"
mount_folder = "/mnt/lake"
dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})

I successfully listed the contents using dbutils.fs.ls. Here's an output image for reference:

与本文相关的文章

发布评论

评论列表(0)

  1. 暂无评论