When I attempt to mount an ADLS Gen Storage account with the below code, I get the error:
IllegalArgumentException: Unsupported Azure Scheme: abfss
container_name = "mycontainer"
storage_account = "MyStorageAccount"
key = "xxxxxxxxxx=="
url = "abfss://" + container_name + "@" + storage_account + ".dfs.core.windows/"
config = "fs.azure.account.key." + storage_account + ".dfs.core.windows"
mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()
mounted_exist = False
for item in mounted_list:
if mount_folder in item[0]:
mounted_exist = True
break
if not mounted_exist:
dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})
I have successfully mounted an ADLS Gen 2 account from Databricks in the past using this method, so I'm uncertain why I'm getting this error?
I wanted to update this question to mention that our current environment prevents us from creating an App Registration and thus prevents us from creating a Service Principal. Which is why I'm trying to mount the storage account using the 'Account Key'
When I attempt to mount an ADLS Gen Storage account with the below code, I get the error:
IllegalArgumentException: Unsupported Azure Scheme: abfss
container_name = "mycontainer"
storage_account = "MyStorageAccount"
key = "xxxxxxxxxx=="
url = "abfss://" + container_name + "@" + storage_account + ".dfs.core.windows/"
config = "fs.azure.account.key." + storage_account + ".dfs.core.windows"
mount_folder = "/mnt/lake"
mounted_list = dbutils.fs.mounts()
mounted_exist = False
for item in mounted_list:
if mount_folder in item[0]:
mounted_exist = True
break
if not mounted_exist:
dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})
I have successfully mounted an ADLS Gen 2 account from Databricks in the past using this method, so I'm uncertain why I'm getting this error?
I wanted to update this question to mention that our current environment prevents us from creating an App Registration and thus prevents us from creating a Service Principal. Which is why I'm trying to mount the storage account using the 'Account Key'
Share Improve this question edited Feb 3 at 15:41 Patterson asked Feb 3 at 15:18 PattersonPatterson 2,8238 gold badges56 silver badges146 bronze badges 2- Do you have permissions for you AZ keyvault? ADB app? – Dileep Raj Narayan Thumula Commented Feb 4 at 3:30
- Hi @DileepRajNarayanThumula, thanks for getting in touch I have access AZ Keyvault. I'm not familiar with the ADB app – Patterson Commented Feb 4 at 9:43
1 Answer
Reset to default 1I encountered the same error when attempting to mount Azure Data Lake Storage using an account key.
Since abfss
typically requires OAuth authentication, trying to mount it using an account key resulted in an error.
However, after modifying the code as shown below, I was able to successfully mount it and list its contents.
container_name = "synpdemocontainer"
storage_account = "allnewsynp"
key = "<YOUR ACCOUNT KEY>"
url = "wasbs://" + container_name + "@" + storage_account + ".blob.core.windows/"
config = "fs.azure.account.key." + storage_account + ".blob.core.windows"
mount_folder = "/mnt/lake"
dbutils.fs.mount(source = url, mount_point = mount_folder, extra_configs = {config : key})
I successfully listed the contents using dbutils.fs.ls
. Here's an output image for reference: