I am trying to access a blob storage like this and it works

from azure.storage.blob import ContainerClient

storage_account_name ='storage_account_name'
storage_container_name = 'storage_container_name'
sas_token = 'sv=<>&si=<>&sr=<>&sig=<>'

account_url = "https://"+storage_account_name+".blob.core.windows/"+storage_container_name+"?"+sas_token

container_client = ContainerClient.from_container_url(container_url=account_url)

blobs = container_client.list_blobs()

for blob in blobs:
    print(blob)

But if I get the secret through dbutils and try to pass it through, it does not.

from azure.storage.blob import ContainerClient

storage_account_name ='storage_account_name'
storage_container_name = 'storage_container_name'
sas_token = dbutils.secrets.get(scope=secret_scope,key='some_key')

account_url = "https://"+storage_account_name+".blob.core.windows/"+storage_container_name+"?"+sas_token

container_client = ContainerClient.from_container_url(container_url=account_url)

blobs = container_client.list_blobs()

for blob in blobs:
    print(blob)

Is it possible to keep on using dbutils and make it work

I am trying to access a blob storage like this and it works

from azure.storage.blob import ContainerClient

storage_account_name ='storage_account_name'
storage_container_name = 'storage_container_name'
sas_token = 'sv=<>&si=<>&sr=<>&sig=<>'

account_url = "https://"+storage_account_name+".blob.core.windows/"+storage_container_name+"?"+sas_token

container_client = ContainerClient.from_container_url(container_url=account_url)

blobs = container_client.list_blobs()

for blob in blobs:
    print(blob)

But if I get the secret through dbutils and try to pass it through, it does not.

from azure.storage.blob import ContainerClient

storage_account_name ='storage_account_name'
storage_container_name = 'storage_container_name'
sas_token = dbutils.secrets.get(scope=secret_scope,key='some_key')

account_url = "https://"+storage_account_name+".blob.core.windows/"+storage_container_name+"?"+sas_token

container_client = ContainerClient.from_container_url(container_url=account_url)

blobs = container_client.list_blobs()

for blob in blobs:
    print(blob)

Is it possible to keep on using dbutils and make it work

• python
• azure-blob-storage
• databricks
• azure-storage
• dbutils

• What is the value that is being returned when you get the sas token from dbutils? What's the error message you are getting? – Gaurav Mantri Commented Mar 31 at 17:18
• dbutils.secrets.get(scope=secret_scope,key='some_key') returns [REDACTED] – smpa01 Commented Mar 31 at 18:06
• What happens if you hardcode the secret_scope? dbutils.secrets.get(scope='secret_scope_name',key='some_key'). You can also use dbutils.secrets.list(secret_scope) to verify that you can see all the key names available in that secret scope – EricL Commented Mar 31 at 21:12

1 Answer 1

To fetch the container files using secret scope dbutils please follow the below steps: You should have Premium Teir Subscription in Azure Databricks to create Secret Scope.

1. Create Secret scope and add secrets :
   databricks secret create-scope scopename --create secret scope
   databricks secrets list-scopes --check the scope is created
   databricks secrets put-secret scopename keyname --create secret inside scope and enter the value of key (copied from storage account SAS key) to get it created

   databricks secrets list-secrets scopename --to check if the secret is created or not

2. Now in Databricks workspace run the following : dbutils.secrets.get(scope="scopename", key="keyname")

Output: '[REDACTED]' is expected because Databricks hides secret values when printed to maintain security.

Now the final code : It fetches the files from the container

Please refer to Microsoft Documents for more details:

Azure Databricks Secrets Creation
Azure Storage Blob Library