I have setup a single KeyVault and 2 Function Apps using Bicep. To test I build the bicep and input the ARM template into "Deploy custom template" on Azure portal. KeyVault deployed with public network access disabled, a private endpoint, and both are linked:

...
var is_private_access = SUBNET_ID != ''

resource KeyVault 'Microsoft.KeyVault/vaults@2024-04-01-preview' = {
  name: key_vault_unique_name
  location: REGION
  properties: {
    accessPolicies: ENABLE_RBAC_AUTHORIZATION ? [] : ACCESS_POLICIES
    createMode: CREATE_MODE
    enabledForDeployment: ENABLED_FOR_DEPLOYMENT
    enabledForDiskEncryption: ENABLED_FOR_DISK_ENCRYPTION
    enabledForTemplateDeployment: ENABLED_FOR_TEMPLATE_DEPLOYMENT
    enablePurgeProtection: ENABLE_PURGE_PROTECTION
    enableRbacAuthorization: ENABLE_RBAC_AUTHORIZATION
    enableSoftDelete: ENABLE_SOFT_DELETE
    networkAcls: {}
    publicNetworkAccess: is_private_access ? 'disabled' : 'enabled'
    sku: {
      family: 'A'
      name: 'Standard'
    }
    softDeleteRetentionInDays: SOFT_DELETE_RETENTION_DAYS
    tenantId: TENANT_ID
    vaultUri: VAULT_URI
  }
}

resource PrivateEndpoint 'Microsoft.Network/privateEndpoints@2021-02-01' = if (is_private_access) {
  name: take('VaultPrivateEndpoint-${KEY_VAULT_UNIQUE_STRING}', 64)
  location: REGION
  properties: {
    subnet: {
      id: SUBNET_ID
    }
    privateLinkServiceConnections: [
      {
        name: '${KeyVault.name}-file-private-link-connection'
        properties: {
          privateLinkServiceId: KeyVault.id
          groupIds: [
            'vault'
          ]
        }
      }
    ]
  }
}

resource VaultPrivateDnsZone 'Microsoft.Network/privateDnsZones@2020-06-01' existing = if (is_private_access) {
  name: 'privatelink.vaultcore.azure'
}

resource PrivateDnsZoneLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = if (is_private_access) {
  parent: VaultPrivateDnsZone
  name: take('virtual-network-link-${KEY_VAULT_UNIQUE_STRING}', 64)
  location: REGION
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: VNET_ID
    }
  }
}

resource PrivateDnsZoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = if (is_private_access) {
  parent: PrivateEndpoint
  name: 'VaultDnsGroup'
  properties: {
    privateDnsZoneConfigs: [
      {
        name: 'vaultConfig'
        properties: {
          privateDnsZoneId: VaultPrivateDnsZone.id
        }
      }
    ]
  }
}

Now the Function Apps get created one at a time, each in its own VNet and subnet. Private endpoints and links are created for each subnet to the Private DNS 'privatelink.vaultcore.azure'. First link goes smoothly, references to secrets can be pulled to fill environment variables of that Function App. Once the second private link deploys neither can connect.

After a ton of troubleshooting I found what happens exactly at that moment. Second link registers an A record in the private DNS zone with the same subdomain (.privatelink.vaultcore.azure) and overrides the IP address. There seems to be no method to deploy 2 private endpoints from different VNets for the same KeyVault.

Manually I managed to find a workaround, overriding the A record to have multiple IPs. This method did not work as part of the ARM template deployment so far since the IPs are generated during deployment and I cannot reference them from the private endpoint resource - else I get an error saying the deployer can't read the endpoint's properties that were not set at initialization.

Additional context: I followed this KeyVault private deployment guide, and found a troubleshooting guide for some similar scenarios. But found the information misleading since they suggest a private DNS zone per VNet when you can't create multiple of those in a single resource group with the same name, which has to be "privatelink.vaultcore.azure".

I have setup a single KeyVault and 2 Function Apps using Bicep. To test I build the bicep and input the ARM template into "Deploy custom template" on Azure portal. KeyVault deployed with public network access disabled, a private endpoint, and both are linked:

...
var is_private_access = SUBNET_ID != ''

resource KeyVault 'Microsoft.KeyVault/vaults@2024-04-01-preview' = {
  name: key_vault_unique_name
  location: REGION
  properties: {
    accessPolicies: ENABLE_RBAC_AUTHORIZATION ? [] : ACCESS_POLICIES
    createMode: CREATE_MODE
    enabledForDeployment: ENABLED_FOR_DEPLOYMENT
    enabledForDiskEncryption: ENABLED_FOR_DISK_ENCRYPTION
    enabledForTemplateDeployment: ENABLED_FOR_TEMPLATE_DEPLOYMENT
    enablePurgeProtection: ENABLE_PURGE_PROTECTION
    enableRbacAuthorization: ENABLE_RBAC_AUTHORIZATION
    enableSoftDelete: ENABLE_SOFT_DELETE
    networkAcls: {}
    publicNetworkAccess: is_private_access ? 'disabled' : 'enabled'
    sku: {
      family: 'A'
      name: 'Standard'
    }
    softDeleteRetentionInDays: SOFT_DELETE_RETENTION_DAYS
    tenantId: TENANT_ID
    vaultUri: VAULT_URI
  }
}

resource PrivateEndpoint 'Microsoft.Network/privateEndpoints@2021-02-01' = if (is_private_access) {
  name: take('VaultPrivateEndpoint-${KEY_VAULT_UNIQUE_STRING}', 64)
  location: REGION
  properties: {
    subnet: {
      id: SUBNET_ID
    }
    privateLinkServiceConnections: [
      {
        name: '${KeyVault.name}-file-private-link-connection'
        properties: {
          privateLinkServiceId: KeyVault.id
          groupIds: [
            'vault'
          ]
        }
      }
    ]
  }
}

resource VaultPrivateDnsZone 'Microsoft.Network/privateDnsZones@2020-06-01' existing = if (is_private_access) {
  name: 'privatelink.vaultcore.azure'
}

resource PrivateDnsZoneLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = if (is_private_access) {
  parent: VaultPrivateDnsZone
  name: take('virtual-network-link-${KEY_VAULT_UNIQUE_STRING}', 64)
  location: REGION
  properties: {
    registrationEnabled: false
    virtualNetwork: {
      id: VNET_ID
    }
  }
}

resource PrivateDnsZoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = if (is_private_access) {
  parent: PrivateEndpoint
  name: 'VaultDnsGroup'
  properties: {
    privateDnsZoneConfigs: [
      {
        name: 'vaultConfig'
        properties: {
          privateDnsZoneId: VaultPrivateDnsZone.id
        }
      }
    ]
  }
}

Now the Function Apps get created one at a time, each in its own VNet and subnet. Private endpoints and links are created for each subnet to the Private DNS 'privatelink.vaultcore.azure'. First link goes smoothly, references to secrets can be pulled to fill environment variables of that Function App. Once the second private link deploys neither can connect.

After a ton of troubleshooting I found what happens exactly at that moment. Second link registers an A record in the private DNS zone with the same subdomain (.privatelink.vaultcore.azure) and overrides the IP address. There seems to be no method to deploy 2 private endpoints from different VNets for the same KeyVault.

Manually I managed to find a workaround, overriding the A record to have multiple IPs. This method did not work as part of the ARM template deployment so far since the IPs are generated during deployment and I cannot reference them from the private endpoint resource - else I get an error saying the deployer can't read the endpoint's properties that were not set at initialization.

Additional context: I followed this KeyVault private deployment guide, and found a troubleshooting guide for some similar scenarios. But found the information misleading since they suggest a private DNS zone per VNet when you can't create multiple of those in a single resource group with the same name, which has to be "privatelink.vaultcore.azure".

• azure
• azure-resource-manager
• azure-keyvault
• azure-bicep
• azure-private-dns

• 1 if your vnets are not peered you will need two different private dns zones. – Thomas Commented Feb 3 at 21:44
• I cannot use different private dns zones. As I said, private dns zone name has to be "privatelink.vaultcore.azure" in this case, KeyVault referencing just works that way (I'd be glad to be wrong here). For now I am trying to use a Powershell script to overwrite the private dns zone relevant A record with all the IP addresses in the middle of the ARM template deployment. It's not ideal, I know, but bicep did not give me enough control to make this work. If that fails I might just create duplicate KeyVaults, one per function app that has references to secrets. – Sahar LaOr Commented Feb 6 at 8:57
• private dns zone name only needs to be unique per resource group. if you have app1 and vnet1 in rg1 and app2 and vnet2 in rg2, you can create a private dns zone for kv in rg1 and rg2. – Thomas Commented Feb 6 at 18:38
• Even so, I do not have the option of creating a separate resource group here. I found a way around it, posting it as an answer. – Sahar LaOr Commented Feb 12 at 12:49

1 Answer 1

I ended up using a powershell script that runs in place of the resource creating the DNS zone group. It saves the current A record IP addresses, creates the group himself, then appends the IP addresses back into the DNS record. That was the only way I found of making it work in my situation.

Bicep resource used: Microsoft.Resources/deploymentScripts@2020-10-01

And here is the script:

param(
  [string] $deploymentResourceGroupName,
  [string] $keyVaultName,
  [string] $privateEndpointName,
  [string] $dnsGroupName
)

# 1
try {
  $originalDnsZoneRecord = Get-AzPrivateDnsRecordSet -ResourceGroupName $deploymentResourceGroupName -ZoneName "privatelink.vaultcore.azure" -Name $keyVaultName -RecordType A -ErrorAction SilentlyContinue
  $originalDnsRecordIpAddresses = if ($originalDnsZoneRecord) {
      $originalDnsZoneRecord.Records.Ipv4Address
  }
}
catch {
  $originalDnsRecordIpAddresses = $()
}

# 2
$zone = Get-AzPrivateDnsZone -ResourceGroupName $deploymentResourceGroupName -Name "privatelink.vaultcore.azure"
$dnsZoneGroupConfig = New-AzPrivateDnsZoneConfig -Name $dnsGroupName -PrivateDnsZoneId $zone.ResourceId
$originalDnsZoneGroup = Get-AzPrivateDnsZoneGroup -ResourceGroupName $deploymentResourceGroupName -PrivateEndpointName $privateEndpointName
$dnsZoneGroupName = ""
if ($originalDnsZoneGroup.Count -eq 0) {
  $dnsZoneGroupName = "vaultDnsZoneGroup"
  Set-AzPrivateDnsZoneGroup -ResourceGroupName $deploymentResourceGroupName -PrivateEndpointName $privateEndpointName -name $dnsZoneGroupName -PrivateDnsZoneConfig $dnsZoneGroupConfig
} else {
  $dnsZoneGroupName = $originalDnsZoneGroup.Name
}

# 3
$newDnsZoneGroup = Get-AzPrivateDnsZoneGroup -ResourceGroupName $deploymentResourceGroupName -PrivateEndpointName $privateEndpointName -name $dnsZoneGroupName
$newIp = $newDnsZoneGroup.PrivateDnsZoneConfigs.RecordSets.IpAddresses

$currentDnsZoneRecord = Get-AzPrivateDnsRecordSet -ResourceGroupName $deploymentResourceGroupName -ZoneName "privatelink.vaultcore.azure" -Name $keyVaultName -RecordType A
$currentDnsRecordIpAddresses = $currentDnsZoneRecord.Records.Ipv4Address

# 4
$allIpAddresses = $($originalDnsRecordIpAddresses; $newIp)
foreach ($ip in $allIpAddresses) {
  if ($currentDnsRecordIpAddresses -NotContains $ip) {
    Add-AzPrivateDnsRecordConfig -RecordSet $currentDnsZoneRecord -Ipv4Address $ip
  }
}
Set-AzPrivateDnsRecordSet -RecordSet $currentDnsZoneRecord