I'm integrating Azure Data Lake Storage (ADLS) as a destination in our on-premises application running on Kubernetes. To authenticate, I'm using a Java KeyStore (JCEKS) file containing our Azure Service Principal (SPN) credentials. Despite configuring everything as per the documentation, I'm encountering a NullPointerException related to the authentication endpoint.

Current Setup:

Proxy Configuration: Proxy chains are set up, and connectivity to Azure has been verified.​

JCEKS File: The JCEKS file includes the following entries:

• fs.azure.account.oauth2.client.endpoint

• fs.azure.account.oauth2.client.id

• fs.azure.account.oauth2.client.secret

Hadoop Configuration: The path to the JCEKS file is specified in the hadoop.security.credential.provider.path property.

Issue Encountered:

When attempting to connect to ADLS, the application throws a NullPointerException, indicating that it cannot resolve the authEndpoint. This suggests that the authentication endpoint value isn't being retrieved correctly from the JCEKS file.

Troubleshooting Steps Taken:

1. Verified JCEKS Content: Ensured that the JCEKS file contains the correct aliases and values for the SPN credentials.
2. Checked File Accessibility: Confirmed that the application has read access to the JCEKS file and that the file path is correct.
3. Hadoop Configuration: Ensured that the following properties are set:

• fs.azure.account.auth.type = OAuth
• fs.azure.account.oauth.provider.type = .apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider

4. Credential Retrieval: Used the Hadoop Credential Shell to list and retrieve credentials from the JCEKS file, which worked as expected.
5. Logging: Enabled detailed logging for the Hadoop Azure module but didn't find additional insights.

Moreover, for more in-depth reference, I checked the Hadoop codebase where the exception is thrown in this line: .java#L1146C15-L1146C82 while fetching the authEndpoint in the below class:

  public AccessTokenProvider getTokenProvider() throws TokenAccessProviderException {

The getMandatoryPasswordString method is returning null for FS_AZURE_ACCOUNT_OAUTH_CLIENT_ENDPOINT, leading to the NullPointerException. This is because it could not derive the client endpoint from my application.

Has anyone encountered a similar issue where Hadoop fails to retrieve SPN credentials from a JCEKS file, resulting in a NullPointerException? Any insights into potential misconfigurations or overlooked steps would be greatly appreciated.

NOTE: Hadoop version: 3.3.2