HADOOP-19736: ABFS. Support for new auth type: User-bound SAS #8051
+567
−13
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
| AUG_03_2023("2023-08-03"), | ||
| NOV_04_2024("2024-11-04"); | ||
| NOV_04_2024("2024-11-04"), | ||
| JULY_05_2025("2025-07-05"); |
There was a problem hiding this comment.
We should follow the same format: JUL_05_2025, what do you think?
| abfsClientContext); | ||
| } | ||
|
|
||
| public AbfsClientHandler(final URL baseUrl, |
There was a problem hiding this comment.
Java doc missing for the constructor
| this.sasTokenProvider = sasTokenProvider; | ||
| } | ||
|
|
||
| public AbfsClient(final URL baseUrl, |
| encryptionContextProvider, abfsClientContext, AbfsServiceType.DFS); | ||
| } | ||
|
|
||
| public AbfsDfsClient(final URL baseUrl, |
| case UserboundSASWithOAuth: | ||
| httpOperation.setRequestProperty(HttpHeaderConfigurations.AUTHORIZATION, | ||
| client.getAccessToken()); | ||
| httpOperation.setMaskForSAS(); //mask sig/oid from url for logs |
| tokenProvider, sasTokenProvider, encryptionContextProvider, | ||
| populateAbfsClientContext()); | ||
| } | ||
| else if (tokenProvider != null) { |
|
|
||
| LOG.trace("Initializing AbfsClient for {}", baseUrl); | ||
| if (tokenProvider != null) { | ||
| if(tokenProvider != null && sasTokenProvider != null){ |
There was a problem hiding this comment.
Space between if and (
| 3. Deployed in-Azure with the Azure VMs providing OAuth 2.0 tokens to the application, "Managed Instance". | ||
| 4. Using Shared Access Signature (SAS) tokens provided by a custom implementation of the SASTokenProvider interface. | ||
| 5. By directly configuring a fixed Shared Access Signature (SAS) token in the account configuration settings files. | ||
| 6. Using user-bound SAS auth type, which is requires OAuth 2.0 setup (point 2 above) and SAS setup (point 4 above) |
There was a problem hiding this comment.
Grammatical mistake: which requires or which is required?
|
|
||
| public static final String FS_AZURE_TEST_APP_SERVICE_PRINCIPAL_OBJECT_ID = "fs.azure.test.app.service.principal.object.id"; | ||
|
|
||
| public static final String FS_AZURE_END_USER_TENANT_ID = "fs.azure.test.end.user.tenant.id"; |
There was a problem hiding this comment.
Rename the variable to FS_AZURE_TEST_END_USER_TENANT_ID
There was a problem hiding this comment.
I think TEST should come first. Check for how other test related configs are defined. TestConfigurationKeys.java have them
| public static final String FS_AZURE_TEST_APP_SERVICE_PRINCIPAL_OBJECT_ID = "fs.azure.test.app.service.principal.object.id"; | ||
|
|
||
| public static final String FS_AZURE_END_USER_TENANT_ID = "fs.azure.test.end.user.tenant.id"; | ||
| public static final String FS_AZURE_END_USER_OBJECT_ID = "fs.azure.test.end.user.object.id"; |
| Dec19("2019-12-12"), | ||
| Feb20("2020-02-10"); | ||
| Feb20("2020-02-10"), | ||
| July5("2025-07-05"); |
There was a problem hiding this comment.
Same here should be JUL
| private final String sduoid; | ||
|
|
||
| public DelegationSASGenerator(byte[] userDelegationKey, String skoid, String sktid, String skt, String ske, String skv) { | ||
| public DelegationSASGenerator(byte[] userDelegationKey, String skoid, String sktid, String skt, String ske, String skv, String skdutid, String sduoid) { |
There was a problem hiding this comment.
add javadoc for what do all these params signify
| qb.addQuery("skv", skv); | ||
|
|
||
| //skdutid and sduoid are required for user bound SAS only | ||
| if(!Objects.equals(skdutid, EMPTY_STRING)){ |
|
|
||
| String stringToSign = sb.toString(); | ||
| LOG.debug("Delegation SAS stringToSign: " + stringToSign.replace("\n", ".")); | ||
| System.out.println("Delegation SAS stringToSign: " + stringToSign.replace("\n", ".")); |
|
|
||
| // Invokes the AAD v2.0 authentication endpoint with a client credentials grant to get an | ||
| // access token. See https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow. | ||
| private String getAuthorizationHeader(String accountName, String appID, String appSecret, String sktid) throws IOException { |
There was a problem hiding this comment.
include params in javadoc as well
| // Invokes the AAD v2.0 authentication endpoint with a client credentials grant to get an | ||
| // access token. See https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow. | ||
| private String getAuthorizationHeader(String accountName, String appID, String appSecret, String sktid) throws IOException { | ||
| String authEndPoint = String.format("https://login.microsoftonline.com/%s/oauth2/v2.0/token", sktid); |
There was a problem hiding this comment.
Add a constant for the string
| return "Bearer " + provider.getToken().getAccessToken(); | ||
| } | ||
|
|
||
| private byte[] getUserDelegationKey(String accountName, String appID, String appSecret, |
| private byte[] getUserDelegationKey(String accountName, String appID, String appSecret, | ||
| String sktid, String skt, String ske, String skv, String skdutid) throws IOException { | ||
|
|
||
| String method = "POST"; |
There was a problem hiding this comment.
we have constants for HTTP methods, can be used here
| final StringBuilder sb = new StringBuilder(128); | ||
| sb.append("https://"); | ||
| sb.append(account); | ||
| sb.append(".blob.core.windows.net/?restype=service&comp=userdelegationkey"); |
There was a problem hiding this comment.
Try to use constants as much as possible
| requestBody.append(skdutid); | ||
| requestBody.append("</DelegatedUserTid></KeyInfo>"); | ||
|
|
||
| // requestBody.append("<?xml version=\"1.0\" encoding=\"utf-8\"?><KeyInfo><Start>"); |
anujmodi2021
requested changes
Nov 3, 2025
|
|
||
| public static ApiVersion getCurrentVersion() { | ||
| return NOV_04_2024; | ||
| return JULY_05_2025; |
There was a problem hiding this comment.
Are we bumping up version for all the requests?
Wasn't this version bump only needed for Auth related APIs?
| this.sasTokenProvider = sasTokenProvider; | ||
| } | ||
|
|
||
| public AbfsClient(final URL baseUrl, |
There was a problem hiding this comment.
Should we simply combine the 3 constructors to accept both AccessTokenProvider, SASTokenProvider and the caller can set what it has and null as other?
| abfsClientContext); | ||
| } | ||
|
|
||
| public AbfsClientHandler(final URL baseUrl, |
There was a problem hiding this comment.
Same as above. We can combine into a single constructor.
But do check if there are any caveats there or a risk of running into NPE
| final AbfsClientContext abfsClientContext) throws IOException { | ||
| URL dfsUrl = changeUrlFromBlobToDfs(baseUrl); | ||
| if (tokenProvider != null) { | ||
| if (tokenProvider != null && sasTokenProvider != null) { |
There was a problem hiding this comment.
Apart from UDS, can there be a case where caller can send both as not null? Makes ure no flow leads to this and also add a test around it.
| encryptionContextProvider, abfsClientContext, AbfsServiceType.DFS); | ||
| } | ||
|
|
||
| public AbfsDfsClient(final URL baseUrl, |
There was a problem hiding this comment.
Is this only for DFS Client?
| * @return sasTokenProvider object based on configurations provided | ||
| * @throws AzureBlobFileSystemException | ||
| */ | ||
| public SASTokenProvider getSASTokenProviderForUserBoundSAS() throws AzureBlobFileSystemException { |
There was a problem hiding this comment.
nit: Can be renamed as getUserBoundSASTokenProvider
| getTokenProviderClass(authType, FS_AZURE_SAS_TOKEN_PROVIDER_TYPE, | ||
| null, SASTokenProvider.class); | ||
|
|
||
| if (customSasTokenProviderImplementation == null) { |
There was a problem hiding this comment.
Similar check for OAuth also needed right?
For UBS, Cx must configure OAuth as well?
| LOG.trace("Fetching SAS Token Provider"); | ||
| sasTokenProvider = abfsConfiguration.getSASTokenProvider(); | ||
| } else { | ||
| } else if(authType == AuthType.UserboundSASWithOAuth){ |
There was a problem hiding this comment.
Here also, it can be combined into a single call with both passed.
|
|
||
| public static final String FS_AZURE_TEST_APP_SERVICE_PRINCIPAL_OBJECT_ID = "fs.azure.test.app.service.principal.object.id"; | ||
|
|
||
| public static final String FS_AZURE_END_USER_TENANT_ID = "fs.azure.test.end.user.tenant.id"; |
There was a problem hiding this comment.
I think TEST should come first. Check for how other test related configs are defined. TestConfigurationKeys.java have them
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
JIRA: https://issues.apache.org/jira/browse/HADOOP-19736
Adding support for new authentication type: user bound SAS
How was this patch tested?
Test suite will be run for the patch