GH-44308: [C++][FS][Azure] Implement SAS token authentication

[Tom-Newton]

Rationale for this change

SAS token auth is sometimes useful and it the last one we haven't implemented.

What changes are included in this PR?

• Implement ConfigureSasCredential
• Update AzureOptions::FromUri so that simply appending a SAS token to a blob storage URI works. e.g. AzureOptions::FromUri("abfs://file_system@account.dfs.core.windows.net/?se=2024-12-12T18:57:47Z&sig=pAs7qEBdI6sjUhqX1nrhNAKsTY%2B1SqLxPK%2BbAxLiopw%3D&sp=racwdxylti&spr=https,http&sr=c&sv=2024-08-04")
  • SAS tokens are made up of a bunch of URI query parameters that I'm not sure we can exhaustively list.
  • Therefore we now assume that any unrecognised URI query parameters are assumed to be part of a SAS token, instead of returning an error status.
• Update CopyFile to use StartCopyFromUri instead of CopyFromUri
  • This avoids the need to generate SAS tokens.
  • Supports blobs bigger than 256MiB
  • This makes [C++][FS][Azure] Test CopyFile() with non account key credential #41315 redundant

Are these changes tested?

Yes

• Added new tests for authenticating with SAS and doing some operations including CopyFile
• Added new tests for AzureOptions::FromUri with a SAS token.

I also made sure to run the tests which connect to real blob storage.

Are there any user-facing changes?

• SAS token in now supported
• Unrecognised URI query parameters are ignored by AzureOptions::FromUri instead of failing fast. IMO this is a regression but still the best option to support SAS token.

• GitHub Issue: [C++][FS][Azure] SAS token authentication #44308

[kou]

We don't have the SAS token specification that includes parameter names used by a SAS token, right?

[Tom-Newton]

Yeah, I had a quick search and couldn't find what we need. If you think it's important I can try a bit harder. The closest I found seemed to be unabbreviated versions of what actually appears in the sas token.

[kou]

I found it: https://learn.microsoft.com/en-us/rest/api/storageservices/create-user-delegation-sas#construct-a-user-delegation-sas

There are many parameters but can we check them...?

[Tom-Newton]

I think that doc is only for user delegated SAS tokens so I unioned it with the parameters for account and service SAS tokens and hopefully the spec is slowly changing.

I wasn't really confident on the best way to define a constant set of strings to do contains checks against in C++. Since there are only 27 values, I ended up with a constexpr array and std::find but please let me know if this is not a good option.

[Tom-Newton]

Wait... I might have just accidentally worked out how to avoid any of the special authentication stuff for copying...

[kou]

+1

[kou]

Can we use std::binary_search() with sorted sas_token_query_parameters?

https://en.cppreference.com/w/cpp/algorithm/binary_search

[Tom-Newton]

Could do but I would be a bit concerned about keeping sas_token_query_parameters sorted. It looks like C++20 allows using std::sort with constexpr but I believe arrow currently uses C++17. If you are concerned about the complexity of the lookup I think my preference would be to use a std::set and forget about trying to make it a constexpr.

[kou]

std::set is OK.

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit ba2b9e5.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 16 possible false positives for unstable benchmarks that are known to sometimes produce them.