Surface databricks API errors as a separate exception

[amoghrajesh]

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Related to #68974 (comment)

What

DatabricksHook._do_api_call / _a_do_api_call flattened every HTTP error into a bare AirflowException whose only detail was a formatted message string. Callers that need to react to a specific response (e.g. a 404 for a run whose history has expired) had no structured way to do so short of parsing that string which is brittle.

Change

Introduces DatabricksApiError(AirflowException) carrying http_status_code, and raises it (with the original error chained via from) from both the sync and async API call path

Backcompat

Fully backcompatible:

• DatabricksApiError is an AirflowException subclass, and nothing catches these errors by a more specific type today, so existing behavior and pytest.raises(AirflowException) assertions are unchanged. It also removes two raise AirflowException

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[amoghrajesh]

@shahar1's approval is valuable here, we were discussing about AirflowException recently!

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test

[amoghrajesh]

Wrong backport!

[potiuk]

Wrong backport!

Fix in #69057