Cannot retrieve Authentication Method in AzureDataExplorerHook using the custom connection fields

[josh-fell]

Apache Airflow Provider(s)

microsoft-azure

Versions of Apache Airflow Providers

apache-airflow-providers-microsoft-azure==1!3.1.1

Apache Airflow version

2.1.3 (latest released)

Operating System

Debian GNU/Linux

Deployment

Astronomer

Deployment details

No response

What happened

When attempting to connect to Azure Data Explorer, the following exception is thrown even though the corresponding "Authentication Method" field is populated in the Azure Data Explorer connection form:

airflow.exceptions.AirflowException: Extra connection option is missing required parameter: `auth_method`

Airflow Connection:

What you expected to happen

Airflow tasks should be able to connect to Azure Data Explorer when properly populating the custom connection form. Or, at the very least, the above exception should not be thrown when an Authentication Method is provided in the connection.

How to reproduce

1. Install the Microsoft Azure provider and create an Airflow Connection with the type Azure Data Explorer.
2. Provide all values for "Auth Username", "Auth Password", "Tenant ID", and "Authentication Method".
3. Finally execute a task which uses the AzureDataExplorerHook

Anything else

Looks like there are a few issues in the AzureDataExplorerHook:

• The get_required_param() method is being passed a value of "auth_method" from the Extras field in the connection form. The Extras field is no longer exposed in the connection form and would never be able to be provided.

def get_required_param(name: str) -> str:
            """Extract required parameter from extra JSON, raise exception if not found"""
            value = conn.extra_dejson.get(name)
            if not value:
                raise AirflowException(f'Extra connection option is missing required parameter: `{name}`')
            return value

        auth_method = get_required_param('auth_method') or get_required_param(
            'extra__azure_data_explorer__auth_method'
        )

• The custom fields mapping for "Tenant ID" and "Authentication Method" are switched so even if these values are provided in the connection form they will not be used properly in the hook.

@staticmethod
    def get_connection_form_widgets() -> Dict[str, Any]:
        """Returns connection widgets to add to connection form"""
        from flask_appbuilder.fieldwidgets import BS3PasswordFieldWidget, BS3TextFieldWidget
        from flask_babel import lazy_gettext
        from wtforms import PasswordField, StringField

        return {
            "extra__azure_data_explorer__auth_method": StringField(
                lazy_gettext('Tenant ID'), widget=BS3TextFieldWidget()
            ),
            "extra__azure_data_explorer__tenant": StringField(
                lazy_gettext('Authentication Method'), widget=BS3TextFieldWidget()
            ),
...

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct