Apache Airflow version: 2.0+
Apache Airflow provider and version: apache-airflow-providers-google==2.2.0
What happened:
My BigQueryToMySqlOperator task always fail with the following error message.
Traceback (most recent call last):
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1113, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1287, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1317, in _execute_task
result = task_copy.execute(context=context)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/transfers/bigquery_to_mysql.py", line 166, in execute
for rows in self._bq_get_data():
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/transfers/bigquery_to_mysql.py", line 138, in _bq_get_data
response = cursor.get_tabledata(
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py", line 2508, in get_tabledata
return self.hook.get_tabledata(*args, **kwargs)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/cloud/hooks/bigquery.py", line 1284, in get_tabledata
rows = self.list_rows(dataset_id, table_id, max_results, selected_fields, page_token, start_index)
File "/opt/python3.8/lib/python3.8/site-packages/airflow/providers/google/common/hooks/base_google.py", line 412, in inner_wrapper
raise AirflowException(
airflow.exceptions.AirflowException: You must use keyword arguments in this methods rather than positional
What you expected to happen:
I expect the task to move data from BigQuery to MySql.
How to reproduce it:
You will need a working gcp connection and mysql connection as well as some sample data to test it out.
An example of the BigQueryToMySqlOperator can be pull from the Operator description.
transfer_data = BigQueryToMySqlOperator(
task_id='task_id',
dataset_table='origin_bq_table',
mysql_table='dest_table_name',
replace=True,
)Anything else we need to know:
The operator is having this issue because the cursor in BigQueryToMySqlOperator calls get_tabledata which calls list_rows with positional arguments.
Calling list_rows with positional arguments triggers the function wrapper fallback_to_default_project_id, which does NOT allow for positional arguements.
Apache Airflow version: 2.0+
Apache Airflow provider and version: apache-airflow-providers-google==2.2.0
What happened:
My
BigQueryToMySqlOperatortask always fail with the following error message.What you expected to happen:
I expect the task to move data from BigQuery to MySql.
How to reproduce it:
You will need a working gcp connection and mysql connection as well as some sample data to test it out.
An example of the BigQueryToMySqlOperator can be pull from the Operator description.
Anything else we need to know:
The operator is having this issue because the cursor in BigQueryToMySqlOperator calls get_tabledata which calls list_rows with positional arguments.
Calling list_rows with positional arguments triggers the function wrapper fallback_to_default_project_id, which does NOT allow for positional arguements.