Example DAGs import error

[anirudhbagri]

Apache Airflow version: 2.0.0b2

Kubernetes version (if you are using kubernetes) (use kubectl version):

Environment:

• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):macOS
• Kernel (e.g. uname -a):
• Install tools:
• Others:

What happened:
Running airflow db init followed by airflow dag list give the following error:

[2020-11-10 19:04:24,553] {dagbag.py:297} ERROR - Failed to import: /Users/abagri/Workspace/airflow2.0/venv/lib/python3.8/site-packages/airflow/example_dags/example_kubernetes_executor_config.py
Traceback (most recent call last):
  File "/Users/abagri/Workspace/airflow2.0/venv/lib/python3.8/site-packages/airflow/models/dagbag.py", line 294, in _load_modules_from_file
    loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/abagri/Workspace/airflow2.0/venv/lib/python3.8/site-packages/airflow/example_dags/example_kubernetes_executor_config.py", line 23, in <module>
    from kubernetes.client import models as k8s
ModuleNotFoundError: No module named 'kubernetes'
[2020-11-10 19:04:24,558] {dagbag.py:297} ERROR - Failed to import: /Users/abagri/Workspace/airflow2.0/venv/lib/python3.8/site-packages/airflow/example_dags/example_dag_decorator.py
Traceback (most recent call last):
  File "/Users/abagri/Workspace/airflow2.0/venv/lib/python3.8/site-packages/airflow/models/dagbag.py", line 294, in _load_modules_from_file
    loader.exec_module(new_module)
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/abagri/Workspace/airflow2.0/venv/lib/python3.8/site-packages/airflow/example_dags/example_dag_decorator.py", line 25, in <module>
    from airflow.providers.http.operators.http import SimpleHttpOperator
ModuleNotFoundError: No module named 'airflow.providers'


*****// Followed by name of other DAG, omiting here *******

What you expected to happen:
Expect the example dags to import files

How to reproduce it:

1. Run airflow db init
2. Run airflow dags list

Anything else we need to know:

[jbampton]

Pretty sure it should be airflow dags list not airflow dag list.

Worked for me on the latest master.

[anirudhbagri]

my bad. it is airflow dags list.
But the error is still showing up.

[turbaszek]

@ashb @potiuk @dimberman I think the problem is that we are using providers and kubernetes code in core examples. The one with SimpleHttpOperator I know how to fix, but how do we want to approach k8s stuff?

[anirudhbagri]

You are right @turbaszek.
Once I added apache-airflow-providers-http to requirements the error goes away.

[potiuk]

This is also part of #11435 - we have a few more core-> providers deps to remove. This is a great start :)