Airflow dynamic task mapping is not working with Datahub's Airflow lineage plugin #26129
Replies: 5 comments 1 reply
-
|
Thanks for opening your first issue here! Be sure to follow the issue template! |
Beta Was this translation helpful? Give feedback.
-
|
Have you contacted the plugin's maintainers? This sounds like a problem with the plugin, and even if not, its maintainers should have a better understanding on what is causing the issue. If you are convinced that this is caused by something in the main Airflow codebase, some logs to provide insight on what's going on would be very helpful. There is close to nothing we can do with the little information available here. |
Beta Was this translation helpful? Give feedback.
-
|
Surely this is an interesting issue and good point but totally mis-directed. You should raise it in DataHub as they are suporting it not us. |
Beta Was this translation helpful? Give feedback.
-
|
I converted it to a discussion - and if you get more information after talking to datahub, we can continue discussing here @langelrhea |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much for your comment! Sorry for the misunderstanding, I will keep this discussion updated in order to help similar issues from others users |
Beta Was this translation helpful? Give feedback.
-
|
Surely. It would be fantastic if you come back after talking to DataHub and report your findings here. |
Beta Was this translation helpful? Give feedback.
-
Apache Airflow version
2.3.4
What happened
Working with Airflow if you follow the steps from Datahub Integration (https://datahubproject.io/docs/lineage/airflow/) the first step is install the required dependency in your airflow (pip install acryl-datahub-airflow-plugin) after install this dependency, if I run my DAG this one fails. However if you uninstall this package, this DAG is completed successfully.
What you think should happen instead
No response
How to reproduce
Execute a DAG and see that this one is completed, after that install datahub plugin (pip install acryl-datahub-airflow-plugin) this DAG won't be completed, then if you uninstall the same package (pip uninstall acryl-datahub-airflow-plugin) this DAG will be completed as the first time.
Operating System
Ubuntu 20.04.5 LTS (Focal Fossa)
Versions of Apache Airflow Providers
apache-airflow-providers-cncf-kubernetes == 4.3.0
apache-airflow-providers-ftp==3.0.0
apache-airflow-providers-http==3.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-sqlite==3.0.0
Deployment
Virtualenv installation
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions