Skip to content

Add parameter to pass role ARN to GlueJobOperator #33408

Merged
vincbeck merged 10 commits into
apache:mainfrom
erdos2n:main
Aug 15, 2023
Merged

Add parameter to pass role ARN to GlueJobOperator #33408
vincbeck merged 10 commits into
apache:mainfrom
erdos2n:main

Conversation

@erdos2n

@erdos2n erdos2n commented Aug 15, 2023

Copy link
Copy Markdown
Contributor

GlueOperator currently requires iam_role_name and endpoint_url to fetch the ARN from AWS. This ARN eventually gets passed into the config for Glue Job.

Resolves #33248

@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Aug 15, 2023
@erdos2n

erdos2n commented Aug 15, 2023

Copy link
Copy Markdown
Contributor Author

@eladkal , @o-nikolas I added test cases for the role_arn, can you let me know if the tests were added correctly?
Thank you

Comment thread airflow/providers/amazon/aws/hooks/glue.py Outdated
Comment thread airflow/providers/amazon/aws/hooks/glue.py Outdated
Comment thread airflow/providers/amazon/aws/hooks/glue.py Outdated
Comment thread airflow/providers/amazon/aws/operators/glue.py Outdated
Comment thread tests/providers/amazon/aws/hooks/test_glue.py
Comment thread tests/providers/amazon/aws/hooks/test_glue.py

@vincbeck vincbeck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vincbeck

Copy link
Copy Markdown
Contributor

You have some static checks failure, please run them to fix these failures.

@erdos2n erdos2n changed the title [WIP] Issue ##33248 - GlueOperator IAM ARN Issue ##33248 - GlueOperator IAM ARN Aug 15, 2023
@vincbeck vincbeck changed the title Issue ##33248 - GlueOperator IAM ARN Add parameter to pass role ARN to GlueJobOperator Aug 15, 2023
@vincbeck vincbeck merged commit 60df705 into apache:main Aug 15, 2023
ferruzzi pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Aug 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GlueOperator: iam_role_arn as a parameter

2 participants