Skip to content

node_dependency_hints for dynamic tasks#2015

Merged
eapolinario merged 20 commits into
flyteorg:masterfrom
Tom-Newton:tomnewton/upsream_output_entity_hints
Jan 31, 2024
Merged

node_dependency_hints for dynamic tasks#2015
eapolinario merged 20 commits into
flyteorg:masterfrom
Tom-Newton:tomnewton/upsream_output_entity_hints

Conversation

@Tom-Newton

@Tom-Newton Tom-Newton commented Dec 1, 2023

Copy link
Copy Markdown
Contributor

Tracking issue

Closes flyteorg/flyte#4516

Why are the changes needed?

Streamline launching launchplans from dynamic tasks.

What changes were proposed in this pull request?

How was this patch tested?

Updated unit tests. I have also been using very similar changes in a real flyte deployment for weeks.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly. - I've updated docstrings. Please let me know if there is more documentation I should update.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

@Tom-Newton

Copy link
Copy Markdown
Contributor Author

Looks like I created a circular imports with the type hints when rebasing

Comment thread flytekit/core/python_function_task.py
@Tom-Newton Tom-Newton force-pushed the tomnewton/upsream_output_entity_hints branch from 6920d5b to 25fa4dd Compare December 13, 2023 18:53
@Tom-Newton

Tom-Newton commented Dec 13, 2023

Copy link
Copy Markdown
Contributor Author

@pingsutw do you think we should go ahead with this change? The issue is still tagged as needs-decision.

If yes, I'll write a unit test for this and request review.

@codecov

codecov Bot commented Dec 13, 2023

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (8861d0a) 85.80% compared to head (5901cff) 85.80%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #2015   +/-   ##
=======================================
  Coverage   85.80%   85.80%           
=======================================
  Files         313      313           
  Lines       23579    23594   +15     
  Branches     3530     3534    +4     
=======================================
+ Hits        20232    20246   +14     
  Misses       2738     2738           
- Partials      609      610    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Tom-Newton Tom-Newton force-pushed the tomnewton/upsream_output_entity_hints branch 3 times, most recently from 23aaf8f to b629c6f Compare December 14, 2023 14:11
@Tom-Newton Tom-Newton changed the title WIP Output entity hints for dynamic tasks Output entity hints for dynamic tasks Dec 14, 2023
@Tom-Newton Tom-Newton force-pushed the tomnewton/upsream_output_entity_hints branch 3 times, most recently from 7d3e25a to 5fc0f47 Compare December 15, 2023 14:15
@Tom-Newton Tom-Newton changed the title Output entity hints for dynamic tasks node_dependency_hints for dynamic tasks Dec 15, 2023
@Tom-Newton Tom-Newton marked this pull request as ready for review December 15, 2023 14:16
@Tom-Newton Tom-Newton force-pushed the tomnewton/upsream_output_entity_hints branch 2 times, most recently from 8f8e46d to 920a5d2 Compare December 19, 2023 20:52
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
@Tom-Newton Tom-Newton force-pushed the tomnewton/upsream_output_entity_hints branch from 920a5d2 to a29cd98 Compare December 20, 2023 00:15
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
@Tom-Newton Tom-Newton force-pushed the tomnewton/upsream_output_entity_hints branch from a29cd98 to 62c5209 Compare December 20, 2023 00:15
Signed-off-by: Thomas Newton <thomas.w.newton@gmail.com>
@Tom-Newton Tom-Newton requested a review from pingsutw December 21, 2023 10:27
…ntity_hints

Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com>
@eapolinario eapolinario merged commit 35f8a8c into flyteorg:master Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Core feature] [flytekit] Add "entity hints" to dynamic tasks

3 participants