Skip to content

Support PodTemplate in ArrayNode#2088

Merged
hamersaw merged 4 commits into
masterfrom
feature/arraynode-pod-template-support
Jan 5, 2024
Merged

Support PodTemplate in ArrayNode#2088
hamersaw merged 4 commits into
masterfrom
feature/arraynode-pod-template-support

Conversation

@hamersaw

@hamersaw hamersaw commented Jan 4, 2024

Copy link
Copy Markdown
Member

Tracking issue

NA

Why are the changes needed?

Currently, ArrayNode does not support operating over tasks that use PodTemplates. This is a very powerful resource that should be enabled.

What changes were proposed in this pull request?

Updating the ArrayNode construct to use underlying function config.

How was this patch tested?

locally / cloud / etc with many workflows, for example:

@task(pod_template=PodTemplate(
    primary_container_name="primary",
    labels={"mylabel": "myvalue"},
))
def echo_2(x: int) -> int:
    time.sleep(1)
    return x

@workflow
def gen_wf_2() -> typing.List[typing.Optional[int]]:
    return map_task(echo_2)(x=list(range(5)))

Setup process

NA

Screenshots

NA

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

NA

Docs link

NA

Signed-off-by: Daniel Rammer <daniel@union.ai>
Signed-off-by: Daniel Rammer <daniel@union.ai>
@codecov

codecov Bot commented Jan 4, 2024

Copy link
Copy Markdown

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (9bf7afa) 85.76% compared to head (282ceef) 85.75%.
Report is 1 commits behind head on master.

Files Patch % Lines
flytekit/core/array_node_map_task.py 57.14% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2088      +/-   ##
==========================================
- Coverage   85.76%   85.75%   -0.01%     
==========================================
  Files         313      313              
  Lines       23426    23433       +7     
  Branches     3510     3511       +1     
==========================================
+ Hits        20092    20096       +4     
- Misses       2727     2730       +3     
  Partials      607      607              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pingsutw and others added 2 commits January 4, 2024 13:50
Signed-off-by: Kevin Su <pingsutw@apache.org>
Signed-off-by: Daniel Rammer <daniel@union.ai>
@hamersaw hamersaw marked this pull request as ready for review January 5, 2024 14:54
@hamersaw hamersaw merged commit b2f3b77 into master Jan 5, 2024
@hamersaw hamersaw deleted the feature/arraynode-pod-template-support branch January 5, 2024 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants