Address resolution by wild-endeavor · Pull Request #1567 · flyteorg/flytekit

wild-endeavor · 2023-03-29T22:53:16Z

TL;DR

This implements in FlyteRemote a quick feature to help the fetching of inputs and outputs and decks from a flyte:// uri.

Type

Bug Fix
Feature
Plugin

Are all requirements met?

Complete description

Other PRs:
flyteorg/flyteadmin#546
flyteorg/flyteidl#391

Tracking Issue

No issues, this is just a use-ability enhancement we found

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

codecov · 2023-03-29T23:06:02Z

Codecov Report

Merging #1567 (0b7ee45) into master (c65397d) will decrease coverage by 0.05%.
The diff coverage is 42.18%.

@@            Coverage Diff             @@
##           master    #1567      +/-   ##
==========================================
- Coverage   71.04%   71.00%   -0.05%     
==========================================
  Files         335      336       +1     
  Lines       30661    30713      +52     
  Branches     5558     5565       +7     
==========================================
+ Hits        21783    21807      +24     
- Misses       8336     8362      +26     
- Partials      542      544       +2

Impacted Files	Coverage Δ
flytekit/clients/friendly.py	`4.89% <0.00%> (-0.15%)`	⬇️
flytekit/clients/raw.py	`7.37% <0.00%> (-0.13%)`	⬇️
flytekit/tools/interactive.py	`0.00% <0.00%> (ø)`
flytekit/remote/remote.py	`39.39% <32.25%> (-0.48%)`	⬇️
flytekit/deck/deck.py	`48.64% <50.00%> (+4.74%)`	⬆️
tests/flytekit/unit/deck/test_deck.py	`100.00% <100.00%> (ø)`
tests/flytekit/unit/remote/test_remote.py	`99.45% <100.00%> (+0.04%)`	⬆️

…his is going away anyways Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

* Multi arch imageSpec (#1630) Multi arch imageSpec (#1630) Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Add executor_path and applications_path to spark config (#1634) * Add executor_path and applications_path to spark config Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> * Add support for env vars to pyflyte run (#1617) * Add support for env vars to pyflyte run Signed-off-by: Kevin Su <pingsutw@apache.org> * bump idl Signed-off-by: Kevin Su <pingsutw@apache.org> * update doc Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Fetch task executions in dynamic (#1636) * fetch task executions in dynamic Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> * Added metrics command to pyflyte (#1513) Signed-off-by: Daniel Rammer <daniel@union.ai> * Add http_proxy to client & Fix deviceflow (#1611) * Add http_proxy to client & Fix deviceflow RB=3890720 Signed-off-by: byhsu <byhsu@linkedin.com> * nit Signed-off-by: byhsu <byhsu@linkedin.com> * lint! Signed-off-by: byhsu <byhsu@linkedin.com> --------- Signed-off-by: byhsu <byhsu@linkedin.com> Co-authored-by: byhsu <byhsu@linkedin.com> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> * Improve variable names (#1642) Signed-off-by: byhsu <byhsu@linkedin.com> Co-authored-by: byhsu <byhsu@linkedin.com> * Address resolution (#1567) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * pyflyte run supports pickle (#1646) Signed-off-by: Kevin Su <pingsutw@apache.org> * Wait for the pod plugin instead of flytekit (#1647) Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Beautify deviceflow prompt (#1625) * Beautify deviceflow prompt Signed-off-by: byhsu <byhsu@linkedin.com> * lint! Signed-off-by: byhsu <byhsu@linkedin.com> * lint Signed-off-by: byhsu <byhsu@linkedin.com> --------- Signed-off-by: byhsu <byhsu@linkedin.com> Co-authored-by: byhsu <byhsu@linkedin.com> * Improve flytekit register (#1643) * Fix pyflyte register Signed-off-by: byhsu <byhsu@linkedin.com> * revert Signed-off-by: byhsu <byhsu@linkedin.com> * lint Signed-off-by: byhsu <byhsu@linkedin.com> --------- Signed-off-by: byhsu <byhsu@linkedin.com> Co-authored-by: byhsu <byhsu@linkedin.com> * Pass verify flag to all authenticators (#1641) Signed-off-by: byhsu <byhsu@linkedin.com> * Allow annotated FlyteFile as task input argument (#1632) * fix: Allow annotated FlyteFile as task input argument Using an annotated FlyteFile type as an input to a task was previously impossible due to an exception being raised in `FlyteFilePathTransformer.to_python_value`. This commit applies the fix previously used in `FlyteFilePathTransformer.to_literal` to permit using annotated FlyteFiles as either inputs and outputs of a task. Issue: #3424 Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> * refactor: Unified handling of annotated types in type engine Issue: #3424 Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> * fix: Use py3.8-compatible types in type engine tests Issue: #3424 Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> --------- Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> * Use logger instead of print statement in sqlalchemy plugin (#1651) * use logging info instead of print Signed-off-by: wirthual <wirthra@gmail.com> * isorted files Signed-off-by: wirthual <wirthra@gmail.com> * import root logger from flytekit Signed-off-by: wirthual <wirthra@gmail.com> --------- Signed-off-by: wirthual <wirthra@gmail.com> * Map over notebook task (#1650) * map over notebook Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * tests Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * add a flag Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * nit Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix tests Signed-off-by: Kevin Su <pingsutw@apache.org> * lint Signed-off-by: Kevin Su <pingsutw@apache.org> --------- Signed-off-by: Kevin Su <pingsutw@apache.org> * Support single literals in tiny url (#1654) Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> * Add support overriding image (#1652) Signed-off-by: Kevin Su <pingsutw@apache.org> * Fix ability to pass None to task with Optional kwarg, add test (#1657) Signed-off-by: Fabio Grätz <fabiogratz@googlemail.com> Co-authored-by: Fabio Grätz <fabiogratz@googlemail.com> * Regenerate plugins requirements Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Regenerate plugins requirements and linting Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> * Regenerate whylogs requirements Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Kevin Su <pingsutw@apache.org> Signed-off-by: Daniel Rammer <daniel@union.ai> Signed-off-by: byhsu <byhsu@linkedin.com> Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Signed-off-by: eduardo apolinario <eapolinario@users.noreply.github.com> Signed-off-by: Adrian Rumpold <a.rumpold@gmail.com> Signed-off-by: wirthual <wirthra@gmail.com> Signed-off-by: Fabio Grätz <fabiogratz@googlemail.com> Co-authored-by: Kevin Su <pingsutw@apache.org> Co-authored-by: Dan Rammer <daniel@union.ai> Co-authored-by: ByronHsu <byronhsu1230@gmail.com> Co-authored-by: byhsu <byhsu@linkedin.com> Co-authored-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com> Co-authored-by: eduardo apolinario <eapolinario@users.noreply.github.com> Co-authored-by: Adrian Rumpold <a.rumpold@gmail.com> Co-authored-by: wirthual <wirthra@gmail.com> Co-authored-by: Fabio M. Graetz, Ph.D <fabiograetz@googlemail.com> Co-authored-by: Fabio Grätz <fabiogratz@googlemail.com>

add calls to new endpoints

54c3b92

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

wild-endeavor added 2 commits March 30, 2023 15:33

exception type somehow changed overnight, catching everything since t…

266c8df

…his is going away anyways Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

use new endpoints

982f831

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

kumare3 reviewed Apr 21, 2023

View reviewed changes

Comment thread flytekit/clients/friendly.py Outdated

kumare3 reviewed Apr 21, 2023

View reviewed changes

Comment thread flytekit/remote/remote.py Outdated

wild-endeavor added 5 commits May 1, 2023 18:34

bump idl

2d63058

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

move out ipython check

be91a25

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

merge master

574603a

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

bump doc requirements

0a0953f

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

add unit test

995f2f8

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

wild-endeavor changed the title ~~[wip] address resolution~~ Address resolution May 2, 2023

wild-endeavor marked this pull request as ready for review May 2, 2023 20:44

wild-endeavor requested review from cosmicBboy, eapolinario and pingsutw as code owners May 2, 2023 20:44

wild-endeavor added 2 commits May 16, 2023 11:27

merge master

b27910d

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

delete weird file

200315a

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

pingsutw previously approved these changes May 16, 2023

View reviewed changes

revert bad test

0b7ee45

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

wild-endeavor dismissed pingsutw’s stale review via 0b7ee45 May 16, 2023 19:55

pingsutw approved these changes May 16, 2023

View reviewed changes

wild-endeavor merged commit 01830e4 into master May 17, 2023

eapolinario pushed a commit that referenced this pull request Jul 10, 2023

Address resolution (#1567)

7bbd44b

Signed-off-by: Yee Hing Tong <wild-endeavor@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Address resolution#1567

Address resolution#1567
wild-endeavor merged 11 commits into
masterfrom
resolve

wild-endeavor commented Mar 29, 2023 •

edited

Loading

Uh oh!

codecov Bot commented Mar 29, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

wild-endeavor commented Mar 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TL;DR

Type

Are all requirements met?

Complete description

Tracking Issue

Uh oh!

codecov Bot commented Mar 29, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wild-endeavor commented Mar 29, 2023 •

edited

Loading

codecov Bot commented Mar 29, 2023 •

edited

Loading