Skip to content

[TUTORIAL][ANSOR] Using the template-free auto-scheduler on CPU#6488

Merged
merrymercy merged 8 commits intoapache:masterfrom
merrymercy:ansor-cpu-tutorial
Sep 17, 2020
Merged

[TUTORIAL][ANSOR] Using the template-free auto-scheduler on CPU#6488
merrymercy merged 8 commits intoapache:masterfrom
merrymercy:ansor-cpu-tutorial

Conversation

@merrymercy
Copy link
Copy Markdown
Member

@merrymercy merrymercy commented Sep 16, 2020 •
edited
Loading

Add a tutorial on how to auto-schedule a subgraph for CPU.

Any feedback on the API is welcomed.

cc @jcf94 @tqchen @junrushao1994 @comaniac @jwfromm

@merrymercy merrymercy changed the title [AutoScheduler] Tutorial on how to auto-schedule a subgraph for CPU [TUTORIAL] Using the template-free auto-scheduler on CPU Sep 16, 2020
jcf94
jcf94 reviewed Sep 16, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@jcf94 jcf94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine to me.

Comment thread python/tvm/auto_scheduler/auto_schedule.py
)


def create_task(func, args, target, target_host=None, hardware_params=None):
Copy link
Copy Markdown
Contributor

@jcf94 jcf94 Sep 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking about what's the different of calling SearchTask() directly, seems they just have different input parameters.

I notice that AutoTVM used an api of autotvm.task.create(), should we keep them the same?

Anyway, it's fine since we may continue to refine our APIs later.

Copy link
Copy Markdown
Member Author

@merrymercy merrymercy Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python does not support constructor overloading, so I have to create a new function for this.

I think both auto_scheduler.create_task and auto_scheduler.task.create are okay.

@merrymercy merrymercy changed the title [TUTORIAL] Using the template-free auto-scheduler on CPU [TUTORIAL][ANSOR] Using the template-free auto-scheduler on CPU Sep 16, 2020
jcf94
jcf94 approved these changes Sep 16, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@jcf94 jcf94 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems we can next migrate the tuning scripts here, and start to reproduce the op/subgraph performance in our paper based on the new master.

junrushao
junrushao approved these changes Sep 16, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@junrushao junrushao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tmoreau89
tmoreau89 reviewed Sep 16, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@tmoreau89 tmoreau89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! I've only left cosmetic comments.

Comment thread tutorials/auto_scheduler/tune_matmul_x86.py Outdated
Comment thread tutorials/auto_scheduler/tune_matmul_x86.py Outdated
Comment thread tutorials/auto_scheduler/tune_matmul_x86.py Outdated
Comment thread tutorials/auto_scheduler/tune_matmul_x86.py Outdated
Comment thread tutorials/auto_scheduler/tune_matmul_x86.py Outdated
comaniac
comaniac approved these changes Sep 16, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@comaniac comaniac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just nits.

Comment thread python/tvm/auto_scheduler/auto_schedule.py Outdated
Comment thread python/tvm/auto_scheduler/auto_schedule.py Outdated
Comment thread tutorials/auto_scheduler/tune_matmul_x86.py Outdated
merrymercy and others added 2 commits September 15, 2020 20:50
@merrymercy @comaniac
Apply suggestions from code review
6dd4fd8 
Co-authored-by: Cody Yu <comaniac0422@gmail.com>
@merrymercy
address comments
77725e7
tmoreau89
tmoreau89 approved these changes Sep 16, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@tmoreau89 tmoreau89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @merrymercy LGTM

masahi
masahi reviewed Sep 16, 2020
View reviewed changes
Comment thread tutorials/auto_scheduler/tune_matmul_x86.py Outdated
mbaret
mbaret approved these changes Sep 16, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@mbaret mbaret left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just had a quick play around with this tutorial and it looks good to me. There is one thing I wonder though, which is that even when I boost the number of trials it never seems to decide to inline the bias add (x86). Is this a limitation of Ansor or is inlining just not performant here?

@merrymercy
Copy link
Copy Markdown
Member Author

merrymercy commented Sep 16, 2020 •
edited
Loading

@mbaret Good catch! It is actually a small bug introduced during the upstream. Now the bug is fixed and you can give it another try.
One of the most important advantages of Ansor is just to try different fusion strategies for an arbitrary subgraph. So it should try both inline and no inline.

@mbaret
Copy link
Copy Markdown
Contributor

mbaret commented Sep 16, 2020

Ah nice, it's working as I expected now.

binarybana
binarybana reviewed Sep 16, 2020
View reviewed changes
Comment thread tutorials/auto_scheduler/tune_matmul_x86.py
# We can kick off the search and let the auto-scheduler do its magic.
# After some measurement trials, it will return the best schedule it found.

sch, args = auto_scheduler.auto_schedule(task, tuning_options=tune_option)
Copy link
Copy Markdown
Contributor

@binarybana binarybana Sep 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be nice to briefly describe what the return values of sch and args are here. I'm assuming that sch is a schedule, but is it encoded in TIR, or some other data structure? Same with args.

Copy link
Copy Markdown
Contributor

@jcf94 jcf94 Sep 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you check the doc string of the auto_scheduler.auto_schedule(), you'll find the explaination of the return values: A te.schedule and the a list of te.Tensor to be used in tvm.lower or tvm.build.

Copy link
Copy Markdown
Member Author

@merrymercy merrymercy Sep 17, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I merged this first and will see whether the docstring link on the tvm website works.
If it is not working, I can add more explanations in the next PR, which is a similar tutorial for GPU.

@merrymercy merrymercy merged commit 8843e76 into apache:master Sep 17, 2020
@merrymercy merrymercy deleted the ansor-cpu-tutorial branch September 17, 2020 02:15
kevinthesun pushed a commit to kevinthesun/tvm that referenced this pull request Sep 17, 2020
@merrymercy @comaniac @kevinthesun
[TUTORIAL][ANSOR] Using the template-free auto-scheduler on CPU (apac…
d5310f1 
…he#6488)

* add tutorial

* add tutorial

* update

* Apply suggestions from code review

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

* address comments

* fix bugs

* add the exmple for resuming the search

* fix lint

Co-authored-by: Cody Yu <comaniac0422@gmail.com>
kevinthesun pushed a commit to kevinthesun/tvm that referenced this pull request Sep 18, 2020
@merrymercy @comaniac @kevinthesun
[TUTORIAL][ANSOR] Using the template-free auto-scheduler on CPU (apac…
4483526 
…he#6488)

* add tutorial

* add tutorial

* update

* Apply suggestions from code review

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

* address comments

* fix bugs

* add the exmple for resuming the search

* fix lint

Co-authored-by: Cody Yu <comaniac0422@gmail.com>
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Sep 18, 2020
@merrymercy @comaniac @trevor-m
[TUTORIAL][ANSOR] Using the template-free auto-scheduler on CPU (apac…
1ffe511 
…he#6488)

* add tutorial

* add tutorial

* update

* Apply suggestions from code review

Co-authored-by: Cody Yu <comaniac0422@gmail.com>

* address comments

* fix bugs

* add the exmple for resuming the search

* fix lint

Co-authored-by: Cody Yu <comaniac0422@gmail.com>
@comaniac comaniac mentioned this pull request Sep 25, 2020
Closed
@ZihengJiang ZihengJiang mentioned this pull request Sep 25, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@masahi masahi masahi left review comments

@jcf94 jcf94 jcf94 approved these changes

@tqchen tqchen tqchen approved these changes

@tmoreau89 tmoreau89 tmoreau89 approved these changes

@comaniac comaniac comaniac approved these changes

@junrushao junrushao junrushao approved these changes

@mbaret mbaret mbaret approved these changes

+1 more reviewer

@binarybana binarybana binarybana left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@merrymercy @mbaret @binarybana @masahi @tqchen @tmoreau89 @comaniac @jcf94 @junrushao @ZihengJiang