[PERF] Performance Regression on Mali GPU

[merrymercy]

#2615 introduced a performance regression to Mali GPU backend. Because some loops cannot be vectorized, the run time of resnet-18 increases to 750ms from original 130ms. Rolling back to its previous commit can fix the problem and reproduce our released benchmark.

This bug was reported by users in the forum.
https://discuss.tvm.ai/t/bad-performance-after-using-tvm-on-nanopc-t4/2006/3
https://discuss.tvm.ai/t/unable-to-reproduce-benchmark-results-on-rk3399-mali-t860/2340/7

@yzhliu has a temporary fix. Please follow up.

cc @icemelon9

[tqchen]

Please confirm the difference of low level ir in the generated code. While it is fine eliminating the floating eager folding, it would be great to know what is the exact cause of the problem. And would reoptimize solve the case

[yzhliu]

will be looking into the details.

[nzlionhevrt]

Hello! I can confirm this, I'm using TVM on NanoPC-T4, Firefly RK3399, Rock Pi 4, all on linux 4.4 kernel, with OpenCL 1.2 support. I can't reproduce benchmark results using version from master branch, but it's okay with older versions.

[yzhliu]

It's very likely due to https://github.com/dmlc/tvm/blob/master/topi/python/topi/mali/conv2d.py#L357
which be eliminated by eager simplification https://github.com/dmlc/tvm/blob/master/src/arithmetic/const_fold.h#L144

I don't quite understand the scenario, looks like removing such produces (if/likely-)condition in the vectorize @merrymercy

[tqchen]

try to change that to tvm.expr.Mul(a, b)

[merrymercy]

After I applied @tqchen 's suggestion, the vectorization error disappeared.

#3131
But I do not have the access to rk3399 right now. Can you verify it @yzhliu ?

[yzhliu]

@merrymercy I tested on the device, it works.

[merrymercy]

closed by #3131