[FQ2I] Support converting dense -> add to qnn.dense -> add -> requantize

[masahi]

Closes #13545

The pattern of dense -> add, where the add is really bias addition, can appear often as the result of converting ONNX Gemm op:

tvm/python/tvm/relay/frontend/onnx.py

Line 1409 in edfeba5

out = out + _expr.const(beta, dtype=dtype) * inputs[2]

Currently, FQ2I tries to convert this add to qnn.add. But if this add is being used for bias addition, out_t.scale and out_t.zero_point variables in fake_quantization_to_integer.py, which are used to initialize the output scale and zp of the QNN binary operators, can be tensors rather than scalars. QNN binary operators do not support such output qparams, which led to the error reported in #13545.

For this reason, apparently we haven't supported converting dense -> add to qnn.dense -> add -> requantize, when add is a bias add, in FQ2I. The pattern of dense -> nn.bias_add can be converted to qnn.dense -> nn.bias_add -> requantize, but we never use nn.bias_add after dense.

So I added a code path in the FQ2I QNN binary op converter, to identify such patterns and use regular binary ops rather than QNN ones.

cc @AndrewZhaoLuo @Icemist @elvin-n

[tvm-bot]

Thanks for contributing to TVM! Please refer to the contributing guidelines https://tvm.apache.org/docs/contribute/ for useful information and tips. Please request code reviews from Reviewers by @-ing them in a comment.

• No users to tag found in teams: fq2i _{See #10317 for details}

_{Generated by tvm-bot}

[Icemist]

LGTM, a little code remark.

[AndrewZhaoLuo]

I would keep the original line of .get('beta', 1.0) since you cannot call float() on None which attr.get can return.

Then below on L1409, you can just do if beta is None --> if 'beta' not in attr.keys() or something.

[AndrewZhaoLuo]

Though the change on L1409 might not be needed since if beta == 1, it can be removed with constant folding.

[masahi]

Constant folding doesn't work when beta is multiplying an output of qnn ops, since we cannot fold over them. The model in #13545 has multiply(1f, dequantize(bias) after dense, which was also causing some issues.

[masahi]

Moved float(beta) to the else block of if beta is None.

[masahi]

Actually the whole purpose of this change was to avoid multiplying by 1.0, since multiply(1f, dequantize(bias) would be converted to qnn.mul(quantize(1), bias) by FQ2I. So I restored the original code cc @Icemist

An alternative would be to add algebraic simplification to the SimpliyfyExpr pass.

[AndrewZhaoLuo]

LGTM! Thanks, though i will wait on @Icemist