[TensorRT] Add transpose_a/b for TensorRT batch_matmul

[ymwangg]

This PR added transpose_a/b for TensorRT batch_matmul, fixed a warning and compilation error with TensorRT-8. It also removed the redundant transpose op in onnx matmul. Tested with both TensorRT-7 and TensorRT-8.
cc @trevor-m @comaniac

[trevor-m]

Thanks @ymwangg! Left some minor comments in review.

[trevor-m]

I don't think you need these relay.transpose on the inputs to test functionality of transa/transb args.

[ymwangg]

Good point, I've changed to using x/y_shape instead.

[comaniac]

LGTM. Leave to @trevor-m to merge after addressing the comments.

[jcf94]

Thanks! @ymwangg

Just a little concern about changing the default behavior of framework frontend, since currently the default topi schedule support for NN format is not as strong as the original NT one.
This may cause confusions to those who have used onnx frontend before or who is using onnx frontend now.

To give an example, I've added an extra config to TensorFlow frontend which uses the NT format by default but provides an option to use the normal format. I think that would be better before we have prepared a strong enough topi.

p.s.: You see, I've also kept the default layout for nn.batch_matmul to be the original NT.

tvm/python/tvm/relay/frontend/tensorflow_ops.py

Lines 1191 to 1199 in 7653972

	if TF_DEFAULT_CONFIGS["use_nt_batch_matmul"]:
	# Strictly convert all batch_matmul to NT format
	input_x = _op.transpose(input_x, axes=[0, 2, 1]) if adj_x else input_x
	input_y = _op.transpose(input_y, axes=[0, 2, 1]) if not adj_y else input_y
	ret = get_relay_op("batch_matmul")(input_x, input_y)
	else:
	ret = get_relay_op("batch_matmul")(
	input_x, input_y, transpose_a=adj_x, transpose_b=adj_y
	)

[ymwangg]

@jcf94 Thanks for the pointer. I will refactor to make NN optional.

[ymwangg]

@trevor-m @jcf94 Please review again when you get a chance.

[jcf94]

LGTM. Thanks! @ymwangg