[ONNX] [Test] fix GRU modification and reduce tolerance for RNN tests

[vvchernov]

During unification of GRU layer for frontends (#8781) the critical bag in GRU implementation on ONNX frontend side was observed. Inspite of unit test of GRU the bag was hidden by high tolerance (~1e-2) set for RNN tests. Moreover after bug fixing GRU unit test still requires high tolerance for correct pass. This fact was strange because local tests show very good coincidence with golden output from pytorch for any GRU modification (~1e-7).

1. I checked LSTM tests: tolerance 1e-6 is enough in most cases but sometimes it needs 1e-5 (it is tolerance for other operations tested for onnx frontend).
2. Bug in GRU was localized and fixed. it was related to implementation of specific GRU modification.
3. tolerance for RNN tests were reduced from 1e-2 to 1e-5
4. Random seed was added for GRU test. The test was passed 1000 times to avoid flaky test

[AndrewZhaoLuo]

Hey all, how did you check the required tolerance? Can you run it 1000 times to ensure these tolerances will pass if you have not already?

[vvchernov]

It is a good question. Unfortunately there is only one test for operation (or its modification) to reduce testing time. Moreover numpy random is used with repeatable randomization. It means that it is not checked fully all cases for operation from which we could define tolerance thresholds. But it checks that operation works for one but orbitrary input for reasonable tolerance. For numerical calculations the tolerance of order of 1e-6 - 1e-5 seems reasonable. In my case it was high enough to hide problem in operation.

[AndrewZhaoLuo]

I just want to make sure the tests are not super flaky.

You can do this by running pytest tests/python/frontend/onnx/test_forward.py::test_gru --repeat 1000 after pip installing pytest-repeat.

Lower than 1 / 1000 failures will seem sufficient.

[vvchernov]

@AndrewZhaoLuo I have added seed for test reproduction and checked it 1000 times.

[AndrewZhaoLuo]

LGTM, though you'll need one of the code owners on the right to merge. @mbrookhart