You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The results from the MonoT5 implementation in this repo don’t match the performance reported in benchmark datasets or the original paper. We need to investigate and verify the implementation details.
Tasks:
Review the code and model loading (especially MonoT5 checkpoint usage)
Verify how inputs are tokenized and formatted
Check if scoring and ranking logic matches what the paper describes
Reproduce one or more benchmark runs to validate performance
Identify any mismatch or deviation from the expected setup