Add a check for methods that hardcode their return type, but shouldn't

[AlexWaygood]

An attempt at implementing #168 (closes #168).

Marking this as a draft for now, as I don't think the error message is very good, and I'm not sure how to improve it. I would also be curious to hear what others think of the overall idea before I put any more work in, as well.

[AlexWaygood]

The number of typeshed hits is pretty small, because I've been using an early prototype of this check to fix typeshed problems prior to filing this PR:

• Improve itertools stubs python/typeshed#7109
• stdlib: Improve many __iter__ and constructor methods python/typeshed#7112
• Misc 3rd-party stubs: improve __enter__ & constructor methods python/typeshed#7114
• Improve classmethods in paramiko stubs python/typeshed#7113
• Use _typeshed.Self in docutils.VersionInfo and os.sched_param python/typeshed#7117

[AlexWaygood]

I think this is more-or-less there now.

[JelleZijlstra]

Thanks for putting this together, I'll take a look soon.

Are the remaining typeshed hits false positives?

[AlexWaygood]

Thanks for putting this together, I'll take a look soon.

Are the remaining typeshed hits false positives?

There are several typing hits which are false positives; I think we should just disable the check in typing, as it's a special case (we don't want Iterator.__iter__ to return _typeshed.Self, as even though it's by far the most common implementation at runtime for iterators just to return self, it's not mandated for a class to do that in order to satisfy the Iterator interface).

There are two hits in builtins: zip and reversed. I think the reversed hit in builtins is a false positive. I need to check with zip.

The Python 2 hits just need a backport of the changes I've already made in the Python 3 stdlib, if we decide to go ahead with this.

[JelleZijlstra]

Maybe we can disable the check in classes that are Protocols? That seems like it would fix the Iterator case, and it's also generally good for Protocols to be less precise about the return type.

[JelleZijlstra]

zip.__iter__ and reversed.__iter__ do return self, I'll submit a PR.

[AlexWaygood]

zip.__iter__ and reversed.__iter__ do return self, I'll submit a PR.

Looks to me like zip.__iter__ returns self but reversed.__iter__ doesn't?

>>> class Foo(reversed): ...
...
>>> type(iter(Foo([1, 2, 3])))
<class 'list_reverseiterator'>
>>> class Foo(zip): ...
...
>>> type(iter(Foo([1, 2], [3, 4])))
<class '__main__.Foo'>

[AlexWaygood]

Maybe we can disable the check in classes that are Protocols? That seems like it would fix the Iterator case, and it's also generally good for Protocols to be less precise about the return type.

I was wrong: the three false positives in typing are BinaryIO.__enter__, TextIO.__enter__, and IO.__iter__. None of these classes are protocols, but, all of these methods are abstract methods. So we can just disable the check for abstract methods, which makes sense in exactly the same way as your suggestion re. protocols!

[AlexWaygood]

Nice, excluding abstractmethods got rid of the typing hits!

[Akuli]

We can merge once the typeshed check is green.

[AlexWaygood]

typeshed backport PR here: python/typeshed#7128

[AlexWaygood]

(python/typeshed#7123 will also be required in order to make the CI go green.)

[AlexWaygood]

Hooray!