bpo-36974: inherit tp_vectorcall_offset unconditionally

[jdemeyer]

This fixes one more corner case in vectorcall inheritance. Bugfix, so needs backport to 3.8.

CC @encukou

https://bugs.python.org/issue36974

[vstinner]

Does this change support the stable ABI? Is it still possible to use a C extension compiled with Python 3.6 on Python 3.9 with this change? The C extension compiled with Python 3.6 can use static types with Python 3.6 PyTypeObject which doesn't have tp_vectorcall field.

My notes on these topics:

• https://pythoncapi.readthedocs.io/type_object.html
• https://pythoncapi.readthedocs.io/stable_abi.html

I decided to give up on my tp_fastcall slot experiment because of these problems.

[jdemeyer]

Isn't the whole "stable ABI for static types" issue gone since #4944?

Regardless of that, tp_vectorcall_offset replaces tp_print, a slot which has always existed.

[jdemeyer]

See also https://www.python.org/dev/peps/pep-0580/#using-tp-print (this is in PEP 580 which was rejected but the same applies to PEP 590).

[jdemeyer]

which doesn't have tp_vectorcall field.

At the moment, tp_vectorcall is not used anywhere. It is meant for implementing vectorcalls of type objects (i.e. for type.__call__) but that hasn't been implemented. I know that @markshannon had some prototypes but there is no PR yet.

[vstinner]

Isn't the whole "stable ABI for static types" issue gone since #4944?

Honestly... I have no idea :-( All I know is that PyQt rely on the stable ABI.

At the moment, tp_vectorcall is not used anywhere. It is meant for implementing vectorcalls of type objects (i.e. for type.call) but that hasn't been implemented.

For example, if Python 3.6 PyTypeObject is 100 bytes long, but tp_vectorcall offset is 108: setting tp_vectorcall field of a Python 3.6 static type would lead to a buffer overflow causing undefined behavior.

I don't know if it's the case. I'm just raising the question.

cc @pitrou

[jdemeyer]

For example, if Python 3.6 PyTypeObject is 100 bytes long, but tp_vectorcall offset is 108: setting tp_vectorcall field of a Python 3.6 static type would lead to a buffer overflow causing undefined behavior.

The thing that matters is tp_vectorcall_offset. This replaces tp_print, so there is no stable ABI issue here. And tp_vectorcall_offset is an offset in the object structure, analogous to tp_dictoffset or tp_weaklistoffset. The layout of PyTypeObject doesn't matter for that.

Your concerns might be valid when we implement vectorcall for the type object itself (i.e. for type.__call__)

[jdemeyer]

Can I please ask again for a review? This is the only open bugfix for PEP 590 (there are many other open tickets regarding PEP 590, but those are enhancements, not bugfixes).

[markshannon]

LGTM.

@vstinner I'd like to merge this if you have no further objections.

[encukou]

I think the questions were answered. Victor, if you have any more, let me know.

[jdemeyer]

Thanks and don't forget to backport to 3.8

[vstinner]

@vstinner I'd like to merge this if you have no further objections.

I only had questions about the ABI compatibility, but I think that it was discussed in length here and in https://bugs.python.org/issue37250 In short, we don't provide any backward compatibility warranty at the ABI level for static types. If you want a stable ABI, avoid static types and use PyType_FromSpec() instead.

[miss-islington]

Thanks @jdemeyer for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

[miss-islington]

I'm having trouble backporting to 3.8. Reason: 'Error 110 while writing to socket. Connection timed out.'. Please retry by removing and re-adding the needs backport to 3.8 label.

[miss-islington]

Thanks @jdemeyer for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

[bedevere-bot]

GH-14342 is a backport of this pull request to the 3.8 branch.