gh-109653: Speedup import of threading module

[danielhollas]

Delayed import of functools speeds up the import threading by ~50% (2ms -> 1ms) in my testing.

Since the functools module is only used in the internal _register_atexit function that is called by concurrent.futures, this seems like a worthwhile win for users of threading module who do not use asyncio.

Part of #109653

CC @AlexWaygood

• Issue: Improve import time of various stdlib modules #109653

[bedevere-app]

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

[danielhollas]

To be precise, compiling python with ./configure --enable-optimizations and measuring with python -Ximporttime -c "import threading", I am getting 2.44ms on main and 1.17ms on this PR.

[ajoino]

Just a thought, is it even necessary to use functools.partial here? Could this not be replaced with lambda: f(*args, **kwargs), that would avoid importing functools at all. Is there something I'm missing here?

[AlexWaygood]

Just a thought, is it even necessary to use functools.partial here? Could this not be replaced with lambda: f(*args, **kwargs), that would avoid importing functools at all. Is there something I'm missing here?

This was my thought as well on first seeing the patch. functools.partial can be faster than a lambda function, but here I doubt it makes a significant difference. (If we wanted to check whether using a lambda here slowed things down, we'd need to do a benchmark using concurrent.futures, since the concurrent.futures module is the only public API that makes use of this private API. It might be possible to write such a benchmark, but it also might be difficult -- not sure.)

[danielhollas]

functools.partial can be faster than a lambda function, but here I doubt it makes a significant difference. (If we wanted to check whether using a lambda here slowed things down, we'd need to do a benchmark using concurrent.futures, since the concurrent.futures module is the only public API that makes use of this private API. It might be possible to write such a benchmark, but it also might be difficult -- not sure.)

Looking at the code, the threading.register_atexit() is only ever called during concurrent.futures import, so I would assume any performance difference here would be marginal?

[AlexWaygood]

Looking at the code, the threading.register_atexit() is only ever called during concurrent.futures import, so I would assume any performance difference here would be marginal?

Oh, great point 😄

In that case, let's just go with a lambda here -- it seems simpler :)

[AlexWaygood]

LGTM, thanks! I'd love to check with a core dev more familiar with subinterpreters before merging, though (since this feature was specifically added to help with subinterpreter support).

@ericsnowcurrently, there's no reason why switching to a lambda rather than functools.partial could be problematic for subinterpreter support, is there?

[danielhollas]

@AlexWaygood thanks!

@ericsnowcurrently, there's no reason why switching to a lambda rather than functools.partial could be problematic for subinterpreter support, is there?

Just a note, if this was a problem, we could still get away with it by simply not doing either: the function is (at least currently) being called without any extra *args or **args arguments so we could make _register_atexit less general and simply pass the callback function directly to _threading_atexits list.

[AlexWaygood]

I can't see a way in which this would cause problems — I'll go ahead and merge, since it's been a few days :)

Thanks @danielhollas!

[ericsnowcurrently]

@ericsnowcurrently, there's no reason why switching to a lambda rather than functools.partial could be problematic for subinterpreter support, is there?

I'm not aware of any such reason.