Skip to content

gh-3108: Avoid materializing f_locals for KI protection - #3110

Merged
graingert merged 39 commits into
python-trio:mainfrom
graingert:ki-with-code-objects
Oct 27, 2024
Merged

gh-3108: Avoid materializing f_locals for KI protection#3110
graingert merged 39 commits into
python-trio:mainfrom
graingert:ki-with-code-objects

Conversation

@graingert

@graingert graingert commented Oct 13, 2024

Copy link
Copy Markdown
Member

Fixes #3108
Fixes #2670

In this pull request, we avoid materializing f_locals by using weak references to code objects instead.

@codecov

codecov Bot commented Oct 13, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.62377%. Comparing base (13d4bad) to head (5d76de2).
Report is 250 commits behind head on main.

Additional details and impacted files
@@                 Coverage Diff                 @@
##                main       #3110         +/-   ##
===================================================
+ Coverage   99.58714%   99.62377%   +0.03662%     
===================================================
  Files            121         122          +1     
  Lines          18166       18340        +174     
  Branches        3268        3281         +13     
===================================================
+ Hits           18091       18271        +180     
+ Misses            52          47          -5     
+ Partials          23          22          -1     
Files with missing lines Coverage Δ
src/trio/_core/_ki.py 100.00000% <100.00000%> (ø)
src/trio/_core/_run.py 99.02344% <100.00000%> (-0.00381%) ⬇️
src/trio/_core/_run_context.py 100.00000% <100.00000%> (ø)
src/trio/_core/_tests/test_ki.py 99.77169% <100.00000%> (+1.93218%) ⬆️
src/trio/_core/_tests/test_run.py 100.00000% <100.00000%> (ø)
src/trio/_tools/gen_exports.py 95.90164% <100.00000%> (ø)

@graingert
graingert force-pushed the ki-with-code-objects branch from e04b3d8 to 5d51c56 Compare October 13, 2024 18:09
@CoolCat467 CoolCat467 changed the title gh-3108: avoid materializing f_locals by using weakrefs to code objec… gh-3108: Avoid directly referencing f_locals Oct 13, 2024
Comment thread src/trio/_core/_run.py
@graingert
graingert marked this pull request as ready for review October 13, 2024 20:48
Comment thread src/trio/_core/_ki.py
@graingert graingert changed the title gh-3108: Avoid directly referencing f_locals gh-3108: Avoid materializing f_locals for KI protection Oct 13, 2024
@graingert
graingert marked this pull request as draft October 14, 2024 06:30
@graingert
graingert force-pushed the ki-with-code-objects branch from d44f023 to ec30d7b Compare October 14, 2024 07:57
Comment thread src/trio/_core/_ki.py Outdated
@graingert
graingert marked this pull request as ready for review October 14, 2024 15:30

@A5rocks A5rocks left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plenty of nitpicks or me being confused; please don't assume I know what I'm doing with these comments. If the question doesn't make sense, it's cause I don't understand and not some weird encoded question.

Comment thread src/trio/_core/_ki.py
Comment thread src/trio/_core/_ki.py Outdated
Comment thread src/trio/_core/_ki.py
Comment thread src/trio/_core/_ki.py Outdated
Comment thread newsfragments/3108.bugfix.rst Outdated
@graingert

Copy link
Copy Markdown
Member Author

It looks like this will fix #1752

and maybe in conjunction with #3112 it will fix this #2454

@graingert
graingert requested a review from A5rocks October 23, 2024 09:49

@A5rocks A5rocks left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch of really minor comments. This could merge as is without any changes IMO.

Comment thread newsfragments/2670.bugfix.rst
Comment thread newsfragments/3108.bugfix.rst
Comment thread src/trio/_core/_ki.py
Comment thread src/trio/_core/_ki.py
Comment thread src/trio/_core/_ki.py
Comment thread src/trio/_core/_run.py Outdated
Comment thread src/trio/_core/_tests/test_ki.py Outdated
Comment thread src/trio/_core/_tests/test_ki.py
@A5rocks

A5rocks commented Oct 24, 2024

Copy link
Copy Markdown
Contributor

Nevermind, I was thinking about whether a deprecation is possible but it isn't because the problem is functions having ki protection only sometimes.

Comment thread src/trio/_core/_tests/test_ki.py Outdated
Comment thread newsfragments/3108.bugfix.rst Outdated
@graingert

Copy link
Copy Markdown
Member Author

pre-commit.ci autofix

@A5rocks A5rocks left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be explicit, since all my comments were fixed and all my questions answered.

@CoolCat467 CoolCat467 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell, this looks pretty great

@graingert
graingert enabled auto-merge (squash) October 27, 2024 08:52
@graingert
graingert merged commit 57452ad into python-trio:main Oct 27, 2024
@graingert
graingert deleted the ki-with-code-objects branch October 27, 2024 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KI protection f_locals materialization results in reference cycles on 3.12 and below @enable_ki_protection breaks inspect.iscoroutinefunction

5 participants