Attempt to fix bottlenecks and performance related to Facade#399
Conversation
| f"""UPDATE collection_status SET ml_status = '{CollectionState.PENDING.value}'""" | ||
| f""" WHERE ml_status = '{CollectionState.ERROR.value}' and ml_data_last_collected is NULL;""" | ||
|
|
||
| with DatabaseSession(logger, engine) as session: |
There was a problem hiding this comment.
yes i know this part of the code needs a rewrite
| @@ -2,6 +2,7 @@ | |||
| import logging | |||
| import random | |||
There was a problem hiding this comment.
[pylint] reported by reviewdog 🐶
W0611: Unused import random (unused-import)
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
5ffec47 to
4fe004e
Compare
This adds a limit for the repo queries that is so high, it should effectively be unlimited (we are trying to count newly added repos in each enabled phase). Signed-off-by: Adrian Edwards <adredwar@redhat.com>
…t is also gated based on whether its a docker build. Signed-off-by: Adrian Edwards <adredwar@redhat.com>
|
been testing this for a few days with the other high priority fixes. Haven't observed the errored repos get scheduled for recollection, but also havent observed the retry task get scheduled so this could just be me on a laptop (where the laptop being asleep overnight causes the task to never run) |
|
observed the task run. it seemed to reset errored repos on my test instance (the collection monitor run after the reset picked up 2 core repos), but no repos have been added to this instance for a while so i think thats ok. the new code has been exercised and the un-run branch is just a log statement. |
Description
This PR attempts to fix a couple non-normal conditions with regard to facade.
These two issues can combine to grind collection to a halt and waste significant resources.
This PR addresses these issues by:
when the errored repos (possibly a lot of them) get their status reset all at once, it causes a flood of collection tasks that are likely to fail again, thus potentially eating up API key usage when it might be needed for large tasks like new collection, contributing to slowness or bottlenecks.
This PR fixes #391
Notes for Reviewers
I still have yet to test this fully. itll probably be complicated to test
Signed commits
Generative AI disclosure
Please select one option:
If AI tools were used, please provide details below:
- What tools were used? Sonnet 4.6 Medium via Cursor
- How were these tools used? lots of back and forth to attempt to diagnose the general collection performance issues. Some AI code completion was used to write the code
- Did you review these outputs before submitting this PR? Yes, at all times I was making sure the AI tool's diagnosis was sensible given my knowledge of the code and all written code was looked at and understood by me.