Disable differing_test_runners health check#108886
Conversation
|
Should we maybe also pin the version of hypothesis we're installing in CI here? https://github.com/python/cpython/blob/main/.github/workflows/build.yml#L434 It seems fairly likely that Hypothesis could add other health checks in the future that would cause our CI to randomly start failing. And it's just generally good practice to pin test dependencies as well, so that we can easily reproduce errors that are happening in CI. |
|
@dependabot and @sobolevn will bump hypothesis from time to time 👍 |
Tools/requirements-tests.txt
Outdated
There was a problem hiding this comment.
Maybe requirements-hypothesis.txt? Tools/requirements-tests.txt feels too similar to the existing Tools/requirements-dev.txt file we already have (and I think it's unlikely we'd use the same file for test dependencies that aren't related to the Hypothesis CI job).
I'm not even sure the file belongs in the Tools/ directory... maybe it should just be a top-level file?
There was a problem hiding this comment.
Dependabot already knows about this dir and Hypothesis is clearly a tool :)
So, let's leave it there.
There was a problem hiding this comment.
We could move them all into some other directory later, maybe .github/dependabot/, but let's get the CI back to green first.
There was a problem hiding this comment.
yeah, Tools/ still doesn't feel ideal to me, but I don't feel strongly about where they should go 👍
AlexWaygood
left a comment
There was a problem hiding this comment.
LGTM, thanks! Will wait a little bit in case Hugo/Zac/Ezio want to chime in, though :)
|
I'm a big fan of pinning your transitive dependencies too with e.g. the We most recently added a new health check in January 2021, but I'd pin anyway. For example, a while ago Astropy's CI started failing because a new Hypothesis release was better at finding numerical bugs, and while you do want to find out about them (updates) you don't want them to break everyone's regression tests (pins). See also this blog post. |
|
Two transitive deps that are installed in CI are: Thanks for your help 👍 |
|
GH-108887 is a backport of this pull request to the 3.12 branch. |
(cherry picked from commit 6ead5bd) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Added in HypothesisWorks/hypothesis@cde4067
Refs #108879
CC @Zac-HD