ci: Enable autodiff tests on x86_64 linux

[sgasho]

View all comments

Enable autodiff tests on x86_64 linux

hmm.....pr job x stage 1 takes 100 - 110min, which is the slowest in the pr jobs.
If we are not happy to wait more than 100min for the pr jobs, maybe we should move this to auto jobs and test them on stage 2

r? @ZuseZ4

[sgasho]

I executed git pull --rebase origin main to see if build steps in autodiff job finish successfully after enzyme submodule update

No problem found

[rustbot]

Warning

If you are changing how CI LLVM is built or linked, make sure to bump
src/bootstrap/download-ci-llvm-stamp.

cc @jieyouxu

[ZuseZ4]

@bors try jobs=x86_64-gnu-autodiff

Rerunning to test potential cache effects.
Once this is done, we should ping Jakub and jieyouxu to discuss where to put it (and probably create an infra thread to discuss). I also don't think they'll accept autodiff as a culprit of measurably longer PR CI, so we'll probably need to move it, if the cache isn't helping enough.

[rust-bors]

☀️ Try build successful (CI)
Build commit: d1a9718 (d1a971826b83fead3b7550c5e4440c9102bb3290)
Base parent: 3c00c96 (3c00c96d3af4d5b5e101e56cc161a608b21366ee)

[ZuseZ4]

Rebuilding a second time, since I'm not sure if artifacts are shared between PR Ci and try runs.
@bors try jobs=x86_64-gnu-autodiff

But so far, even just building LLVM+rustc+std+enzyme takes 95 minutes, 10 minutes more than miri, the next slowest job. Total time will always be a bit more than that, and autodiff will always have less users than something general like Miri, so that's hard to defend.

If the try results come back unchanged, can you move it to auto and mark it as allowed to fail? I don't think it needs to be blocking yet.

[rust-bors]

☀️ Try build successful (CI)
Build commit: 96310de (96310dea88820c3264b54ee2b5011698005372aa)
Base parent: 36714a9 (36714a9983d6ba11203d8bb87a1b372247fbcf06)

[rust-bors]

☀️ Try build successful (CI)
Build commit: 54d3e38 (54d3e384f63af5d12a1b8677d914c8ed9ced0844)
Base parent: f10db29 (f10db292a3733b5c67c8da8c7661195ff4b05774)

[sgasho]

Just moved the job to the right place. No other changes.
I think this is ready to land, since the last try run passed.

[sgasho]

@bors try jobs=x86_64-gnu-autodiff

Rerunning to test potential cache effects. Once this is done, we should ping Jakub and jieyouxu to discuss where to put it (and probably create an infra thread to discuss). I also don't think they'll accept autodiff as a culprit of measurably longer PR CI, so we'll probably need to move it, if the cache isn't helping enough.

@Kobzol @jieyouxu cc: @ZuseZ4

Can we say this PR is ready to land - adds optional autodiff tests in x86 linux?
Looking at the last try run (https://github.com/rust-lang/rust/actions/runs/28874206231/job/85644955058), at least job time is acceptable?
I'll create an infra thread on Zulip if we need to.

[jieyouxu]

This looks fine as an optional-* job and CI time from last round is like 1h, seems fine.

I'll let Jakub take a look in case I'm missing any context, otherwise r=me.

View changes since this review

[jieyouxu]

@bors squash

[rust-bors]

🔨 3 commits were squashed into 630cb89.

[jieyouxu]

Actually I'll just r+ this, this is optional job anyway.
@bors r+ rollup

[rust-bors]

📌 Commit 630cb89 has been approved by jieyouxu

It is now in the queue for this repository.

[Kobzol]

Does it break often? I wonder if we should just enable it by default, because the optional jobs kinda suck now, we don't get any notification if they fail, outside of some unrelated CI postprocessing jobs sometimes failing. Probably that would require an MCP.

But starting with an optional job is fine by me. It would be nice to document it in rustc-dev-guid and add the doc-url to the job though.

[jieyouxu]

Ah d'oh I knew I forgot sth...
@bors r- (since not in a rollup yet)

[rust-bors]

This pull request was unapproved.

View changes since this unapproval

[sgasho]

Indeed, I don't think autodiff tests will break too often since #156777 landed.
Let me just check the autodiff job still works fine after pulling the current main. If it works fine, I think we can merge it without optional- unless I'm missing something. llvm submodule update happened last week so maybe good chance to test the robustness

cc: @ZuseZ4

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[sgasho]

@bors try jobs=x86_64-gnu-autodiff

[rust-bors]

☀️ Try build successful (CI)
Build commit: 7215d1b (7215d1be8eb0e150e6d50a7fe36403cec80dbf4f)
Base parent: cb014fc (cb014fcb149e44d8aa16a8f66c9f2b1762f034f8)

[sgasho]

@ZuseZ4
Would you be comfortable making this job required from the start by removing continue_on_error: true?
I reran autodiff tests with the latest main branch and they succeeded. That means llvm ir checks updates in #156777, which I created 3 weeks ago, are still fine against the latest main