Skip to content

OCPBUGS-77056: Revert #825 "Make external cert validation asynchronous (v2 — race condition fixes)" - #829

Merged
stbenjam merged 1 commit into
openshift:masterfrom
redhat-chai-bot:revert-pr-825
Jul 29, 2026
Merged

OCPBUGS-77056: Revert #825 "Make external cert validation asynchronous (v2 — race condition fixes)"#829
stbenjam merged 1 commit into
openshift:masterfrom
redhat-chai-bot:revert-pr-825

Conversation

@bentito

@bentito bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Reverts #825 ; tracked by OCPBUGS-77056

Per OpenShift policy, we are reverting this change to restore CI signal.

CC: @bentito

This reverts commit c73bada, reversing changes made to 682319a.

…sync-sar-resurrect-v2"

This reverts commit c73bada, reversing
changes made to 682319a.
@openshift-ci-robot openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 29, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@bentito: This pull request references Jira Issue OCPBUGS-77056, which is invalid:

  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Reverts #825 ; tracked by OCPBUGS-77056

Per OpenShift policy, we are reverting this change to restore CI signal.

CC: @bentito

This reverts commit c73bada, reversing changes made to 682319a.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

The pull request updates external-certificate secret handling and validation, removes SAR completion status recording, serializes route status admission, and changes conflict retry behavior. Writer leases now use a single worker loop and revised constructors. Debug image builds no longer force architecture settings. Certificate persistence, Kubernetes client configuration, endpoint-slice tests, fake HAProxy startup, and related tests are also updated.

Possibly related PRs

  • openshift/router#825: Overlaps in external-certificate handling, SAR completion behavior, secret-manager wiring, and writer-lease construction.

Suggested reviewers: redhat-chai-bot

🚥 Pre-merge checks | ✅ 13 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Structure And Quality ⚠️ Warning Secret-manager tests use nil stop channels and bare <-doneCh waits, so informer goroutines can leak and tests can hang; factory helper also uses os.Setenv with no cleanup. Use a real stop channel with defer close(stopCh), add bounded waits via context.WithTimeout/time.After, and restore env with t.Setenv or t.Cleanup.
✅ Passed checks (13 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately states this is a revert of PR #825 and names the affected change.
Description check ✅ Passed The description directly explains that this change reverts #825 to restore CI signal.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed No Ginkgo titles were added or modified; scans found no It/Describe/Context/When usages in the changed test files, and remaining test names are static Go test funcs.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the diff only changes unit tests and code under pkg/, with no It/Describe/Context/When blocks or MicroShift-unsupported APIs.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the changed test files are standard Go unit tests (Test*) with no SNO/HA assumptions.
Topology-Aware Scheduling Compatibility ✅ Passed No deployment manifests or scheduling fields (nodeSelector/affinity/spread/PDB/replica logic) were added; the patch only changes router/controller logic, tests, and a debug Makefile.
Ote Binary Stdout Contract ✅ Passed No added stdout writes in main/TestMain/init or suite setup; router_test.go change only updates writerlease.New args.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PASS: Changed tests are plain Go unit tests (no Ginkgo/It/Describe imports) and the diff adds no e2e files or external connectivity.
No-Weak-Crypto ✅ Passed Diff-wide scan found no added MD5/SHA1/DES/RC4/3DES/Blowfish/ECB usage, no crypto/subtle, and no secret/token comparisons in the PR patch.
Container-Privileges ✅ Passed The PR only changes Go code/tests and no touched file contains privileged/host* or allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs ✅ Passed No new logs expose secret contents or other sensitive values; changed logs only mention route/secret metadata or file paths.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bentito

bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 29, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@bentito: This pull request references Jira Issue OCPBUGS-77056, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @melvinjoseph86

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign thealisyed for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Nitpick comments (2)
pkg/router/template/configmanager/haproxy/testing/haproxy.go (1)

110-139: 🩺 Stability & Availability | 🔵 Trivial | ⚖️ Poor tradeoff

Provide a caller-owned context for listener startup.

Line 112 violates the configured noctx rule. Thread a context into the fake-server startup path and use net.ListenConfig.Listen; pair cancellation with listener closure if it must stop the accept loop. Verify all fakeHAProxy.Start callers before changing its API.

As per path instructions, “context.Context for cancellation and timeouts.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/router/template/configmanager/haproxy/testing/haproxy.go` around lines
110 - 139, Update the fakeHAProxy.Start startup path and all callers to accept
and propagate a caller-owned context, then use net.ListenConfig.Listen with that
context instead of net.Listen. Ensure cancellation closes the listener when
needed so the accept loop exits cleanly, while preserving the existing startup
error reporting and verifying every fakeHAProxy.Start call is updated
consistently.

Sources: Path instructions, Linters/SAST tools

pkg/router/controller/route_secret_manager_test.go (1)

1264-1291: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Informers never stop and doneCh waits have no timeout.

context.TODO().Done() is nil, so informer.Run never returns and each subtest leaks a reflector goroutine. The bare <-recorder.doneCh also turns any regression into a silent CI hang rather than a test failure.

♻️ Suggested test scaffolding
+			ctx, cancel := context.WithCancel(context.Background())
+			defer cancel()
 			informer := fakeSecretInformer(kubeClient, "sandbox", "tls-secret")
-			go informer.Run(context.TODO().Done())
+			go informer.Run(ctx.Done())
 
 			// wait for informer to start
-			if !cache.WaitForCacheSync(context.TODO().Done(), informer.HasSynced) {
+			if !cache.WaitForCacheSync(ctx.Done(), informer.HasSynced) {
 				t.Fatal("cache not synced yet")
 			}
@@
-			// wait until route's status is updated
-			<-recorder.doneCh
+			// wait until route's status is updated
+			select {
+			case <-recorder.doneCh:
+			case <-time.After(30 * time.Second):
+				t.Fatal("timed out waiting for route status to be recorded")
+			}

Same pattern applies in TestSecretDelete and TestSecretRecreation.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/router/controller/route_secret_manager_test.go` around lines 1264 - 1291,
Update the informer-based tests, including TestSecretDelete and
TestSecretRecreation, to use cancellable contexts instead of context.TODO(),
defer cancellation, and ensure each informer stops after the test. Replace bare
recorder.doneCh waits with bounded timeout-based waits that fail the test when
the route status is not updated promptly, including cleanup-safe handling for
informer shutdown.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/router/controller/factory/factory_endpointslices_test.go`:
- Around line 80-88: Update newEndpointSliceTestSetup to accept a *testing.T
parameter and replace os.Setenv with t.Setenv for scoped feature-gate
configuration; update both callers to pass their testing.T instance and remove
the process-wide environment mutation.

In `@pkg/router/controller/route_secret_manager.go`:
- Around line 311-315: Update the DeleteFunc callback to safely handle informer
tombstones by extracting the *kapi.Secret from either the direct object or
cache.DeletedFinalStateUnknown payload before accessing secret.Name. Avoid
unchecked type assertions and return without processing when the tombstone
payload is invalid, while preserving the existing key generation and deletion
log behavior for valid secrets.

In `@pkg/router/routeapihelpers/validation.go`:
- Around line 526-563: Update ValidateTLSExternalCertificate and its callers to
accept and propagate a context.Context, then pass that context to all three
authorizationutil.Authorize calls and the secretsGetter Get operation instead of
context.TODO(). Preserve the existing validation and error-handling behavior
while allowing route event processing to cancel or time out these API requests.
- Around line 520-532: The ValidateTLSExternalCertificate helper must guard its
TLS inputs before dereferencing tls.ExternalCertificate.Name. Add an early
validation return for nil TLS or nil ExternalCertificate, or explicitly document
and enforce the required caller precondition using the existing
hasExternalCertificate flow; preserve current validation behavior for valid
external certificates.

In `@pkg/router/template/certmanager.go`:
- Around line 181-187: Update simpleCertificateWriter.WriteCertificate to
preserve atomic replacement: write the certificate to a temporary file in the
same directory, close it and apply the required permissions, then commit it with
os.Rename to the final .pem path. Ensure failures clean up the temporary file
and leave the existing certificate untouched.

In `@pkg/router/writerlease/writerlease.go`:
- Around line 253-255: Update the follower lease handling around the writer
lease queue logic to replace the blocking time.Sleep(remaining) with cancellable
delayed queueing. Use the existing stopCh or context cancellation mechanism so
shutdown interrupts the delay promptly, and ensure the worker is released while
waiting before re-adding key to l.queue.
- Around line 142-151: Update WriterLease.Run to wait for its worker goroutine
to exit after the queue is shut down, using synchronization around the single
goroutine started there. Preserve the existing crash handling and worker
cleanup, and return only after the worker has completed.

---

Nitpick comments:
In `@pkg/router/controller/route_secret_manager_test.go`:
- Around line 1264-1291: Update the informer-based tests, including
TestSecretDelete and TestSecretRecreation, to use cancellable contexts instead
of context.TODO(), defer cancellation, and ensure each informer stops after the
test. Replace bare recorder.doneCh waits with bounded timeout-based waits that
fail the test when the route status is not updated promptly, including
cleanup-safe handling for informer shutdown.

In `@pkg/router/template/configmanager/haproxy/testing/haproxy.go`:
- Around line 110-139: Update the fakeHAProxy.Start startup path and all callers
to accept and propagate a caller-owned context, then use net.ListenConfig.Listen
with that context instead of net.Listen. Ensure cancellation closes the listener
when needed so the accept loop exits cleanly, while preserving the existing
startup error reporting and verifying every fakeHAProxy.Start call is updated
consistently.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 0d16cea7-c3ff-4afc-b414-b2c8c70a460d

📥 Commits

Reviewing files that changed from the base of the PR and between c73bada and 5c27a38.

⛔ Files ignored due to path filters (3)
  • vendor/github.com/openshift/library-go/pkg/authorization/authorizationutil/subject.go is excluded by !**/vendor/**, !vendor/**
  • vendor/github.com/openshift/library-go/pkg/authorization/authorizationutil/util.go is excluded by !**/vendor/**, !vendor/**
  • vendor/modules.txt is excluded by !**/vendor/**, !vendor/**
📒 Files selected for processing (19)
  • hack/Makefile.debug
  • pkg/cmd/infra/router/clientcmd.go
  • pkg/cmd/infra/router/template.go
  • pkg/router/controller/contention.go
  • pkg/router/controller/factory/factory_endpointslices_test.go
  • pkg/router/controller/route_secret_manager.go
  • pkg/router/controller/route_secret_manager_test.go
  • pkg/router/controller/shared_secret_manager.go
  • pkg/router/controller/shared_secret_manager_test.go
  • pkg/router/controller/status.go
  • pkg/router/controller/status_test.go
  • pkg/router/routeapihelpers/validation.go
  • pkg/router/routeapihelpers/validation_test.go
  • pkg/router/router_test.go
  • pkg/router/template/certmanager.go
  • pkg/router/template/certmanager_test.go
  • pkg/router/template/configmanager/haproxy/testing/haproxy.go
  • pkg/router/writerlease/writerlease.go
  • pkg/router/writerlease/writerlease_test.go
💤 Files with no reviewable changes (5)
  • pkg/router/controller/shared_secret_manager_test.go
  • pkg/cmd/infra/router/clientcmd.go
  • pkg/router/controller/shared_secret_manager.go
  • pkg/router/template/certmanager_test.go
  • pkg/router/controller/status_test.go

Comment thread pkg/router/controller/factory/factory_endpointslices_test.go
Comment thread pkg/router/controller/route_secret_manager.go
Comment thread pkg/router/routeapihelpers/validation.go
Comment thread pkg/router/routeapihelpers/validation.go
Comment thread pkg/router/template/certmanager.go
Comment thread pkg/router/writerlease/writerlease.go
Comment thread pkg/router/writerlease/writerlease.go
@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Jul 29, 2026
@bentito

bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/payload-aggregate periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance 10

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@bentito: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/d58bfa20-8b6e-11f1-9542-03214933e98d-0

@bentito

bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 29, 2026
@bentito

bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/hold cancel

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 29, 2026
@bentito

bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/payload-aggregate periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade 10

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@bentito: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-ci-5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-rt-upgrade

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b3134aa0-8b70-11f1-9977-4d41ff45fa85-0

@bentito

bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/payload-job periodic-ci-openshift-release-main-nightly-5.0-e2e-metal-ipi-ovn-ipv4 periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview

@openshift-ci

openshift-ci Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

@bentito: trigger 2 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-main-nightly-5.0-e2e-metal-ipi-ovn-ipv4
  • periodic-ci-openshift-release-main-ci-5.0-e2e-aws-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/be81ed60-8b70-11f1-825c-80b1bfcbccc7-0

@bentito

bentito commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@stbenjam

Copy link
Copy Markdown
Member

/hold cancel
/lgtm
/label approved
/verified by CI

We are fairly certain this is why 5.0.0-0.nightly-2026-07-29-133253 was rejected

https://htmlpreview.github.io/?https://gist.githubusercontent.com/stbenjam/2aaaef0417349169c79cf0fa62f81887/raw/5df915bba36e2e1017a1a572409a588b10ae88ca/payload-analysis-5.0.0-0.nightly-2026-07-29-133253-summary.html

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 29, 2026
@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jul 29, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@stbenjam: This PR has been marked as verified by CI.

Details

In response to this:

/hold cancel
/lgtm
/label approved
/verified by CI

We are fairly certain this is why 5.0.0-0.nightly-2026-07-29-133253 was rejected

https://htmlpreview.github.io/?https://gist.githubusercontent.com/stbenjam/2aaaef0417349169c79cf0fa62f81887/raw/5df915bba36e2e1017a1a572409a588b10ae88ca/payload-analysis-5.0.0-0.nightly-2026-07-29-133253-summary.html

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 29, 2026
@stbenjam
stbenjam merged commit d6ddfd8 into openshift:master Jul 29, 2026
11 of 13 checks passed
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@bentito: Jira Issue OCPBUGS-77056: Some pull requests linked via external trackers have merged:

The following pull request, linked via external tracker, has not merged:

All associated pull requests must be merged or unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with /jira refresh.

Jira Issue OCPBUGS-77056 has not been moved to the MODIFIED state.

This PR is marked as verified. If the remaining PRs listed above are marked as verified before merging, the issue will automatically be moved to VERIFIED after all of the changes from the PRs are available in an accepted nightly payload.

Details

In response to this:

Reverts #825 ; tracked by OCPBUGS-77056

Per OpenShift policy, we are reverting this change to restore CI signal.

CC: @bentito

This reverts commit c73bada, reversing changes made to 682319a.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants