feat(resourcemanager): Unify organization model and onboarding gate#675
Merged
Conversation
mattdjenkinson
requested review from
JoseSzycho,
kevwilliams,
savme and
scotwells
5 tasks
Contributor
|
The tight coupling this introduces between the billing service and milo's core system seems rigid. Wonder if we can consider a more dynamic model where services can register onboarding gates with an organization that they're responsible for providing status for? |
Contributor
Author
|
@scotwells we could do something like introduce an |
Contributor
|
🤖 I automatically added missing newlines at the end of 21 file(s) in this PR. All files should now end with a newline character as per coding standards. |
scotwells
previously approved these changes
Jun 30, 2026
Collapse Personal/Standard into a single org with contactInfo and an OnboardingComplete status condition reconciled from billing readiness.
🤖 Automatically added newlines to 6 file(s) Co-Authored-By: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
The onboarding controller imports go.miloapis.com/billing but go.sum was not updated when the module was added, breaking CI builds.
The info and created-timestamp generators were accidentally merged into one YAML object, duplicating resource/families keys and failing kustomize validation.
Remove leftover spec.type from e2e org fixtures, drop empty spec from grant-creation assert, and treat missing BillingAccount API as no billing account in test clusters without billing CRDs.
The organization controller registered a BillingAccount informer even in test clusters without the billing API, contributing to controller-manager restarts and webhook connection refused failures during parallel chainsaw runs. Also extend owner-protection namespace cleanup timeout for CI load.
…feature flag Keep deprecated spec.type in the schema and switch webhook, onboarding, and membership behavior based on the feature gate so environments can roll out gradually without divergent CRD bundles.
Fix compile error in organization_defaults.go so webhook tests and CI builds succeed again.
The deployment passes --feature-gates=$(FEATURE_GATES) but the controller-manager command did not expose the flag, causing the pod to crash on startup during test-environment-validation.
E2E tests run with UnifiedOrganizations disabled, so organization creates must include spec.type until the test environment enables the unified gate.
🤖 Automatically added newlines to 21 file(s) Co-Authored-By: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
The create rules dereference audit.responseObject.metadata.name in their summary, but a rejected create (409/422/admission-deny) returns a Status object with no metadata.name. The rules still match (keyed on verb/request), so CEL raises "no such key: name", the event goes to the DLQ, and retries fail identically -- a slow DLQ leak (same class as the gateway policy leak that fired DLQSlowLeak in prod). Guard the leaf and fall back to audit.objectRef.name across the resourcemanager (project, organization), iam (role, group, serviceaccount), and identity (serviceaccount) create rules.
…creates The create summaries guarded responseObject.metadata.name and fell back to audit.objectRef.name, but a generateName create rejected before a name is assigned has an EMPTY objectRef.name. The else branch then derefs audit.objectRef.name on a Status responseObject and raises "no such key: name" again, so the event still dead-letters. The assigned name is carried on the Status at responseObject.details.name. Replace each create summary with a per-level-guarded fallback chain: responseObject.metadata.name -> objectRef.name -> responseObject.details.name -> literal The details branch is guarded with has(responseObject.details) because some Status responses carry no details. Confirmed live in the sibling NSO repo: activity-processor logs show generateName creates (Connector) re-failing the DLQ retry on the objectRef.name branch. Same latent gap exists for every metadata.name create rule here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Update unified-organizations migration guidance to reflect that legacy quota policies ship in the default datum bundle and the unified overlay is opt-in. Restore spec.type on the organization sample for legacy gate-off validation.
mattdjenkinson
force-pushed
the
feat/unified-organizations
branch
from
dace47b to
5f413f6
Compare
mattdjenkinson
changed the base branch from
main
to
perf/shared-etcd-client-pooling
Leader-only webhook startup caused connection refused on multi-replica deployments because Service endpoints include non-leader pods.
Unified orgs use generateName, so admission webhooks run before the final name exists. Defer namespace and owner OrganizationMembership creation to the organization controller after the apiserver assigns the name.
Drop the per-replica multicluster manager from webhook startup and wait for the webhook server to become ready before leader election proceeds. Note webhooks remain leader-only until HA note admission is implemented.
The synchronous webhook readiness wait caused controller-manager to exit when CSI serving certs were not yet available. Start webhooks in the background after certs appear instead of failing the main process.
JoseSzycho
previously approved these changes
Jul 1, 2026
JoseSzycho
left a comment
JoseSzycho
left a comment
Contributor
There was a problem hiding this comment.
I'd suggest to review the logic for a last organization member in a separate PR. We need to prevent the last member to be remove from the ORG, and possible review the scenario in which the Org creator leaves the organization, as it should change the owner of it.
Apply go fmt import ordering and refresh generated resourcemanager API docs plus chainsaw test documentation.
The organization context handler was rewriting /organizations/{name}/status
to /status, causing status updates to 404 and blocking onboarding reconciliation.
…ooling Combine the 2xx response guard from the base branch with generateName-safe summary fallbacks from unified organizations.
scotwells
approved these changes
Jul 6, 2026
mattdjenkinson
added a commit
to datum-cloud/cloud-portal
that referenced
this pull request
Jul 8, 2026
## Summary New users had no guided path from signup to a billable org with a project. This PR replaces the old `complete-profile` step with a multi-screen onboarding flow: profile review (when needed), account country, org contact + billing setup, then provisioning for the first project. Existing orgs that never finished billing get sent back through setup via middleware, so we don't strand people on org pages they can't actually use. Org creation from the organisations list now goes through the same billing form, with cleanup if someone abandons mid-flow. Needs: milo-os/milo#675 datum-cloud/datum#254 Key behaviours: - Onboarding routes: `/onboarding/profile` → `/onboarding/account` → `/onboarding/billing` → `/onboarding/provisioning` - Billing step collects org contact info, creates the org + billing account, and attaches a Stripe payment method before provisioning runs - `useSetupOnboardingBilling` waits on membership `RolesApplied` before billing POSTs, to avoid racing OpenFGA - Legacy org middleware redirects incomplete orgs to `/onboarding/billing?orgId=…` (contact info, billing account, or active payment method missing) - Org billing page no longer requires linked projects; orgs with a billing account but zero projects redirect to the account-level billing detail view - Billing address "same as contact" prefill with auto-uncheck when the contact address changes (`billing-address.ts` + unit tests) - 404 page primary action is context-aware: missing org → Organizations list, missing project → Projects (via `/project` redirect) - CI: publish workflow split from test workflow; Docker build context trimmed; Cypress coverage for onboarding, org creation, and not-found routing ## Breaking changes - `/onboarding/complete-profile` removed; users are routed through the new onboarding paths instead - Users with zero orgs are redirected into onboarding from most authenticated routes (invitations and fraud-review paths are exempt) ## Notes for reviewers - `org-billing-setup-form.tsx` and `use-setup-onboarding-billing.ts` are the main billing wiring; the provisioning page reuses `useSetupOnboardingProject` for the first project - 404 E2E navigation uses `window.location.assign` on the link href because client-side routing cannot exit the root error boundary in production builds - `.github/workflows/publish.yml` is new; branch pushes publish container images without re-running the full test suite (tests stay on `pull_request` in `ci.yml`)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Personal and Standard org types split the same resource into two code paths and locked people into workspaces they could not grow out of. This change removes
spec.type, adds org-levelcontactInfo, and introduces a single onboarding gate the API can enforce.Portal clients create orgs with
generateName: org-and a display name only. The mutating webhook defaults that prefix for non-admin callers; the validating webhook rejects user-chosen slugs. An org is onboarding-complete when contact info (email and name), a billing account in the org namespace, andDefaultPaymentMethodReadyon that account are all present. The organization controller writes that outcome tostatus.conditions[OnboardingComplete].Related enhancement: datum-cloud/enhancements#770. Portal UI work is tracked separately and is not in this PR.
Supersedes #673 (reopened after branch restore).
Test plan
go build ./...go test ./internal/controllers/resourcemanager/... ./internal/webhooks/resourcemanager/...spec.typegenerateName: org-assigns an opaque name and owner membershipOnboardingCompleteflips to True when contact info and billing payment method are readyBreaking changes
Organization.spec.typeis removed. Clients must stop sending or reading it.OrganizationMembership.status.organization.typeis removed;contactEmailis cached instead.metadata.generateNamewith prefixorg-.Notes for reviewers
go.miloapis.com/billingfor typed reads in the onboarding reconciler.docs/runbooks/unified-organizations-migration.md.Related to #636