Overhaul Explanation and Reference sections

[wardbryan3]

Summary

Restructures the Explanation and Reference documentation sections to follow LXD-style themed groupings, purges how-to contamination from reference pages, deduplicates concept content, and relocates misplaced content.

Changes

Index restructure (LXD-style themed groups)

• explanation/index.md: Flat list replaced with two themed groups: "Core concepts" (cross-cutting mental models) and "By area" (concept deep-dives per product area). Dropped the "Developer tools" group (CLI architecture moved to reference; SDK/API links already pointed to reference).
• reference/index.md: Flat list replaced with three themed groups: "By area" (mirroring how-to-guides areas), "Developer tools & internals" (cross-cutting dev material), and "Frequently used reference pages" (shortcut block).

How-to contamination purge

• Eight editable reference sub-indexes (clusters, clusters/federations, jobs, storage, teams, licenses, settings/iam, settings/compute-providers) had "Get started / Quickstart / Next steps" CTA blocks replaced with a single minimalist "## See also" section (two links: matching explanation page + primary how-to guide).
• reference/clusters/on-premises.mdx (734-line how-to guide with numbered steps, code blocks, troubleshooting) moved to how-to-guides/clusters/on-premises.mdx. A short lookup-style reference stub with a methods comparison table left in its place. 13 inbound links across 11 files repointed to the how-to guide.
• reference/workbench/index.mdx rewritten from a phase-based structure (Workspace/Develop/Compute/Train/Serve with sectionLabel dividers) to the flat "What you'll find inside" + "See also" pattern used by every other reference index. Sidebar sectionLabel dividers removed.

Concept deduplication

• Four explanation/concepts/ pages trimmed of tables that duplicated area-specific .mdx files. Each concept page now keeps the "why" and relationships; area pages own the detail (tables, lifecycles, modes). Cross-references sections added.
• explanation/concepts/jobs-and-pipelines.md: "Where to go next" renamed to "Cross-references" with a link to /explanation/jobs added for parity with sibling concept pages.

Content relocation

• explanation/cli-architecture.md deleted (byte-identical duplicate of the build-pulled reference/developer-tools/cli/architecture.md). The reference copy is now surfaced in the sidebar under Reference > Developer tools > CLI > Architecture. Redirect added in docusaurus.config.js from /explanation/cli-architecture to /reference/developer-tools/cli/architecture.
• explanation/workbench/overview.mdx promoted to top-level explanation/workbench.mdx (matching the pattern of every other area). Deep-dive pages (pipeline-anatomy, sweep-algorithms, predictive-vs-llm) remain in the workbench/ subfolder. Redirect added from /explanation/workbench/overview to /explanation/workbench.

Build status

just build succeeds. Only remaining broken-link warning is a pre-existing slurm-juju link in gitignored build-pulled v8x CLI content (out of scope).

Known follow-up

~29 deeper editable reference sub-indexes and leaf pages (e.g., clusters/kubernetes/index.mdx, workbench/pipelines/index.mdx, workbench/sessions/notebooks.md, workbench/endpoints/pods.mdx) still contain "Get started / Next steps / Quickstart" how-to contamination. These were outside the original spec's 8-file scope. A follow-up task should complete the purge across those pages.

[Copilot]

Pull request overview

This PR reorganizes the docs IA for Explanation and Reference to better separate conceptual material from task-oriented guidance, and to align navigation with LXD-style themed groupings. It also relocates a large on-premises cluster setup guide out of Reference into How-to Guides, with redirects and link updates to preserve existing URLs.

Changes:

• Restructures explanation/ and reference/ landing pages and sidebar navigation into themed groupings, and removes Workbench sidebar phase dividers.
• Purges “Get started / Next steps” CTA contamination from several reference index pages, replacing it with minimal “See also” cross-links.
• Moves the long on-premises cluster setup guide into how-to-guides/, leaving a lookup-style reference stub and adding redirects for moved/deleted pages.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
sidebars-main.js	Updates sidebar structure: adds on-premises how-to entry, promotes Workbench explanation doc ID, removes CLI architecture from Explanation, reshapes Workbench reference nav.
docusaurus.config.js	Adds redirects for removed/moved Explanation pages (cli-architecture, workbench/overview).
docs/tutorial.mdx	Repoints on-premises guide link from Reference to How-to Guides.
docs/reference/workbench/index.mdx	Rewrites Workbench reference landing page into a flat “What you’ll find inside” + “See also” pattern.
docs/reference/teams/index.md	Replaces “Get started/Next steps” CTA block with “See also” cross-links.
docs/reference/storage.md	Replaces “Get started/Next steps” CTA block with “See also” cross-links.
docs/reference/settings/iam/index.md	Replaces “Get started/Next steps” CTA block with “See also” cross-links.
docs/reference/settings/compute-providers/on-premises.md	Repoints on-premises cluster guide links to the new how-to location.
docs/reference/settings/compute-providers/index.md	Replaces “Get started/Next steps” CTA block with “See also” cross-links.
docs/reference/licenses/index.md	Replaces “Get started/Next steps” CTA block with “See also” cross-links.
docs/reference/jobs/index.mdx	Replaces “Get started/Next steps” CTA block with “See also” cross-links.
docs/reference/index.md	Restructures the Reference landing page into “By area”, “Developer tools & internals”, and “Frequently used reference pages”.
docs/reference/developer-tools/cli/index.md	Updates Architecture link to point at the Reference copy.
docs/reference/clusters/slurm/index.mdx	Repoints on-premises links to the new how-to location.
docs/reference/clusters/on-premises.mdx	Replaces the former long how-to content with a short methods comparison + “See also”.
docs/reference/clusters/kubernetes/index.mdx	Repoints on-premises links to the new how-to location.
docs/reference/clusters/index.md	Adds a “See also” section (but still contains outdated on-premises setup wording/link; see comment).
docs/reference/clusters/federations/index.md	Replaces “Get started/Next steps” CTA block with “See also” cross-links.
docs/how-to-guides/clusters/troubleshooting.mdx	Repoints on-premises link to the new how-to location.
docs/how-to-guides/clusters/slurm/create.mdx	Repoints on-premises link to the new how-to location.
docs/how-to-guides/clusters/on-premises.mdx	Adds the relocated on-premises clusters setup guide (full how-to content).
docs/how-to-guides/clusters/kubernetes/create.mdx	Repoints on-premises link to the new how-to location.
docs/how-to-guides/clusters/get-started.mdx	Repoints on-premises link to the new how-to location.
docs/how-to-guides/clusters/federations/tutorials.mdx	Repoints on-premises link to the new how-to location.
docs/how-to-guides/cloud-accounts/connect-cloud-account.mdx	Repoints on-premises link to the new how-to location.
docs/explanation/workbench.mdx	Adds a top-level Workbench “Concepts” page (moved/promoted from the prior workbench/overview).
docs/explanation/index.md	Restructures Explanation landing page into “Core concepts” and “By area”; updates Workbench link target.
docs/explanation/concepts/workbench-ml-stack.md	Updates cross-references to point at the new Workbench concepts page.
docs/explanation/concepts/teams-and-iam.md	Deduplicates role table content and adds cross-references to area pages.
docs/explanation/concepts/jobs-and-pipelines.md	Renames “Where to go next” to “Cross-references” and adds an area-page link.
docs/explanation/concepts/compute-and-clusters.md	Removes duplicated comparison tables and adds cross-references to area pages.
docs/explanation/cli-architecture.md	Deletes duplicated Explanation copy of CLI architecture (redirected to Reference).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.