Add versioned v1 backfill artifact contracts#3681
Open
cotti wants to merge 1 commit into
Open
Conversation
Defines the six document families the changelog backfill pipeline passes between its stages (inventory, overrides, semantic model, plan, provenance, ledger), each wrapped in a small envelope carrying the document kind and schema version so readers fail fast and explicitly on documents they do not understand. Adds canonical JSON serialization and deterministic SHA-256 hashing so plans are content-addressed, plus first-class identity types (canonical PR/issue URLs, synthetic file identities). Contracts and serialization only: no CLI surface changes, no behavior changes to existing changelog code. Part of elastic/docs-eng-team#672
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the versioned v1 contracts for the six document families the changelog backfill pipeline passes between its stages, so components can interoperate deterministically and plans can be content-addressed. Contracts and serialization only — no CLI surface changes (
docs/cli-schema.jsonis untouched), no behavior changes to existing changelog code, and everything is new files under:src/services/Elastic.Changelog/Backfill/— the contracts (namespaceElastic.Changelog.Backfill), with aREADME.mdgiving a plain-language tourtests/Elastic.Changelog.Tests/Backfill/— the testsThe six document families
inventoryoverridessemantic-modelProductReleaseshape from the epic: category family, optional precise type / breaking-change subtype, title, description, impact/action, highlight, product references, canonical PR/issue URLs, areas, source locations) plus structured triage diagnosticsplancreate-bundle,create-amend,create-supplemental-bundle,skip-existing,manual-review,conflict), pinned to all inputs: source commits, input document hashes, current-state snapshot, optional enrichment snapshot, and the deployed scrubber allowlist identity (content SHA-256 and/or deployment commit)provenanceledgerIdentity types are first-class: canonical PR URL, canonical issue URL, and synthetic file identity (stable name + checksum, the thing amend retractions match on). PR/issue references must always be full canonical URLs — bare numbers are rejected by validation.
Envelopes and schema versioning
Every persisted document is wrapped in an envelope carrying
artifact(the family name) andschema_version.BackfillDocuments.Deserialize<T>checks both before parsing the payload and throwsBackfillDocumentExceptionwith an actionable message on: invalid JSON, a non-object top level, a missing header field, an unknown artifact name, a different document kind than requested, or a schema version other than the one this code supports (exact match per family, all currently 1). There is no silent best-effort parsing. Deserialized documents are treated as external input:requiredfields are enforced by the serializer, semantic validation runs on every read and write and reports all problems at once with their location (e.g.releases[0]: entries[2]: …), and documents over a 64M-character safety bound are refused.Canonicalization and hashing
A document's identity is SHA-256 over the UTF-8 bytes of the canonical form of its envelope, written
sha256:+ 64 lower-case hex. Canonical form (CanonicalJson):\r\n/\rinside strings normalized to\nThe pretty-printed on-disk form never affects the hash: reading a file back and re-hashing gives the same answer.
AOT
All serialization goes through the source-generated
BackfillJsonContext(snake_case properties, kebab-case string enums viaJsonStringEnumMemberName, nulls omitted). No reflection; the AOT publish of docs-builder is warning-free with these types included.Test plan
All verified locally:
dotnet formatclean ·./build.sh build --skip-dirty-checkgreen ·dotnet publish -c Release(AOT) zero trim/AOT warnings ·tests/Elastic.Changelog.Tests: 873 passed, 0 failed (67 new) ·docs/cli-schema.jsonuntouchedPart of elastic/docs-eng-team#672
Note: issue #672 also resolves where checked-in inventories, overrides, plans, provenance, and ledgers live long term; that placement discussion can follow up on this PR without changing these contracts.