Prevent secret leaks from environment table wrapping - #3561
Conversation
There was a problem hiding this comment.
Review: Prevent secret leaks from environment table wrapping
The fix correctly addresses the root cause from #3548: NoWrap + Ellipsis() on the value column keeps each env value on one physical console row (so runner-side ::add-mask:: can still match), MakeSingleLine neutralizes embedded CR/LF as a belt-and-suspenders measure, and the switch from Markup.Escape(...) strings to Text cells is a nice side benefit — Text never interprets markup, so it's safer against control-sequence injection than the old escaped-string approach. Tests cover the wrap/mask/CRLF cases well, including a dedicated narrow-width render check.
One architectural point worth considering, not a blocker:
Sensitive-name detection duplicates an existing configurable pattern instead of reusing it. HttpRequestFormatter/HttpResponseFormatter already solve the "which names are secret-ish" problem via HttpLoggingOptions.SensitiveHeaderNames — a public, IOptions-configurable, user-overridable list with a documented default. This PR introduces a second, parallel mechanism: SensitiveEnvironmentVariableNameParts is a private static readonly string[] baked into PipelineInitializer, with no way for a consuming pipeline to extend or override it.
Why that matters here specifically: the substring match (name.Contains("PWD", ...), .Contains("KEY", ...)) is intentionally broad per the issue's spec, but it also means standard, non-secret shell variables like PWD/OLDPWD are now always masked, and any consumer whose own naming convention happens to contain one of these substrings (e.g. a TURNKEY_ENDPOINT or PRIMARY_KEY_COLUMN env var) loses visibility into that value in the startup table with no opt-out. The HttpLoggingOptions precedent exists precisely to let consumers tune this kind of heuristic per-project.
Suggestion: lift SensitiveEnvironmentVariableNameParts into a small options record (or extend SecretMaskingOptions) following the HttpLoggingOptions.SensitiveHeaderNames shape — public static Default list, IOptions<T>-injected, user-overridable via PipelineBuilder.Services. That keeps the two "mask by name heuristic" implementations in the codebase consistent, and gives consumers an escape hatch for false positives without needing another PR/fork. Not required for this fix to be correct or safe to ship, but worth a follow-up given PWD will now be masked in effectively every pipeline run.
Everything else looks solid — validation section in the PR description matches what I see in the diff, and I didn't find any correctness issues with the masking/truncation logic itself.
There was a problem hiding this comment.
Code review
No new changes since the previous review (the only commit added since 00ac5f7 is a merge of main, with no functional diff to the touched files) — so this is a verification pass rather than a fresh review.
Independently verified locally against the current HEAD (a7ccb9b):
dotnet build ModularPipelines.sln -c Release: 0 errors, 0 warningsPipelineInitializerTestsvia TUnit--treenode-filter: 10/10 passed
The masking/truncation logic in CreateEnvironmentVariablesTable holds up: sensitive-name detection short-circuits before obfuscation runs, NoWrap + .Ellipsis() keeps each value on one physical console row, and MakeSingleLine escapes embedded CR/LF as a second line of defense. Switching from Markup.Escape(...) strings to Text cells is a good safety improvement too. No correctness issues found.
Outstanding architectural point from the prior review — still unaddressed, still not a blocker:
SensitiveEnvironmentVariableNameParts (PipelineInitializer.cs#L24-L32) is a hardcoded private static readonly string[] with no way for a consuming pipeline to extend or override it. This duplicates a pattern the codebase already solves correctly elsewhere:
SecretMaskingOptions(src/ModularPipelines/Options/SecretMaskingOptions.cs) already exists as a DI-configurable options record for exactly this kind of "how do we detect/mask secrets" behavior — consumers configure it viabuilder.Services.Configure<SecretMaskingOptions>(...).HttpLoggingOptions.SensitiveHeaderNamesfollows the same shape (IReadOnlyList<string>+ staticDefault) for the equivalent "which names are secret-ish" problem on the HTTP logging side.
Concretely, the substring match here is intentionally broad (Contains("KEY"), Contains("PWD"), etc.), so it now always masks PWD/OLDPWD and any variable whose name happens to contain one of these substrings (e.g. TURNKEY_ENDPOINT, PRIMARY_KEY_COLUMN). Every consumer of this framework loses visibility into those values in the startup table, with no opt-out short of a fork or a follow-up PR to this repo.
Suggested follow-up: fold the sensitive name-part list into SecretMaskingOptions and inject it into PipelineInitializer the same way ISecretObfuscator is already injected. That keeps the two "mask by name heuristic" mechanisms in the codebase consistent and gives consumers a real escape hatch for false positives.
Not required for this fix to be correct or safe to ship — flagging again since it's still open, for the author/maintainer to decide whether to track as a follow-up issue.
There was a problem hiding this comment.
Code review
Reviewed the secret-masking / table-rendering changes in PipelineInitializer.CreateEnvironmentVariablesTable. The masking logic itself (name-heuristic short-circuit before obfuscation, CR/LF normalization applied after obfuscation, ordinal-ignore-case matching, Text instead of Markup.Escape to avoid re-interpretation) is sound and the new tests cover the intended behaviors well. One high-signal correctness bug found in the layout change — see inline comment.
No CLAUDE.md compliance issues found (only the root CLAUDE.md applies to these files, and the changes don't touch build/CI/module/codegen conventions it governs).
df0a3b6 to
2587cd8
Compare
|
Addressed the unresolved narrow-terminal review finding in
TDD: the focused test failed before the production change because the name rendered one character per line; it now passes. Validation:
@codex review |
|
Codex Review: Didn't find any major issues. Hooray! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
2587cd8 to
6ee031e
Compare
|
Rebased onto current @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ee031ef63
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Code reviewNo issues found. Checked for bugs, CLAUDE.md compliance, and correctness of the secret-masking logic. What I checked:
No CLAUDE.md violations, no compile/logic issues, no gaps in the masking heuristic that would defeat the PR's stated purpose. |
6ee031e to
9962bc3
Compare
|
Addressed the P2 |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9962bc32f1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Code reviewNo issues found. This is a follow-up review of the changes since the last Claude review — specifically commit What changed since the last review:
Validation performed:
Nothing further to flag — the PWD/OLDPWD fix is correct, narrowly scoped, and tested. |
9962bc3 to
5409ad9
Compare
|
Addressed authentication-bearing environment names in @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5409ad9e42
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Code reviewReviewed the latest commit, What changed: Correctness: the exemption check runs before the substring Architectural point (repeating from earlier review rounds, still unaddressed — not a blocker): this commit is a good illustration of why the earlier suggestion still stands. Each round has added one more hardcoded substring to The codebase already has the right shape for this in two places:
Since this PR keeps discovering new substrings/exemptions through review iteration, that's a reasonable signal the heuristic has stabilized enough to lift into an options record (sensitive name-parts + exemption list) injected into No bugs or CLAUDE.md violations found in this commit's diff. Tests are well-targeted for the new behavior. |
|
Addressed the latest P1 review by masking documented |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf8a51b782
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Code reviewReviewed the latest commit, What changed: adds a new Correctness: the new Architectural point (repeating from earlier review rounds — still unaddressed, still not a blocker, but worth escalating): this commit adds a third hardcoded, private list to this same mechanism —
Folding Not required for this fix to be correct or safe to ship. Given this is now the third round adding to the same hardcoded heuristic, I'd suggest tracking the options-record refactor as a concrete follow-up issue rather than continuing to patch the array in-place. |
|
Expanded masking to standard, .NET, and Azure connection-string environment variable conventions with regression coverage. @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: df0faa546b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Code reviewVerification pass on the current head ( What changed: Correctness:
Independently verified (core solution, per this repo's CLAUDE.md guidance):
No bugs or CLAUDE.md violations found in this commit's diff. Recurring architectural point (carried over from many earlier rounds, not touched by this commit, still non-blocking): Safe to merge as-is. |
Code review — PR #3561Reviewed the current diff ( Correctness holds up. The core fix is right: rendering the Value column through a custom 1. Architectural point raised on 2026-07-30 and never addressed across 30+ commits — now much harder to justify deferringEvery one of This isn't hypothetical risk — it's the demonstrated behavior of this exact PR. The commit history is a 30+ round game of whack-a-mole: The codebase already has the right shape for this problem: Why this matters for merge timing specifically: shipping this as private hardcoded arrays means every future gap-fix has to go through this same slow, single-file, one-var-at-a-time PR cycle again. Lifting it to options now — before more call sites or docs come to depend on the current shape — is a small, mechanical change; doing it later is a breaking API change for anyone who's since worked around the lack of extensibility. 2. Minor:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 70c385547d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0106f8a674
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Code review — PR #3561
Reviewed the current head (0106f8a) fresh, independently verified against source, and cross-checked against the full review history on this PR (46+ prior review rounds).
Independently verified:
dotnet build ModularPipelines.sln -c Release(core solution, per this repo's CLAUDE.md) — 0 errors, 0 warnings.PipelineInitializerTests(--treenode-filter '/*/*/PipelineInitializerTests/*') — 71/71 passed.
Correctness: The masking/rendering logic in PipelineInitializer.CreateEnvironmentVariablesTable is sound. The SafeEnvironmentValueRenderable.CreateRenderable re-evaluates RequiresUnsafeRendering against the actual maxWidth Spectre passes at Render time (not a cached estimate), so even if Measure sees a stale/larger width during column negotiation, the final Render call always makes the security-relevant decision using ground truth — the worst case from a wider Measure pass is a slightly-too-wide column, never a leaked value. The partial-obfuscation fallback (treat "obfuscator changed it but didn't collapse to exactly the mask" as unsafe) correctly closes the composite-value leak. Sensitive-name detection, the exemption lists, and SecretMaskingOptions.MaskValue honoring all check out against their test coverage.
Every previously-flagged Codex/Claude P1/P2 finding across this PR's long history has a corresponding "Fixed in <commit>" resolution with regression coverage — I didn't find any that were marked fixed but actually regressed in the final squashed diff.
Two points carried over from the last review round, still open, still non-blocking:
- Architectural:
SensitiveEnvironmentVariableNameParts/SensitiveEnvironmentVariableDelimitedNameParts/SensitiveEnvironmentVariableNames/NonSensitiveEnvironmentVariableNamesremain hardcodedprivate static readonlyarrays inPipelineInitializer, even though the constructor already injectsIOptions<SecretMaskingOptions>forMaskValue. Given this PR's own commit history is a live demonstration of the problem — 30+ rounds each adding one more name/exemption as a new provider convention surfaced — lifting these four lists ontoSecretMaskingOptionsas publicIReadOnlyList<string>properties (mirroringHttpLoggingOptions.SensitiveHeaderNames'sIReadOnlyList<string>+ staticDefaultshape) would let consumers extend/override the heuristic viaservices.Configure<SecretMaskingOptions>(...)instead of needing a new PR here every time. The DI seam for this is already sitting unused in this same file. Worth a tracked follow-up issue rather than blocking this fix. - Minor/dead code:
MakeSingleLine's\r/\nreplacement is unreachable with real content by the time it's called —RequiresUnsafeRenderingalready forces a full mask whenever the unchanged value contains a control character (value.Any(char.IsControl)), and in the changed-value branch it only skips masking whenobfuscatedValueis already exactly the mask (which never contains\r/\n). Worth deletingMakeSingleLineand its two.Replacecalls next time this file is touched, so a future reader isn't misled about which layer owns CR/LF safety.
Nothing here blocks merge — the fix for #3548 is correct, thoroughly tested (71 tests covering the full matrix of sensitive names, exemptions, width edge cases, and configured-mask behavior), and safe to ship as-is.
Mask CloudAMQP service URLs while preserving standard Git configuration key names.
There was a problem hiding this comment.
Code review — PR #3561
Reviewed the current head (8086ac7), independently verified, and diffed against the last full review round (4838268943 at 0106f8a, 2026-08-02T11:27:52Z). Only one commit landed since then.
Delta since last review (8086ac7, "refine environment masking"):
CLOUDAMQP_URLadded toSensitiveEnvironmentVariableNames— correct, it's a credential-bearing AMQP connection URL.- New
IsGitConfigKeyNamecarve-out forGIT_CONFIG_KEY_<n>(checked before theKEYsubstring match applies): these hold Git config key names (e.g.http.extraHeader), not values, so exempting them from theKEYsubstring match is correct —GIT_CONFIG_VALUE_<n>(the actual value) is still forced-sensitive a few lines below, which is the right split. - Both additions have matching regression tests. Build/tests aren't independently re-run here since this is a narrow, low-risk delta already covered by the existing 30+ test matrix.
No correctness issues in this delta.
Two points carried over from prior rounds — still open, now spanning 9 consecutive review rounds:
-
Architectural (repeatedly flagged since the first review on 2026-07-30):
SensitiveEnvironmentVariableNameParts/SensitiveEnvironmentVariableDelimitedNameParts/SensitiveEnvironmentVariableNames/NonSensitiveEnvironmentVariableNamesare still hardcodedprivate static readonlyarrays inPipelineInitializer, even though this class already injectsIOptions<SecretMaskingOptions>. This PR's own history is the strongest evidence for the fix: 30+ commits, each adding one more provider-specific name or exemption (Redis, Azure Storage, NuGet, CloudAMQP, Git config keys, Yarn/npm auth modes, GitLab/CI author metadata...) as new conventions surfaced one at a time. That pattern doesn't converge — there's no bound on the number of CI systems, cloud providers, and in-house tools with their own env-var naming conventions, so this list will need a new upstream PR forever.The codebase already has the right shape for this elsewhere:
HttpLoggingOptions.SensitiveHeaderNamesis a public, DI-configurableIReadOnlyList<string>with a documentedDefault, so a consumer whose own naming convention isn't recognized can extend it viaservices.Configure<HttpLoggingOptions>(...)without waiting on a release. Lifting these four lists ontoSecretMaskingOptionsthe same way would (a) stop new provider-specific names from requiring a new PR/release cycle, and (b) let a user affected by an unrecognized-but-sensitive name in their environment fix it themselves immediately, which is strictly better for a security-hygiene feature than "file an issue and wait."This has been raised in essentially every review round on this PR and acknowledged each time as reasonable-but-non-blocking. Given it's now round 9 and still no tracking issue exists for it (checked open/closed issues — none reference this), I'd suggest either implementing it as part of this PR (the DI seam for
IOptions<SecretMaskingOptions>is already sitting right there, unused for this purpose) or filing a tracked follow-up issue before merging, so the recommendation doesn't just get silently dropped once this PR closes. -
Minor/dead code (flagged in the immediately preceding round, still unaddressed):
MakeSingleLine's\r/\nreplacement is unreachable with real content given the current guard ordering inSafeEnvironmentValueRenderable.CreateRenderable—RequiresUnsafeRenderingalready forces the mask value whenever an unchanged value contains any control character, and in the changed branch it only takes the "safe" path whenobfuscatedValuealready equals the mask string exactly (which won't contain\r/\nin practice). Worth deletingMakeSingleLineand its two call sites'.Replacecalls next time this file is touched, so a future reader isn't misled into thinking this method is the layer responsible for CR/LF safety.
Nothing here blocks merge — the fix for #3548 (single-line, width-aware, name-heuristic-backed masking of the environment-variables trace table) is correct and thoroughly tested. The two items above are about the long-term maintainability of the mechanism, not its current correctness.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8086ac7d93
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c92ccad9b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Code review — PR #3561Reviewed the changes since the last Claude review ( Correctness holds up, and it's a nice piece of design in its own right.
No bugs or CLAUDE.md violations found in this diff. Recurring architectural point (carried over from many earlier rounds, still non-blocking): Safe to merge as-is. |
Keep canonical numeric Git config keys visible only when their values do not contain URI user-info.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e918b642b9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Code review — PR #3561Reviewed the only commit since the last Claude review round ( What changed: Correctness:
Independently verified locally (per this repo's CLAUDE.md guidance — core solution only, via the agent build guard):
No bugs or CLAUDE.md violations found in this commit's diff. Recurring architectural point (unaddressed, still non-blocking): as flagged across many prior rounds on this PR, Safe to merge as-is. |
Align the nested-module code-fix fixture with formatting preserved by the current main implementation.
Code review — PR #3561Reviewed the two commits landed since the last Claude review round ( Delta since last review:
Independently verified:
No bugs or CLAUDE.md violations in this delta — it's a mechanical, correct fixture update. Carried over from prior rounds, still open, still non-blocking (untouched by this delta):
Nothing here blocks merge — the fix for #3548 remains correct and thoroughly tested; this round's changes are outside the feature itself. |
Treat the container authorization token file as a diagnostic path while continuing to mask the bearer token variable.
|
Pushed |
Code review — PR #3561Reviewed the only commit since the last Claude review round ( What changed:
Correctness check: Independently verified:
No bugs or CLAUDE.md violations in this delta — it's a minimal, correctly-scoped, well-tested fix that follows the exact pattern already established for the other credential-file exemptions in this list. Carried over from prior rounds, still open, still non-blocking (untouched by this delta):
Safe to merge as-is. |
Summary
Textcells so long or multiline values never become multiple physical log linesSecretMaskingOptions.MaskValuefor name-based masking, including the safe default fallback for blank configurationPWD,OLDPWD, andSSH_AUTH_SOCKdiagnosticsValidation
PipelineInitializerTests: 25 passed, covering narrow-console truncation, CR/LF normalization, sensitive-name conventions, exemptions, and configured mask behaviorModularPipelines.slnRelease build: 0 errors; 227 existing warningsRotation/expiry of any credentials exposed by historical logs remains an external owner/security action.
Closes #3548
Follow-up review fix
PipelineInitializerTests(26/26 passing).Render-width follow-up
CONFIGvalue at width 40 plus the exact safe-fit boundary.PipelineInitializerTests: 28/28 passing; scoped formatter clean.Header-width follow-up
Nameheader width when calculating safe raw-value capacity, preventing Spectre padding around one-character variable names from exposing truncated raw values.PipelineInitializerTests(29/29 passing); scoped whitespace formatter clean.Partial-obfuscation follow-up
PipelineInitializerTests: 30/30 passed; scoped whitespace formatter clean.