Skip to content

docs: address whitepaper review feedback (+ one code-driven HTL correction)#2

Draft
sanity wants to merge 6 commits into
mainfrom
whitepaper-review-improvements
Draft

docs: address whitepaper review feedback (+ one code-driven HTL correction)#2
sanity wants to merge 6 commits into
mainfrom
whitepaper-review-improvements

Conversation

@sanity

@sanity sanity commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What this is

Draft response to an end-to-end review of the whitepaper. The review was largely praise ("genuinely good... unusually honest"), with a set of specific, mostly-cheap improvements. This PR implements the ones that are clearly right, plus one factual correction the review did not raise but that I found while checking the codebase. Nothing here is merged — it's for your read, especially the code-driven correction.

The PDF compiles cleanly (make → 26 pages, no undefined refs).

Branched from origin/main (the version the reviewer actually read — the local main checkout was 5 commits behind, which is why an earlier look "had no figures").

Reviewer-driven changes (each commit is one point)

Commit Change Why
Fig 6 caption Reframe "stays above 1/d ⇒ good" as a qualitative consistency check; defer exponent fitting Kleinberg's bound is exponent-sensitive both ways — too many long links degrade greedy routing as much as too few. "Above the reference" = more long-range mass than 1/d, not per se better.
§6.4 New paragraph: honest peers behind a shared prefix (campus NAT, CGNAT, cloud region) collapse to the same ring location The Sybil analysis covers adversarial grinding but not the honest-collision direction, which strains gap-targeting, replication diversity, and the geographic-diversity assumption the app-layer defense itself leans on.
§5.5 One paragraph: the isotonic estimator conflates a neighbour's link quality with the quality of the topology behind it; aging window tracks the composite Reviewer's exact point; the paper already had the aging-window mechanism, this just names what it's doing.
§7.4 + open-problem 6 "Durable storage of cold state": the platform makes no durability guarantee for cold contracts Confirmed against core: storage is byte-budget LRU + demand-ordered eviction with no re-replication (a comment literally says loss "must be repaired by a re-PUT"). Cold state can be evicted from every host and lost — same property the original Freenet had.
§7.4 Reputation caveat on "merge is total" Accumulate-only reputation fits; revocation/slashing/stake is spend-shaped and inherits the double-spend limitation — so part of the well-fit list has a foot in the poorly-fit class.
§7.5 #4 Promote the §6.4 Sybil circularity into open-problem 4 App-layer defenses are routed through the very layer they secure; the open-problems list undersold how load-bearing this is.
§7.5 #2 Add a property-based test harness (QuickCheck-style monoid-law + Property 1 checks) as a concrete near-term step Turns the contract-author obligation from an honor system into a supported workflow. Framed as intended tooling, not as shipped (see "not in this PR").

⚠️ Code-driven correction (please scrutinise)

The paper's claim that request routing switches to a random walk above HTL 7 is factually wrong against current freenet-core. I traced every next-hop path: GET/PUT/SUBSCRIBE/UPDATE all forward greedily / performance-ranked; HTL is only a termination bound. The HTL-random-walk branch existed only in the CONNECT path and was removed in Nov 2025 (freenet-core #2065); the dead helper was deleted May 2026 (#4089). rnd_if_htl_above now survives solely as a config value read in one tracing::debug! line. The status table even marked this "Deployed."

Fixed in three places: §5.4 mechanism description, the §5.7 caveats (removed two references to a non-existent random-walk control), and the §7.2 status-table row. Also made the §5.7 / open-problem-1 sim-sweep framing honest.

Deliberately not in this PR

The routing scaling sweep — the reviewer's headline ask ("you have the harness, just run it"). This rests on a false premise. I traced the Turmoil harness: it is a correctness/liveness/convergence harness, not a scaling one. It doesn't measure hops-vs-N, N is a compile-time constant, there's no adaptive on/off toggle, the random-walk knob is dead (above), and organic topology formation stalls around N≈16 in the sim's time budget (N≥24 never finishes; the only way to N=40–80 injects a hand-wired topology that bypasses the routing you'd want to measure). So the sweep is genuine research/engineering — new hop-aggregation + a scaling fix — not a turn of an existing crank. The paper's modesty here is correct; I've adjusted it only so it doesn't imply the harness is turnkey-ready. Decision for you: invest in extending the harness (and the N≈16 formation ceiling may itself be a bug worth a look), or leave it deferred.

Building the property-test harness itself — that's freenet-stdlib/scaffold work, a separate PR in a different repo. Flagged here, not built.

[AI-assisted - Claude]

🤖 Generated with Claude Code

sanity and others added 6 commits July 2, 2026 09:26
Reviewer point: Kleinberg's bound is sensitive to the tail exponent in
both directions -- too many long-range links degrade greedy routing as
much as too few -- so the empirical curve lying above the 1/d reference
is not per se better, just more long-range mass than a pure 1/d law.
Reframe the caption as a qualitative consistency check and defer exponent
fitting (and its effect on hop count) to the scaling study.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N1sYYv4aMottkZ5S7kVmam
Reviewer point: hashing the address prefix (/24, /48) means every honest
peer behind a shared prefix -- a university/corporate NAT, a CGNAT pool,
a popular cloud region -- collapses to the SAME ring location, at
ring-distance zero. The Sybil analysis covers the adversarial (grinding)
direction but not this honest-collision direction, which strains
gap-targeting, replication diversity, and the geographic-diversity
assumption the application-layer Sybil defense itself relies on. Note the
tension: perturbing co-located peers apart reintroduces grinding freedom.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N1sYYv4aMottkZ5S7kVmam
…L claim (code-driven)

Two changes to the routing section:

1. Reviewer point (§5.5): the per-neighbour isotonic estimator, indexed
   by distance-to-target, folds together the neighbour's own link quality
   and the quality of the sub-network reachable through it; the latter
   shifts under churn faster, and the aging window is what keeps the curve
   tracking the composite. Added an acknowledging paragraph.

2. Code-driven correction (§5.4, §5.7): the paper claimed request routing
   switches to a random walk above HTL 7. This is FALSE against current
   freenet-core. Request forwarding (GET/PUT/SUBSCRIBE/UPDATE) is greedy /
   performance-ranked at every hop; HTL is only a termination bound. The
   HTL-random-walk branch existed ONLY in the CONNECT path and was removed
   in Nov 2025 (freenet-core #2065); the dead helper was deleted May 2026
   (#4089). rnd_if_htl_above now survives solely as a config value read in
   one tracing::debug! line. Corrected the §5.4 mechanism description and
   removed the two §5.7 references to a non-existent random-walk control;
   also made the sim-sweep framing honest (the harness validates
   correctness/convergence, it does not emit a hop-count-vs-N curve today).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N1sYYv4aMottkZ5S7kVmam
…ss (reviewer) + HTL status/open-problem correction (code-driven)

Reviewer-driven additions:
- §7.4 "Durable storage of cold state": the platform makes no durability
  guarantee for a contract nobody reads/subscribes/updates. Storage is a
  byte-budgeted LRU with demand-ordered eviction; no re-replication
  maintains a replica count (a loss is repaired only by a fresh PUT), so
  cold state can be evicted from every host and lost. Same property the
  original Freenet had; bears on the long-lived apps the paper foregrounds.
  Also added as open problem 6.
- §7.4 reputation caveat: "merge is total" bites the well-fit list --
  accumulate-only reputation fits, but revocation/slashing/stake is
  spend-shaped and inherits the double-spend limitation.
- §7.5 open problem 4: promoted the §6.4 Sybil *circularity* (app-layer
  defenses are routed through the layer they secure) into the list.
- §7.5 open problem 2: added a property-based test harness (QuickCheck-style
  monoid-law + sync-soundness checks) as a concrete near-term step that
  turns the contract-author obligation into a supported workflow.

Code-driven correction:
- Status table row "random walk above 7" -> "greedy/performance-ranked
  every hop" (see §5 correction commit).
- Open problem 1 (scaling): note the sim harness must be extended to emit
  hop-count-vs-N data it does not produce today.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N1sYYv4aMottkZ5S7kVmam
…#4642)

Audited the paper's hosting/subscription/eviction claims against
freenet-core origin/main (0.2.94). Several described the pre-epic system.
All facts below verified directly against origin/main sources, not the
local worktree (which was 46 commits stale).

- §4.7 subscription trees: renewal is INTEREST-GATED, not an unconditional
  background task. A peer renews a lease only while `contract_in_use`
  (local client subscription OR downstream subscriber); an idle host stops
  renewing and its branch collapses inward (#4697, the #3763 storm fix).
  Lease/renewal numbers (8 min / 2 min) confirmed still correct.

- §4.7: updates do not propagate along the subscription tree alone. A peer
  that hosts a contract announces it to neighbours; UPDATE broadcast targets
  are the union of the subscriber set and the neighbour host-advertisement
  mesh, deduplicated (`neighbor_hosting.rs`). Added.

- §5.5 GET/SUBSCRIBE: a routing terminus consults neighbour host
  advertisements before reporting a miss (#4646, "terminal advertisement
  consult"). Findability is routing + advertisement, not routing alone.
  Added; the paper previously implied routing alone.

- §6.6 + §7.2: storage is NOT "an LRU cache" with "user-configured budgets".
  Eviction is subscriber-primary: victims ordered ascending by
  (local_subs, downstream_subs, recency_seq, key); ties broken by least-recent
  REAL GET/PUT (SUBSCRIBE and renewal traffic deliberately do not refresh
  recency). The budget default is RAM-derived (capability-relative), with an
  operator override winning. Added a status row for the advertisement mesh.

- §7.4 durability: corrected. The paragraph added in the previous commit
  described an "LRU with a demand-ordered eviction policy and a short
  protective TTL whose size the operator sets" -- three of those four are now
  wrong. Greedy-Dual keep_score/predicted_demand are demoted to telemetry and
  no longer drive eviction; `min_ttl` was dropped 2026-07-08; the budget
  default is RAM-derived. Also: subscriber count is an ordering, NOT a hard
  pin (a subscribed contract is shed as a last resort under normal
  over-budget pressure), and eviction reclaims persisted state, parameters,
  and the code blob from disk -- which strengthens the no-durability claim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N1sYYv4aMottkZ5S7kVmam
- §6.6: the hosting budget is denominated in on-disk STATE bytes; only its
  DEFAULT is RAM-derived (clamp(RAM/8, 128MiB, 1GiB)). Previous wording
  conflated the denomination with the sizing rationale.
- §5.5 PUT: drop 'first peer with capacity' (placement is location-routed,
  and a disk-budget gate (#4702) can refuse the write); routing and hosting
  co-occur with no cache tier beneath hosting; note host announcement.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N1sYYv4aMottkZ5S7kVmam
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant