v1.11.0a4 - Age Restrictions & Snappiness#678
Open
Conversation
Adds two new browser filter options ("Age Rating" for the normalized
metron_age_rating and "Age Rating (Tagged)" for the original age_rating FK)
and reworks the FTS schema so the two age ratings are indexed separately.
- Rename the ComicFTS age_rating column to tagged_age_rating and add a
new metron_age_rating column. Search aliases "age" and "age_rating" now
map to metron_age_rating; tagged_age_rating is keyed-only.
- Register metron_age_rating in BROWSER_FILTER_KEYS and the filter/choice
serializers so the frontend menu surfaces it automatically. Frontend
filter sub-menu adds an FILTER_TITLE_OVERRIDES map so the two ratings
render as "Age Rating" and "Age Rating (Tagged)".
- Importer: drop metron_age_rating from NON_FTS_FIELDS so the plain
CharField flows into FTS, and apply FTS_FIELD_RENAME_MAP when writing
the age_rating FK's linked name to FTS so it lands under
tagged_age_rating.
- Migration 0040: add SettingsBrowserFilters.metron_age_rating JSONField
and drop+recreate the codex_comicfts virtual table with the new schema
(it is rebuilt on the next librarian sync).
- Update importer test fixtures to reflect the new FTS key layout.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Consolidates all age-rating code into codex/models/age_rating.py. Adds metron_name/metron_index fields to AgeRating and computes them in presave() via to_metron_age_rating(). Drops Comic.metron_age_rating in favor of Comic.age_rating.metron_name, eliminating a redundant CharField and keeping the tagged/normalized pair FK-local. Renames for consistency: GroupAuth.metron_age_rating -> age_rating_metron; SettingsBrowserFilters.age_rating -> age_rating_tagged and metron_age_rating -> age_rating_metron; ComicFTS columns tagged_age_rating/metron_age_rating -> age_rating_tagged/age_rating_metron. Frontend mirrors with metronAgeRating -> ageRatingMetron. AgeRatingACLMixin now uses precomputed metron_index for O(1) allowed-set resolution (integer range query over indices rather than per-group rating_index() lookups). Filter menu now populates age_rating_tagged and age_rating_metron directly from the AgeRating model. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Replace the `AgeRating.metron_name` / `metron_index` denormalized cache with a proper `AgeRatingMetron` lookup table so the browser filter rows `SettingsBrowserFilters.age_rating_metron` actually resolve — fixes the `no such column: codex_settingsbrowserfilters.age_rating_metron` crash on branches where the column was missing. - Add `AgeRatingMetron(NamedModel)` with an `index` sort key. Seeded from `MetronAgeRatingEnum` in migration 0039 and re-asserted on every startup via `init_age_rating_metron()`; rows are never deleted. - Replace `AgeRating.metron_name`/`metron_index` with a single `metron = FK(AgeRatingMetron, SET_NULL)`; `presave()` resolves the FK from `to_metron_age_rating(name)` so re-import heals any stale links. - Migrate filter / ACL / FTS / choices / saved-settings paths to traverse `age_rating__metron__*` instead of the dropped cache columns. Collapse the Age-Rating-Metron-specific branches in `BrowserChoicesView` into the generic m2m-style path by registering `AgeRatingMetron` in `_FIELD_TO_REL_MODEL_MAP` and `_BACK_REL_MAP` (uses the `agerating__` reverse to hop back to `Comic`). - Serializers: add `AgeRatingMetronSerializer(pk, name, index)` and nest it on `AgeRatingSerializer`; drop the char-PK override for `age_rating_metron` choices so it uses integer PKs like `age_rating_tagged`. - Frontend metadata ratings: read `ar.metron?.name` instead of the removed `ar.metronName`. - Migration 0039 is modified in place (not yet released): creates `AgeRatingMetron`, seeds it, adds the FK, backfills from the old cache columns, then drops them. `makemigrations --dry-run` is clean. Frontend ignore files were resorted by `bin/sort-ignore.sh` and `bun.lock` picked up a minor vitest resolution bump while running the linter; included here to keep the tree clean. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
- ``frontend/src/api/v3/browser.js``: five URL template literals had
stray spaces inserted around the ``/`` and ``?`` separators, so every
request built from them resolved to ``/`` on the server. The reader's
``updateGroupBookmarks`` PATCH landed on the site root and Django
replied ``Method Not Allowed: /``. The same breakage affected
``getGroupDownloadURL``, ``getLazyImport``, ``loadSavedSettings``, and
``deleteSavedSettings``. Restore the compact form so each URL resolves
to its real endpoint.
- ``codex/views/browser/choices.py``: the Age-Rating filter submenu was
alphabetical even though ``AgeRatingMetron.Meta.ordering = ("index",)``
should have surfaced enum order. ``.distinct()`` combined with
``.values("pk", "name")`` strips the default ordering because
``index`` is not in the SELECT projection. Include ``index`` in the
values list and add an explicit ``.order_by("index")`` so the submenu
renders Everyone → Adult instead of alphabetically.
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* Move age-rating ACL from groups to per-user with anonymous flag Rename UserActive to UserAuth and move the age_rating_metron FK from GroupAuth onto UserAuth. Each user now owns a single ceiling; the old per-group restriction and its union logic are gone. Add a new ANONYMOUS_USER_AGE_RATING (AA) admin flag to cap anonymous sessions independently, and flip AGE_RATING_DEFAULT (AR) from Adult to Everyone so untagged comics are visible by default. Rewrite AgeRatingACLMixin as a single lazy Q built from SQL subqueries over UserAuth and AdminFlag so the filter composes without reading values in Python. Admins pick up a new read-only GET /admin/age-rating-metron endpoint that feeds the user-edit dropdown. Frontend: the Users tab gains an Age Rating column and a read-only anonymous-session display sourced from the AA flag; the Groups tab loses its age-rating column and picker; the Flags tab gains a settable AA row. Help copy on the Groups tab now points users at Users/Flags for age-rating changes. Tests cover the ACL as a single-query filter for a ranked user, an unrestricted user, an anonymous session, and the AR-default behavior for untagged/unknown-rated comics, plus a migration-shape test and a serializer roundtrip. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * sort ignore files Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
The two age-rating flags (AGE_RATING_DEFAULT, ANONYMOUS_USER_AGE_RATING) previously stored their rating as a free-form name string in AdminFlag.value, parallel to a frontend-only METRON_AGE_RATING_CHOICES enum. That left the ACL filter matching rating names across a denormalized join and the admin UI driving a hardcoded choice list independent of the live AgeRatingMetron seed table. Add a nullable age_rating_metron FK on AdminFlag (SET_NULL) and migrate existing AR/AA value strings into it. The ACL mixin now hops the FK with a single Subquery/Exists on age_rating_metron__index, dropping the name match and its Python fallback. The admin flag-tab binds the <v-select> directly against the AgeRatingMetron store slice, and user-tab resolves the flag FK through the same store — so deleting a metron row cleanly nulls the flag (falls back to the seeded default at next boot) instead of silently desyncing. Drops the now-unused METRON_AGE_RATING_CHOICES JSON emitter, the SELECTABLE_RATINGS constant, and the Flag load on group-tab. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The two age-rating flags (AGE_RATING_DEFAULT, ANONYMOUS_USER_AGE_RATING) previously stored their rating as a free-form name string in AdminFlag.value, parallel to a frontend-only METRON_AGE_RATING_CHOICES enum. That left the ACL filter matching rating names across a denormalized join and the admin UI driving a hardcoded choice list independent of the live AgeRatingMetron seed table. Add a nullable age_rating_metron FK on AdminFlag (SET_NULL) and migrate existing AR/AA value strings into it. The ACL mixin now hops the FK with a single Subquery/Exists on age_rating_metron__index, dropping the name match and its Python fallback. The admin flag-tab binds the <v-select> directly against the AgeRatingMetron store slice, and user-tab resolves the flag FK through the same store — so deleting a metron row cleanly nulls the flag (falls back to the seeded default at next boot) instead of silently desyncing. Drops the now-unused METRON_AGE_RATING_CHOICES JSON emitter, the SELECTABLE_RATINGS constant, and the Flag load on group-tab. Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* replace AdminFlag.value string with typed age_rating_metron FK
The two age-rating flags (AGE_RATING_DEFAULT, ANONYMOUS_USER_AGE_RATING)
previously stored their rating as a free-form name string in
AdminFlag.value, parallel to a frontend-only METRON_AGE_RATING_CHOICES
enum. That left the ACL filter matching rating names across a denormalized
join and the admin UI driving a hardcoded choice list independent of the
live AgeRatingMetron seed table.
Add a nullable age_rating_metron FK on AdminFlag (SET_NULL) and migrate
existing AR/AA value strings into it. The ACL mixin now hops the FK with
a single Subquery/Exists on age_rating_metron__index, dropping the name
match and its Python fallback. The admin flag-tab binds the <v-select>
directly against the AgeRatingMetron store slice, and user-tab resolves
the flag FK through the same store — so deleting a metron row cleanly
nulls the flag (falls back to the seeded default at next boot) instead
of silently desyncing.
Drops the now-unused METRON_AGE_RATING_CHOICES JSON emitter, the
SELECTABLE_RATINGS constant, and the Flag load on group-tab.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* sort ignore files
Drive-by re-sort from ``make fix`` — ``bin/sort-ignore.sh`` canonicalises
the line order across all six ignore files. No behaviour change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* sqlite: tune pragmas on connect + drop userauth from cachalot uncacheable
Two tweaks aimed at the ACL hot path:
``_SQLITE_PRAGMAS`` runs on every new DB connection via
``DATABASES.default.OPTIONS.init_command``. ``synchronous=NORMAL`` drops
per-commit fsyncs (safe under WAL, ~2-5x write throughput),
``temp_store=MEMORY`` keeps scratch space off disk for ORDER BY /
GROUP BY / subqueries, and ``mmap_size=256MiB`` + ``cache_size=64MiB``
serve most repeat reads out of RAM on 64-bit hosts.
``CACHALOT_UNCACHABLE_TABLES`` shrinks from the older opt-out list to
just the two internal Django tables (``django_migrations``,
``django_session``) that really do need bypass. ``codex_userauth`` is
now cache-eligible: every write path goes through Django ORM save/
update_or_create, both of which fire ``post_save`` so cachalot's
invalidation hook catches the update. The only high-frequency writer
(``user_active`` touch) is throttled to 1-hour resolution, bounding
cache churn.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* denormalize Comic.age_rating_metron_index for the ACL filter
Adds a nullable integer column on Comic mirroring
``age_rating.metron.index`` + a composite index on
``(library_id, age_rating_metron_index)``. The ACL age-rating filter
(next commit) compares this column directly, collapsing the old
two-hop join (``comic -> age_rating -> metron``) to a single
index-friendly local predicate.
Semantics of the stored value:
* NULL — comic has no ``age_rating`` FK, or its AgeRating didn't map
to a canonical AgeRatingMetron. Handled via the ``AR`` default fallback.
* ``-1`` (``UNRANKED_METRON_INDEX``) — tagged ``Unknown``; same
fallback path.
* ``0..5`` — ranked rating, compared directly against the user ceiling.
``Comic.presave`` keeps the column in sync. The presave reads
``age_rating.metron_id`` off the local FK column (via ``getattr``) rather
than following ``.metron``; the importer's bulk path doesn't
``select_related("metron")``, so the relation traversal would fire one
lazy query per comic. ``get_metron_index`` serves lookups out of a
process-lifetime pk -> index map (7-row table, sealed at seed), so the
steady-state cost is a dict hit. The cache is invalidated at startup
after ``init_age_rating_metron`` re-asserts the seed.
Migration 0041 is the one-shot backfill: a single correlated SQL
``UPDATE`` scoped to ``age_rating_id IS NOT NULL``. SQLite evaluates the
subquery row-by-row over the FK indexes, so the whole migration is one
statement regardless of library size (~10-20s on a 600k-row library,
dominated by WAL fsync).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* rewrite ACL filter with per-request scalar memo
Restructures ``codex.views.auth`` around three scalar values resolved
once per request and reused across every model filter:
* ``visible_library_pks`` — frozenset of library pks, built from three
tiny (cachalot-cached) lookups via Python set algebra: ungrouped
libraries | (include-groups - exclude-groups). Group ACL becomes
``library_id__in=<pks>`` with no M2M traversal at Comic scan time.
* ``max_idx`` — Python int rating ceiling. Authenticated users read
``UserAuth.age_rating_metron__index`` in one FK hop; anonymous
sessions read the ``AA`` admin flag's FK index. ``UNRESTRICTED_RATING_INDEX``
(999) sentinel lets unrestricted users pass ``__lte`` without branching.
* ``default_fits`` — bool answering "does the ``AR`` default's rating
fit under ``max_idx``?". Gates whether null/unknown-rated comics
inherit the default and pass the user's ceiling.
``GroupACLMixin.init_group_acl`` seeds a three-slot memo; each scalar
is lazily populated on first access and reused for every subsequent
``get_acl_filter(model, user)`` call. A browser request that filters
7+ models now pays three tiny bookkeeping queries total instead of
three-per-model.
The resulting Q is two index-friendly predicates:
* ``library_id__in=(…)`` against the precomputed set.
* ``age_rating_metron_index`` compared to ``max_idx`` (ranked branch)
OR-optional with ``isnull=True`` / ``=UNRANKED_METRON_INDEX``
(null/unknown branch, only when ``default_fits``).
The composite index on ``(library_id, age_rating_metron_index)`` from
the previous commit serves both halves index-only.
Classmethod one-shot entry points remain on ``AgeRatingACLMixin`` and
``GroupACLFilterMixin`` for tests and isolated callers without a
request. Tests assert:
* the scalar-already-resolved path is one DB query (hot path contract);
* the classmethod one-shot path is at most three queries (test budget);
* ``Comic.presave`` populates ``age_rating_metron_index`` correctly for
ranked, ``Unknown``, and null-rated comics.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
The JSD nightly janitor was added in #681 to clean up duplicate ``codex_comicfts`` rows produced by the (now-fixed) sync.py iteration bug. With the source bugs fixed (#681 watermark walk; #683/#684 delete+create swap), no new duplicates can land, so a permanent recurring task isn't earning its keep — it would just mask any future regression instead of surfacing it. Move the cleanup to the migration boundary instead: * ``codex/migrations/0039_…`` gains a ``RunSQL`` step that runs the same ``DELETE … WHERE rowid NOT IN (SELECT MIN(rowid) … GROUP BY comic_id)`` ahead of the existing FTS DROP+CREATE. For v1.10 -> v1.11 fresh upgrades the DROP makes the dedupe a no-op; the step keeps the migration idempotent if a future change preserves data instead of dropping. * Drop the ``JSD`` entry from ``_LIBRARIAN_STATUS_CHOICES`` since the task it referenced no longer exists. * Remove ``JanitorFTSDedupeTask``, ``JanitorDBFTSDedupeStatus``, the ``fts_dedupe`` function and method, and all wiring in ``janitor.py`` (``_NIGHTLY_TASK_CLASSES``, ``_JANITOR_METHOD_MAP``, ``_JANITOR_STATII``). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related fixes to ``ensure_db_schema``:
1. Stop running ``fts_integrity_check`` / ``fts_rebuild`` ahead of
migrations. Migrations like 0039 DROP + CREATE ``codex_comicfts``,
so any rebuild done before migrate is wasted: build a fresh FTS
index, then immediately drop the whole table. FK and integrity
checks stay pre-migration since corrupt source data can fail
``AlterField`` / ``RunPython`` mid-migrate, but FTS health is a
runtime concern that doesn't gate any other migration.
2. After ``call_command("migrate")``, queue a
``JanitorFTSIntegrityCheckTask`` if ``FTS_INTEGRITY_CHECK`` is on.
The task auto-queues a rebuild on failure (existing wiring in
``JanitorIntegrity.fts_integrity_check``), so the corruption case
is self-healing without blocking codex startup. Browse, reader,
and OPDS work immediately; search degrades only until
``SearchIndexSyncTask`` repopulates the table.
3. Cache the ``_has_unapplied_migrations()`` result in
``ensure_db_schema`` so the backup and repair paths share one
answer instead of each issuing its own SELECT against
``django_migrations``.
``CODEX_FTS_REBUILD`` is no longer read by startup. The setting is
still defined in ``codex/settings/__init__.py`` for any future
caller; the admin tasks endpoint
(``codex/views/admin/tasks.py:91``) still reaches the rebuild task
directly via ``JanitorFTSRebuildTask``, so operators retain a
manual rebuild path.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ajslater
changed the title
A 2022/2023-era importer bug let directories slip into codex_comic with paths lacking a comic suffix (.cbz/.cbr/.cb7/.cbt/.pdf). The runtime importer rejects these now via match_comic, but the stale rows persist. Add a one-shot cleanup pass to migration 0039 that finds Comic rows whose path lacks a comic suffix, stats them on disk, and deletes any that are directories, missing, or unreadable. Real files with non-comic names are surfaced for manual review rather than blindly deleted. Runs before _backfill_age_rating_metron_index so the backfill doesn't touch rows about to be deleted. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… in 0039 (#690) Two paired changes for the ``Folder.DoesNotExist`` traceback that fired during ``update_comics`` for a small set of comics in v1.11.0a0-era databases. **Code change.** ``CreateForeignKeyLinksImporter.get_comic_fk_links`` gains a ``for_create`` keyword. The update call site (``_update_comic_values``) passes ``False`` so ``_get_comic_folder_fk_link`` is skipped; the create call site (``_bulk_create_comic``) passes ``True`` and behavior is unchanged. The update path was unconditionally re-resolving the comic's parent folder via ``Folder.objects.get(path=str(Path(comic.path).parent))`` on every comic update — wasted work, and brittle against any string-form drift between ``Folder.path`` and the path computed from ``Comic.path``. ``MovedComicsImporter._prepare_moved_comic`` already sets ``comic.parent_folder_id`` directly during the move phase, and unchanged comics carry a valid FK from their prior import, so re-resolution was never necessary on the update path. **Migration step.** ``0039`` gains a ``RunPython`` step that runs ``fix_parent_folder_drift`` once at migrate time. The function detects ``Comic`` rows whose ``parent_folder_id`` points at a real ``Folder`` whose path no longer matches ``str(Path(comic.path).parent)``, re-points them at the matching ``Folder`` if one exists in the library, or warns about orphans (parent dir has no Folder row at all — the next import will recreate the hierarchy). No-op on a consistent database. The function lives in ``codex/librarian/scribe/janitor/integrity/foreign_keys.py`` next to ``fix_foreign_keys`` so it's importable both from the migration and from a Django shell if drift ever recurs. It accepts an optional ``apps_registry`` argument (defaults to the live registry) so the migration can pass its time-locked ``apps`` parameter. No nightly janitor task; the drift was observed once across years of operation in a single ~1 second event, so a recurring check would be permanent maintenance for a fixed-or-effectively-fixed bug. Same reasoning as the JSD dedupe path. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drop inode comparison from the database-vs-disk snapshot diff. On Docker bind mounts and similar filesystems, inodes can be reassigned across remounts even when file contents are unchanged, causing every comic in the library to be flagged as modified on startup. mtime+size is the authoritative change signal; inode is only useful as a hint for move detection within a single diff. - Remove _check_unchanged_for_inode_changes (inode-mismatch check) - Drop inode-equality requirement from _find_modified_paths - Drop the falsy-inode branch of the malformed-stat gate; len/None checks still force-modify genuinely malformed stored stats - _find_moved_paths still uses inode but now degrades to delete+add when inodes don't match anything Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The FTS sync strategy (delete-then-bulk_create over bulk_update) is already settled in production code; the supporting benchmark has served its purpose and its 29 lint errors block ``make fix`` repo-wide. Drop it and the two doc-comment pointers back to it. The rationale stays in the surrounding comments. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CoverThread only needs the first archive image as a thumbnail; it doesn't honor ComicInfo.xml's <PageType=\"FrontCover\"> hints. Reading the metadata on every comic to look for that hint dominates the cost of cover extraction and emits a flood of debug-bucket Marshmallow Union ValidationError lines that read like real failures at DEBUG. comicbox 3.0.0a2 added a `skip_metadata=True` flag to get_cover_page that bypasses the metadata path and reads archive index 0 directly. Pin to that release and pass the flag from the cover renderer. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bursts of CoverCreateTasks (e.g. parallel cover-endpoint cache misses) previously each opened and closed their own CreateCoversStatus, flickering the LibrarianStatus row off-and-on between tasks. Now the cover thread drains the queue's contiguous prefix of CoverCreateTasks into a single status lifecycle: total grows from 1 -> N as new tasks arrive, complete climbs to match, row clears once. Drain stops at the first non-create cover task to preserve enqueue order — the importer enqueues CoverRemoveTask before CoverCreateTask for updated comics, and _filter_pending_pks would silently no-op the regen if the create ran first. Also wraps future.result() in except CancelledError so pool shutdown during a burst doesn't crash the worker thread.
…ipy exclude (#694) * Reduce cognitive complexity of two burst/drift functions Both `CoverCreateThread.process_cover_create_burst` (cognitive 20) and `fix_parent_folder_drift` (cognitive 18, cyclomatic C/13) tripped complexipy's threshold. Split each into focused helpers without changing behavior: - `process_cover_create_burst` -> `_split_pending_pks`, `_render_burst_batch`, `_drain_burst_loop` (now cognitive 4). - `fix_parent_folder_drift` -> `_build_folder_lookups`, `_classify_comic_drift`, `_apply_parent_folder_repoints`, `_log_drift_orphans` (now cognitive 1). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix complexipy exclude pattern that hid every function `*/.*` matched too broadly under complexipy's path resolver and ended up excluding every Python file under the configured `paths`, so `make complexity` reported zero functions analyzed even when several exceeded the threshold. Switching to `.*` excludes hidden dotfiles/dirs without nuking the rest of the tree. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…d FTS dedupe (#693) Three review-driven cleanups to the 0039 migration so it's safer and faster on real-world installs: - ``_remove_non_comic_comics`` now refuses to delete unless it can stat-probe at least one comic-suffix path successfully, and the deletable queryset is restricted to ``page_count == 0`` rows (the phantom-row signature from the original importer bug). Together these mean a misconfigured Docker volume can no longer turn the cleanup into a mass-delete: a real comic with an unusual name has ``page_count >= 1`` and is never a candidate, and an unmounted filesystem trips the sentinel and skips the step entirely. - The composite ACL index ``codex_comic_lib_ari_idx`` is now built *after* ``_backfill_age_rating_metron_index`` so SQLite does one sorted index build instead of incrementally maintaining the index during the bulk UPDATE. - The FTS dedupe ``RunSQL`` (and its ``_FTS_DEDUPE_SQL`` constant) has been removed — it ran immediately before ``DROP TABLE codex_comicfts``, so it was dead code on this path. Also swap the ``print()`` calls in the cleanup function for the ``loguru.logger`` already imported at the top of the file. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ajslater
changed the title
…e SQL (#695) Two small startup hardening changes prompted by a debugging session that wasted ~90 minutes on a stale-WAL pollution. **Stale WAL/SHM warning.** When a backup is dropped on top of an existing ``codex.sqlite3`` (e.g., during alpha-version rollback testing), or a previous startup was killed before clean shutdown, the leftover ``-wal`` / ``-shm`` / ``-journal`` siblings get silently merged into the next open. The combined state is inconsistent and SQLite raises ``database disk image is malformed`` from inside Django's ``register_functions`` probe — before any of codex's own integrity checks get a chance to run, and without any log message that points at the real cause. Add ``_warn_on_stale_wal_siblings`` that fires before the first connection. Detects siblings whose mtime is *newer* than the main DB file (the signal that they're not just a normal in-flight WAL), logs a single WARNING that names the recovery command. We don't auto-delete: stale-looking siblings can also be a legitimate fast-restart WAL waiting to checkpoint, and silently nuking them would lose the last unckpointed transaction. **``_rebuild_db`` actually works now.** The drastic recovery path (gated on the ``rebuild_db`` sentinel file) piped ``sqlite3 .recover`` output through ``cursor.execute`` one line at a time. ``.recover`` happily emits multi-line ``CREATE TABLE`` / ``CREATE TRIGGER`` definitions, and the first such statement broke the rebuild with ``sqlite3.OperationalError: near ".": syntax error``. Read the recovery script as a single SQL blob and feed it to ``executescript`` instead. Also raise the subprocess timeout from 15s to 300s so a real-sized library actually finishes recovery, and use a ``with`` block so the subprocess always gets cleaned up on error. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
#696) The poller emits ``dirs_moved`` for inode-matched delete/add pairs. When the destination path of a "move" is already a known ``Folder`` row, ``_remove_move_collisions`` correctly drops the move (it would explode the unique constraint and isn't needed — the destination is already correctly tracked) and lets the source-side absence fall out via the next delete pass. This is normal reconciliation, not a fault. The previous WARNING made it look like something had gone wrong: ``` 2026-05-02 03:54:40 | WARNING | Not moving folders to destinations that would collide with existing database folders: ['…', '…', '…'] ``` …and prompted operators to investigate something that needed no investigating. Replace with: * one-line INFO: ``Resolved 3 phantom folder moves by skipping the rename and leaving the destinations in place.`` * DEBUG with the actual path list, for anyone who does want to see which folders were involved. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ajslater
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
configured.
multiprocessing & caching.
import.