Skip to content

Sketch out quantization tasks and calls into DiskANN#1726

Draft
kevin-montrose wants to merge 22 commits into
release/v1from
users/kmontrose/vectorSetQuantization
Draft

Sketch out quantization tasks and calls into DiskANN#1726
kevin-montrose wants to merge 22 commits into
release/v1from
users/kmontrose/vectorSetQuantization

Conversation

@kevin-montrose
Copy link
Copy Markdown
Contributor

@kevin-montrose kevin-montrose commented Apr 22, 2026

Change expected return from DiskANN's insert so we can signal a request to quantize.

Spin up tasks to pump background quantization when that is requested.

Tasks take two forms:

  • A single task per Vector Set index which calls build_quant_table(context, index)
  • After that finishes and true is returned, some number that proceed in parallel and call backfill_quant_vectors(context, index, task_index, task_count)

It's on DiskANN to find vectors to backfill and split them up based on task_index / task_count.

Since locks are not held over the entire backfill, DiskANN also needs to validate that quantization is still needed when build_quant_table is called (recreation and restarts can mess with that) and backfill_quant_vectors needs to gracefully deal with concurrent calls.


Upon further discussion, we're also cleaning up quantizer options and the FFI between Garnet and DiskANN around vector formats.

Namely:

  • Rename quantizer XPREQ8 -> XNOQUANT_U8
  • Add XBIN_I8
  • Rename format XB8 -> XU8
  • Add format XI8
  • Garnet now knows the "natural" format of different quantizers and only passes vectors of those formats to DiskANN in insert & search calls

Old options are aliased so we can keep XPREQ8 and XB8 in tests and whatnot without issue.

@kevin-montrose kevin-montrose changed the base branch from main to release/v1 May 12, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant