Saving and loading in-memory btrees to disk

[backurs]

This PR allows to save and load in-memory (:memory:) indices. Previously, it was possible to save/load only on-disk indices.

It also adds is_memory variable to SavedParams struct so that we know whether the original bf-tree index was in-memory or on-disk.

We also add two tests on whether in-memory saving/loading works both in pq and non-pq settings.

[Copilot]

Pull request overview

This PR extends BfTreeProvider persistence to support saving and loading indices that were originally backed by in-memory (:memory:) BfTrees, including both non-quantized and PQ-quantized variants.

Changes:

• Added snapshot_to_disk(...) helpers on vector/quant/neighbor providers to persist in-memory BfTrees to disk.
• Extended persisted SavedParams with an is_memory flag and updated load/save flows to round-trip in-memory indices.
• Added async tests covering save/load for in-memory indices with and without PQ quantization.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
diskann-providers/src/model/graph/provider/async_/bf_tree/vector_provider.rs	Adds a convenience method to snapshot in-memory vector BfTree to disk.
diskann-providers/src/model/graph/provider/async_/bf_tree/quant_vector_provider.rs	Adds a convenience method to snapshot in-memory quant-vector BfTree to disk.
diskann-providers/src/model/graph/provider/async_/bf_tree/neighbor_provider.rs	Adds a convenience method to snapshot in-memory neighbor-list BfTree to disk.
diskann-providers/src/model/graph/provider/async_/bf_tree/provider.rs	Persists is_memory, adjusts save/load logic for in-memory backends, and adds new tests.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

[metajack]

This looks fine to me. The repeated pattern of loading and saving all the various providers could maybe be factored into two helper functions? I'm not sure how hard that is, so feel free to do what is reasonable.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.96%. Comparing base (4db2797) to head (56ec094).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #820      +/-   ##
==========================================
- Coverage   90.64%   88.96%   -1.69%     
==========================================
  Files         432      442      +10     
  Lines       79629    81906    +2277     
==========================================
+ Hits        72182    72870     +688     
- Misses       7447     9036    +1589     

Flag	Coverage Δ
miri	88.96% <ø> (-1.69%)	⬇️
unittests	88.82% <ø> (-1.80%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 52 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[backurs]

This looks fine to me. The repeated pattern of loading and saving all the various providers could maybe be factored into two helper functions? I'm not sure how hard that is, so feel free to do what is reasonable.

Good suggestion. For loading I created the following function:

fn load_bftree(
    params: &BfTreeParams,
    snapshot_path: std::path::PathBuf,
    is_memory: bool,
) -> Result<BfTree, ANNError> {
    let config = params.to_config(&snapshot_path, is_memory);
    if is_memory {
        BfTree::new_from_snapshot_disk_to_memory(snapshot_path, config)
            .map_err(|e| ANNError::from(super::ConfigError(e)))
    } else {
        BfTree::new_from_snapshot(config, None)
            .map_err(|e| ANNError::from(super::ConfigError(e)))
    }
}

and I'm calling it in 5 places.

For saving I was able to refactor the code with help of traits but the resulting code was complicated and longer than the original code. So, for saving, perhaps we can leave the current solution.