crate publishing workflow

.github/PUBLISH_CRATES.md

@@ -0,0 +1,111 @@
+# Release Process
+
+Publishing DiskANN crates to [crates.io](https://crates.io).
+
+## Overview
+
+All workspace crates are published together with synchronized version numbers using `cargo publish --workspace`, which automatically resolves dependency order and waits for each crate to be indexed before publishing its dependents. The Rust toolchain version is read from [`rust-toolchain.toml`](../../rust-toolchain.toml).
+
+Releases follow a pull-request workflow: bump the version on a branch, open a PR, let the dry-run check pass, merge, then tag the release via the GitHub UI.
+
+## Prerequisites
+
+1. **CRATES_IO_TOKEN Secret**: A crates.io API token configured as a GitHub repository secret named `CRATES_IO_TOKEN` with publish permissions for all DiskANN crates.
+   - Create a token: [crates.io/settings/tokens](https://crates.io/settings/tokens)
+   - Add the secret: Repository Settings → Secrets and variables → Actions → New repository secret
+
+2. **Maintainer Access**: Write access to the repository and owner/maintainer of all crates on crates.io.
+
+## Dry-Run Testing
+
+A `cargo publish --workspace --dry-run` runs **automatically** as a pull-request check whenever `Cargo.toml` is changed. You can also trigger a dry-run manually:
+
+### Manual: GitHub Actions
+
+1. Navigate to: `https://github.com/microsoft/DiskANN/actions/workflows/publish.yml`
+2. Click **Run workflow**, select your branch, keep **dry-run = true**
+3. Watch the workflow — look for successful `cargo publish --workspace --dry-run`
+
+### Manual: Local
+
+```bash
+cargo publish --locked --workspace --dry-run
+```
+
+### What Dry-Run Tests
+
+- Crate metadata and packaging validation
+- Dependency resolution and publish ordering
+- Build verification
+
+### What It Does NOT Test
+
+- Actual publishing, registry token auth, upload reliability
+
+## Release Steps
+
+1. **Create a release branch** from `main`:
+
+   ```bash
+   git checkout main && git pull
+   git checkout -b release-0.46.0
+   ```
+
+2. **Update version** in root `Cargo.toml`:
+
+   - Set `workspace.package.version`:
+
+     ```toml
+     [workspace.package]
+     version = "0.46.0"
+     ```
+
+   - Update **all internal crate entries** under `[workspace.dependencies]` to match:
+
+     ```toml
+     diskann-wide = { path = "diskann-wide", version = "0.46.0" }
+     diskann-vector = { path = "diskann-vector", version = "0.46.0" }
+     # ... etc
+     ```
+
+   Member crates inherit `workspace.package.version` via `version.workspace = true`,
+   but `[workspace.dependencies]` versions must be set explicitly (they're baked into
+   published manifests for crates.io consumers). 
+
+3. **Update CHANGELOG** (if applicable).
+
+4. **Push and open a pull request** to `main`:
+
+   ```bash
+   git commit -am "Bump version to 0.46.0"
+   git push origin release-0.46.0
+   ```
+
+   Open a PR on GitHub. The **Publish to crates.io / Dry-run publish test** check runs automatically.
+
+5. **Wait for checks** — the dry-run and CI must both pass before merge.
+
+6. **Merge the PR** into `main`.
+
+7. **Create a release** via the GitHub UI:
+   - Go to **Releases → Draft a new release**
+   - Create a new tag `v0.46.0` targeting `main`
+   - Add release notes describing changes
+   - Click **Publish release**
+
+   Pushing the tag triggers the real publish workflow.
+
+8. **Verify** the published crates — confirm the new version appears in the output:
+
+   ```bash
+   cargo search diskann --limit 20
+   ```
+
+## Pre-release Checklist
+
+- [ ] All CI checks pass on the PR
+- [ ] Version number is updated in `Cargo.toml`
+- [ ] CHANGELOG is updated (if applicable)
+- [ ] Documentation is up to date
+- [ ] Breaking changes are clearly documented
+- [ ] **Dry-run publish check passes on the PR**

.github/workflows/publish.yml

@@ -0,0 +1,123 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT license.
+
+# Publishes all workspace crates to crates.io.
+#
+# Triggers:
+#   - push tag v{major}.{minor}.{patch} → real publish
+#   - pull_request touching Cargo.toml   → automatic dry-run (pre-merge check)
+#   - workflow_dispatch                  → manual dry-run or real publish
+#
+# Requires CRATES_IO_TOKEN secret. Rust toolchain version is read from rust-toolchain.toml.
+
+name: Publish to crates.io
+
+on:
+  push:
+    tags:
+      - 'v[0-9]+.[0-9]+.[0-9]+'
+  pull_request:
+    branches: ["main"]
+    paths:
+      - 'Cargo.toml'
+  workflow_dispatch:
+    inputs:
+      dry_run:
+        description: 'Run in dry-run mode (test without actually publishing)'
+        required: false
+        default: 'true'
+        type: choice
+        options:
+          - 'true'
+          - 'false'
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: true
+
+env:
+  RUST_BACKTRACE: 1
+
+defaults:
+  run:
+    shell: bash
+
+permissions:
+  contents: read
+
+jobs:
+  publish:
+    name: >-
+      ${{
+        (github.event_name == 'pull_request' || github.event.inputs.dry_run == 'true')
+          && 'Dry-run publish test'
+          || 'Publish crates to crates.io'
+      }}
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          lfs: true
+
+      - name: Read Rust version from rust-toolchain.toml
+        id: rust-version
+        run: |
+          RUST_VERSION=$(sed -n 's/^channel = "\(.*\)"/\1/p' rust-toolchain.toml)
+          echo "channel=$RUST_VERSION" >> "$GITHUB_OUTPUT"
+
+      - name: Install Rust ${{ steps.rust-version.outputs.channel }}
+        uses: dtolnay/rust-toolchain@master
+        with:
+          toolchain: ${{ steps.rust-version.outputs.channel }}
+
+      - uses: Swatinem/rust-cache@v2
+
+      - name: Prevent publish from non-main branch
+        if: >-
+          github.event_name == 'workflow_dispatch'
+          && github.event.inputs.dry_run != 'true'
+          && github.ref != 'refs/heads/main'
+        run: |
+          echo "::error::Live publishing is only allowed from main. Use dry-run for other branches."
+          exit 1
+
+      - name: Verify version matches tag
+        if: github.event_name == 'push'
+        run: |
+          TAG_VERSION="${GITHUB_REF#refs/tags/v}"
+          CARGO_VERSION=$(grep -A 5 '^\[workspace\.package\]' Cargo.toml | grep 'version = ' | head -n1 | sed 's/.*"\(.*\)".*/\1/')
+          if [ "$TAG_VERSION" != "$CARGO_VERSION" ]; then
+            echo "::error::Tag version ($TAG_VERSION) does not match Cargo.toml version ($CARGO_VERSION)"
+            exit 1
+          fi
+
+      - name: Verify all crates use workspace version
+        run: |
+          bad_crates=()
+          for manifest in $(cargo metadata --no-deps --format-version 1 | jq -r '.packages[].manifest_path'); do
+            dir=$(dirname "$manifest")
+            name=$(basename "$dir")
+            if [ "$manifest" != "$(pwd)/Cargo.toml" ] && ! grep -qE 'version\s*=\s*\{\s*workspace\s*=\s*true\s*\}|version\.workspace\s*=\s*true' "$manifest"; then
+              bad_crates+=("$name")
+            fi
+          done
+          if [ ${#bad_crates[@]} -gt 0 ]; then
+            echo "::error::The following crates do not use version.workspace = true: ${bad_crates[*]}"
+            exit 1
+          fi
+
+      - name: Publish workspace crates
+        env:
+          CARGO_REGISTRY_TOKEN: ${{ secrets.CRATES_IO_TOKEN }}
+          DRY_RUN: ${{ github.event.inputs.dry_run || 'false' }}
+        run: |
+          DRY_RUN_FLAG=""
+          if [ "${{ github.event_name }}" = "pull_request" ] || [ "$DRY_RUN" = "true" ]; then
+            DRY_RUN_FLAG="--dry-run"
+            echo "🧪 DRY-RUN MODE"
+          else
+            echo "📦 LIVE MODE - Publishing to crates.io"
+          fi
+          cargo publish --locked --workspace $DRY_RUN_FLAG