docs: consolidate low-resource guidance into config guide; README points to it

This commit is contained in:
Andy Lee
2025-08-13 14:08:23 -07:00
parent 58711bff7e
commit 07e5f10204
2 changed files with 51 additions and 4 deletions

View File

@@ -71,10 +71,7 @@ source .venv/bin/activate
uv pip install leann
```
> Low-resource setup: No local GPU? That's fine.
> - Use OpenAI embeddings (`--embedding-mode openai --embedding-model text-embedding-3-small`) for fast indexing without local compute.
> - Or run remote builds with SkyPilot using the template at `sky/leann-build.yaml` (see Configuration Guide for details).
> - If build/search is slow, consider these two options or disable recomputation with `--no-recompute` (trade storage for speed).
> Low-resource? See “Low-resource setups” in the [Configuration Guide](docs/configuration-guide.md#low-resource-setups).
<details>
<summary>

View File

@@ -278,6 +278,56 @@ LEANN's recomputation feature provides exact distance calculations but can be di
- Need extremely low latency (< 100ms)
- Running a read-heavy workload where storage cost is acceptable
## Low-resource setups
If you dont have a local GPU or builds/searches are too slow, use one or more of the options below.
### 1) Use OpenAI embeddings (no local compute)
Fastest path with zero local GPU requirements. Set your API key and use OpenAI embeddings during build and search:
```bash
export OPENAI_API_KEY=sk-...
# Build with OpenAI embeddings
leann build my-index \
--embedding-mode openai \
--embedding-model text-embedding-3-small
# Search with OpenAI embeddings (recompute at query time)
leann search my-index "your query" \
--recompute-embeddings
```
### 2) Run remote builds with SkyPilot (cloud GPU)
Offload embedding generation and index building to a GPU VM using SkyPilot. A template is provided at `sky/leann-build.yaml`.
```bash
# One-time: install and configure SkyPilot
pip install skypilot
sky launch -c leann-gpu sky/leann-build.yaml
# Build remotely (template installs uv + leann CLI)
sky exec leann-gpu -- "leann build my-index --docs ~/leann-data --backend hnsw --complexity 64 --graph-degree 32"
```
Details: see “Running Builds on SkyPilot (Optional)” below.
### 3) Disable recomputation to trade storage for speed
If you need lower latency and have more storage/memory, disable recomputation. This stores full embeddings and avoids recomputing at search time.
```bash
# Build without recomputation
leann build my-index --no-recompute
# Search without recomputation
leann search my-index "your query" --no-recompute
```
Trade-offs: lower query-time latency, but significantly higher storage usage.
## Running Builds on SkyPilot (Optional)
You can offload embedding generation and index building to a cloud GPU VM using SkyPilot, without changing any LEANN code. This is useful when your local machine lacks a GPU or you want faster throughput.