[EXP] Update the benchmark code (#71)
* chore(hnsw): reorder imports to satisfy ruff I001 * chore: sync changes; fix Ruff import order; update examples, benchmarks, and dependencies - Fix import order in packages/leann-backend-hnsw/leann_backend_hnsw/hnsw_backend.py (Ruff I001) - Update benchmarks/run_evaluation.py - Update apps/base_rag_example.py and leann-core API usage - Add benchmarks/data/README.md - Update uv.lock - Misc cleanup - Note: added paru-bin as an embedded git repo; consider making it a submodule (git rm --cached paru-bin) if unintended * chore: remove unintended embedded repo paru-bin and ignore it Fix CI: avoid missing .gitmodules entry by removing gitlink and adding to .gitignore. * ci: retrigger after removing unintended gitlink (paru-bin) * feat(benchmarks): add --batch-size option and plumb through to HNSW search (default 0) * feat(hnsw): add batch_size to LeannSearcher.search and LeannChat.ask; forward only for HNSW backend * chore(logging): surface recompute and batching params; enable INFO logging in benchmark * feat(embeddings): add optional manual tokenization path (HF tokenizer+model) with mean pooling; default remains SentenceTransformer.encode * fix micro bench and fix pre commit * update readme --------- Co-authored-by: yichuan-w <yichuan-w@users.noreply.github.com>
This commit is contained in:
15
README.md
15
README.md
@@ -176,8 +176,7 @@ response = chat.ask("How much storage does LEANN save?", top_k=1)
|
||||
|
||||
LEANN supports RAG on various data sources including documents (`.pdf`, `.txt`, `.md`), Apple Mail, Google Search History, WeChat, and more.
|
||||
|
||||
**AST-Aware Code Chunking** - LEANN also features intelligent code chunking that preserves semantic boundaries (functions, classes, methods) for Python, Java, C#, and TypeScript files, providing improved code understanding compared to traditional text-based approaches.
|
||||
📖 Read the [AST Chunking Guide →](docs/ast_chunking_guide.md) to learn more.
|
||||
|
||||
|
||||
### Generation Model Setup
|
||||
|
||||
@@ -477,6 +476,15 @@ Once the index is built, you can ask questions like:
|
||||
|
||||
### 🚀 Claude Code Integration: Transform Your Development Workflow!
|
||||
|
||||
<details>
|
||||
<summary><strong>NEW!! AST‑Aware Code Chunking</strong></summary>
|
||||
|
||||
LEANN features intelligent code chunking that preserves semantic boundaries (functions, classes, methods) for Python, Java, C#, and TypeScript, improving code understanding compared to text-based chunking.
|
||||
|
||||
📖 Read the [AST Chunking Guide →](docs/ast_chunking_guide.md)
|
||||
|
||||
</details>
|
||||
|
||||
**The future of code assistance is here.** Transform your development workflow with LEANN's native MCP integration for Claude Code. Index your entire codebase and get intelligent code assistance directly in your IDE.
|
||||
|
||||
**Key features:**
|
||||
@@ -697,6 +705,9 @@ MIT License - see [LICENSE](LICENSE) for details.
|
||||
|
||||
Core Contributors: [Yichuan Wang](https://yichuan-w.github.io/) & [Zhifei Li](https://github.com/andylizf).
|
||||
|
||||
Active Contributors: [Gabriel Dehan](https://github.com/gabriel-dehan)
|
||||
|
||||
|
||||
We welcome more contributors! Feel free to open issues or submit PRs.
|
||||
|
||||
This work is done at [**Berkeley Sky Computing Lab**](https://sky.cs.berkeley.edu/).
|
||||
|
||||
Reference in New Issue
Block a user