add mac support in this repo
This commit is contained in:
1
.gitignore
vendored
1
.gitignore
vendored
@@ -8,6 +8,7 @@ demo/indices/
|
|||||||
*pycache*
|
*pycache*
|
||||||
outputs/
|
outputs/
|
||||||
*.pkl
|
*.pkl
|
||||||
|
*.pdf
|
||||||
.history/
|
.history/
|
||||||
scripts/
|
scripts/
|
||||||
lm_eval.egg-info/
|
lm_eval.egg-info/
|
||||||
|
|||||||
23
README.md
23
README.md
@@ -85,6 +85,27 @@ This demo showcases how to build a RAG system for PDF documents using Leann.
|
|||||||
uv run examples/main_cli_example.py
|
uv run examples/main_cli_example.py
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## ⚙️ Developer Build Instructions (macOS/Linux)
|
||||||
|
|
||||||
|
If you are building or modifying the C++ backends (e.g., DiskANN, HNSW), please ensure the following dependencies are installed:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
brew install boost protobuf zeromq
|
||||||
|
```
|
||||||
|
|
||||||
|
> On Linux, use your package manager (e.g., `apt install libboost-all-dev protobuf-compiler libprotobuf-dev libzmq3-dev`).
|
||||||
|
|
||||||
|
### Regenerating Protobuf Files
|
||||||
|
If you modify any `.proto` files (such as `embedding.proto`), or if you see errors about protobuf version mismatch, **regenerate the C++ protobuf files** to match your installed version:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From the leann/packages/leann-backend-diskann directory:
|
||||||
|
protoc --cpp_out=third_party/DiskANN/include --proto_path=third_party embedding.proto
|
||||||
|
protoc --cpp_out=third_party/DiskANN/src --proto_path=third_party embedding.proto
|
||||||
|
```
|
||||||
|
|
||||||
|
This ensures the generated files are compatible with your system's protobuf library.
|
||||||
|
|
||||||
## ✨ Features
|
## ✨ Features
|
||||||
|
|
||||||
### 🔥 Core Features
|
### 🔥 Core Features
|
||||||
@@ -259,7 +280,7 @@ export NCCL_DEBUG_SUBSYS=INIT,GRAPH
|
|||||||
export NCCL_IB_DISABLE=1
|
export NCCL_IB_DISABLE=1
|
||||||
export NCCL_NET_PLUGIN=none
|
export NCCL_NET_PLUGIN=none
|
||||||
export NCCL_SOCKET_IFNAME=ens5
|
export NCCL_SOCKET_IFNAME=ens5
|
||||||
|
```
|
||||||
|
|
||||||
## 📈 Roadmap
|
## 📈 Roadmap
|
||||||
|
|
||||||
|
|||||||
Binary file not shown.
@@ -25,7 +25,7 @@ file_extractor: dict[str, BaseReader] = {
|
|||||||
".md": reader,
|
".md": reader,
|
||||||
}
|
}
|
||||||
node_parser = DoclingNodeParser(
|
node_parser = DoclingNodeParser(
|
||||||
chunker=HybridChunker(tokenizer="Qwen/Qwen3-Embedding-4B", max_tokens=128)
|
chunker=HybridChunker(tokenizer="facebook/contriever", max_tokens=128)
|
||||||
)
|
)
|
||||||
print("Loading documents...")
|
print("Loading documents...")
|
||||||
documents = SimpleDirectoryReader(
|
documents = SimpleDirectoryReader(
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ dependencies = [
|
|||||||
"colorama",
|
"colorama",
|
||||||
"boto3",
|
"boto3",
|
||||||
"protobuf==4.25.3",
|
"protobuf==4.25.3",
|
||||||
"sglang[all]",
|
"sglang",
|
||||||
"ollama",
|
"ollama",
|
||||||
"requests>=2.25.0",
|
"requests>=2.25.0",
|
||||||
"sentence-transformers>=2.2.0",
|
"sentence-transformers>=2.2.0",
|
||||||
|
|||||||
Reference in New Issue
Block a user