Compare commits

..

21 Commits

Author SHA1 Message Date
yichuan520030910320
dc4987591b skip several macos test because strange issue on ci 2025-07-28 16:47:18 -07:00
Andy Lee
d8b6ae8d1a fix: disable OpenMP parallelism in CI to avoid libomp crashes
- Set OMP_NUM_THREADS=1 to avoid OpenMP thread synchronization issues
- Set MKL_NUM_THREADS=1 for single-threaded MKL operations
- This prevents segfaults in LayerNorm on macOS CI runners
- Addresses the libomp compatibility issues with PyTorch on Apple Silicon
2025-07-28 16:31:41 -07:00
Andy Lee
f2ffcf5665 fix: use --find-links to install platform-specific wheels
- Let uv automatically select the correct wheel for the current platform
- Fixes error when trying to install macOS wheels on Linux
- Simplifies the installation logic
2025-07-28 16:15:28 -07:00
yichuan520030910320
27d0d73f99 add some env in ci 2025-07-28 16:11:44 -07:00
Andy Lee
b124709bcd fix: use virtual environment in CI instead of system packages
- uv-managed Python environments don't allow --system installs
- Create and activate virtual environment before installing packages
- Update all CI steps to use the virtual environment
2025-07-28 16:04:49 -07:00
Andy Lee
78251a6d4c fix: remove Python 3.10+ dependencies for compatibility
- Comment out llama-index-readers-docling and llama-index-node-parser-docling
- These packages require Python >= 3.10 and were causing CI failures on Python 3.9
- Regenerate uv.lock file to resolve dependency conflicts
2025-07-28 15:50:05 -07:00
Andy Lee
16c833da86 fix: handle MPS memory issues in CI tests
- Use smaller MiniLM-L6-v2 model (384 dimensions) for README tests in CI
- Skip other memory-intensive tests in CI environment
- Add minimal CI tests that don't require model loading
- Set CI environment variable and disable MPS fallback
- Ensure README examples always run correctly in CI
2025-07-28 15:26:23 -07:00
Andy Lee
c246cb4a01 fix: align Python version requirements to 3.9
- Update root project to support Python 3.9, matching subpackages
- Restore macOS Python 3.9 support in CI
- This fixes the CI failure for Python 3.9 environments
2025-07-28 15:09:59 -07:00
Andy Lee
0f34aee5db fix: update macOS deployment target for DiskANN to 13.3
- DiskANN uses sgesdd_ LAPACK function which is only available on macOS 13.3+
- Update MACOSX_DEPLOYMENT_TARGET from 11.0 to 13.3 for DiskANN builds
- This fixes the compilation error on GitHub Actions macOS runners
2025-07-28 15:00:50 -07:00
Andy Lee
3e53d3d264 docs: remove obsolete C++ ABI compatibility warnings
- Remove outdated macOS C++ compatibility warnings from README
- Simplify CI workflow by removing macOS-specific failure handling
- All tests now pass consistently on macOS after ABI fixes
2025-07-28 14:54:47 -07:00
Andy Lee
22c8f861bc Merge branch 'main' into fix-macos-abi 2025-07-28 14:52:15 -07:00
Andy Lee
a52e3c583a chore: update lock file with test dependencies 2025-07-28 14:50:21 -07:00
Andy Lee
ab339886dd fix: add --distance-metric support to DiskANN embedding server and remove obsolete macOS ABI test markers
- Add --distance-metric parameter to diskann_embedding_server.py for consistency with other backends
- Remove pytest.skip and pytest.xfail markers for macOS C++ ABI issues as they have been fixed
- Fix test assertions to handle SearchResult objects correctly
- All tests now pass on macOS with the C++ ABI compatibility fixes
2025-07-28 14:49:51 -07:00
Andy Lee
8c988cf98b refactor: improve test structure and fix main_cli example
- Move pytest configuration from pytest.ini to pyproject.toml
- Remove unnecessary run_tests.py script (use test extras instead)
- Fix main_cli_example.py to properly use command line arguments for LLM config
- Add test_readme_examples.py to test code examples from README
- Refactor tests to use pytest fixtures and parametrization
- Update test documentation to reflect new structure
- Set proper environment variables in CI for test execution
2025-07-28 14:25:48 -07:00
Andy Lee
ac5fd844a5 fix: improve macOS C++ compatibility and add CI tests 2025-07-28 14:01:52 -07:00
Andy Lee
4b4b825fec Merge remote-tracking branch 'origin/main' into fix/openai-embeddings-cosine-distance 2025-07-28 10:17:55 -07:00
Andy Lee
34ef0db42f fix: Improve OpenAI embeddings handling in HNSW backend 2025-07-28 10:15:56 -07:00
Andy Lee
41812c7d22 feat: add --use-existing-index option to google_history_reader_leann.py
- Allow using existing index without rebuilding
- Useful for testing pre-built indices
2025-07-28 00:36:57 -07:00
Andy Lee
2047a1a128 feat: add OpenAI embeddings support to google_history_reader_leann.py
- Add --embedding-model and --embedding-mode arguments
- Support automatic detection of normalized embeddings
- Works correctly with cosine distance for OpenAI embeddings
2025-07-27 23:10:20 -07:00
Andy Lee
402e8f97ad style: format 2025-07-27 20:25:40 -07:00
Andy Lee
9a5c197acd fix: auto-detect normalized embeddings and use cosine distance
- Add automatic detection for normalized embedding models (OpenAI, Voyage AI, Cohere)
- Automatically set distance_metric='cosine' for normalized embeddings
- Add warnings when using non-optimal distance metrics
- Implement manual L2 normalization in HNSW backend (custom Faiss build lacks normalize_L2)
- Fix DiskANN zmq_port compatibility with lazy loading strategy
- Add documentation for normalized embeddings feature

This fixes the low accuracy issue when using OpenAI text-embedding-3-small model with default MIPS metric.
2025-07-27 20:21:05 -07:00
4 changed files with 6 additions and 6 deletions

View File

@@ -4,8 +4,8 @@ build-backend = "scikit_build_core.build"
[project]
name = "leann-backend-diskann"
version = "0.1.16"
dependencies = ["leann-core==0.1.16", "numpy", "protobuf>=3.19.0"]
version = "0.1.15"
dependencies = ["leann-core==0.1.15", "numpy", "protobuf>=3.19.0"]
[tool.scikit-build]
# Key: simplified CMake path

View File

@@ -6,10 +6,10 @@ build-backend = "scikit_build_core.build"
[project]
name = "leann-backend-hnsw"
version = "0.1.16"
version = "0.1.15"
description = "Custom-built HNSW (Faiss) backend for the Leann toolkit."
dependencies = [
"leann-core==0.1.16",
"leann-core==0.1.15",
"numpy",
"pyzmq>=23.0.0",
"msgpack>=1.0.0",

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "leann-core"
version = "0.1.16"
version = "0.1.15"
description = "Core API and plugin system for LEANN"
readme = "README.md"
requires-python = ">=3.9"

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "leann"
version = "0.1.16"
version = "0.1.15"
description = "LEANN - The smallest vector index in the world. RAG Everything with LEANN!"
readme = "README.md"
requires-python = ">=3.9"