- Add explicit Python finding logic to DiskANN CMakeLists.txt
- Change Development to Development.Module to avoid Embed requirement
- Pass Python variables through CMake cache to submodules
- Remove macOS from build matrix to simplify CI
- Keep macOS configurations for future reference but they won't be used
- This speeds up CI and focuses on the primary target (Colab/Linux)
- Cap pyzmq version to <27 for manylinux2014 compatibility
- Pre-install pyzmq binary wheel before tests using CIBW_BEFORE_TEST
- Force pip to use only binary wheels with --only-binary :all:
- Set MACOSX_DEPLOYMENT_TARGET=11.0 for macOS builds
- Add pyzmq to test requirements to use pre-built wheels
- Configure deployment target in both workflow and pyproject.toml
- Skip ARM64 tests on GitHub Actions to avoid cross-compilation issues
- Use Development.Module instead of Development component
- Find Python in main Faiss CMakeLists.txt before python subdirectory
- Add debug output to trace Python variable passing
- Set Python_FIND_VIRTUALENV=ONLY for Faiss
- Re-enable FAISS_ENABLE_PYTHON since we need the Python bindings
- Use Development.Module component for better compatibility
- Pass Python information to Faiss through CMake cache variables
- Add CMAKE_PREFIX_PATH= to help CMake find Python in manylinux
- Set FAISS_ENABLE_PYTHON to OFF since we use our own Cython bindings
- This avoids the CMake Python finding issues in manylinux environments
- Simplify CMakeLists.txt by removing unnecessary Python finding logic
- Keep swig installation for other potential uses
- Modify faiss CMakeLists.txt to try both FindPython and FindPython3
- Add scikit-build configuration to help with Python detection
- Simplify Linux environment variables in cibuildwheel
- Add CMake Python detection before faiss configuration
- Add libomp for macOS to fix OpenMP linking error
- Add python3-devel for Linux (though may not be needed in manylinux)
- Install numpy before building to satisfy faiss CMake requirements
- Add before-build configuration to cibuildwheel
- Switch from manylinux_2_28 to manylinux2014 (provides manylinux_2_17)
- This should produce wheels compatible with manylinux_2_35_x86_64 requirement
- Update package manager from dnf to yum for CentOS 7
- Use cmake3 with symlink for compatibility
- Remove Windows support as not needed
- Move Python finding hints to cibuildwheel environment variables
- Keep pyproject.toml clean to avoid breaking normal builds
- Target manylinux_2_28 for better Colab compatibility
- Add Python_FIND_VIRTUALENV hints to pyproject.toml for CMake
- Create standardized cibuildwheel workflow using manylinux_2_28
- Simplify system dependency installation
- Add global cibuildwheel configuration in root pyproject.toml
- Create streamlined test workflow for manylinux compatibility
When embedding server is launched as a subprocess, it may run in a different
working directory. Using absolute paths ensures the server can always find
the metadata file regardless of where it's launched from.
- Add gcc-c++ and cmake to dependencies
- Create libzmq.pc file if missing (CentOS 7 issue)
- Set PKG_CONFIG_PATH through CIBW_ENVIRONMENT_LINUX
- Add protobuf-devel to ensure all headers are available
- Fix shell variable escaping in heredoc
- Add yum cache cleaning and updating
- Make package installations more resilient with fallbacks
- Use pkgconfig instead of pkg-config (CentOS 7 naming)
- Handle optional packages that might not be available
- Add error handling for package installation failures
- Build pure Python packages (leann-core, leann) with standard build tool
- Only use cibuildwheel for C extension packages (leann-backend-hnsw, leann-backend-diskann)
- Build pure Python packages only once on ubuntu-latest
- Add Python setup for building pure packages
- Add package listing step for debugging
- Add multiple safe.directory configurations to cover different possible paths
- This fixes 'detected dubious ownership in repository' error
- Ensures git works properly in manylinux2014 containers
- Upgrade all GitHub Actions to v4 (v3 is deprecated)
- Use manual git checkout in manylinux2014 containers to avoid Node.js issues
- Update artifact naming to ensure uniqueness (required by v4)
- Add fail-fast: false to build strategies
- This maintains manylinux2014 compatibility while using latest actions
- Add optional use_cibuildwheel parameter to release workflow
- Create separate CI workflow for testing cibuildwheel
- Support conditional build workflow selection in release process
- This allows building wheels compatible with Google Colab and older systems
- Maintains backward compatibility with existing build process
- Use cibuildwheel for professional wheel building
- Specifically use manylinux2014 for Google Colab compatibility
- Supports Python 3.9-3.12 on Linux and macOS
- Handles monorepo structure with separate builds per package
- Includes basic import tests for each package
- This should resolve compatibility issues with older systems like Google Colab
- Use actions/checkout@v3 instead of v4 (Node.js 16 vs 20)
- Use actions/setup-python@v4 instead of v5
- Use actions/upload-artifact@v3 and download-artifact@v3
- This fixes GLIBC version errors in manylinux2014 containers
- manylinux2014 (CentOS 7) has glibc 2.17 but Node.js 20 needs 2.25+
- Add manylinux2014 Docker containers for Linux builds
- This will generate wheels compatible with older Linux systems (CentOS 7+, Ubuntu 16.04+)
- Separate build logic for container vs regular environments
- Install appropriate system dependencies for yum-based manylinux environment
- Use pip instead of uv in containers for better compatibility
- Fix Python version format for manylinux container paths
- Remove --plat manylinux2014_x86_64 flag that was causing build failures
- Let auditwheel automatically determine the appropriate manylinux tag
- Add auditwheel show command to display compatibility info
- This fixes the 'too-recent versioned symbols' error
- Change auditwheel --plat to manylinux2014_x86_64
- This ensures wheels work on Ubuntu 16.04+ instead of requiring 24.04+
- Fixes compatibility issues for users on Ubuntu 22.04 and similar systems
- Enhanced bump_version.sh to automatically update leann-core dependency versions
- Script now updates both package versions and their leann-core dependencies
- This ensures version consistency across all packages during release
No more manual dependency version updates needed!
- Fix version dependencies: update backend packages to depend on leann-core==0.1.1
- Remove duplicate ci.yml workflow (keeping build-and-publish.yml as main CI)
- Update release-manual.yml to reference correct CI workflow name
This fixes the dependency resolution error and eliminates duplicate builds.
- Manual Release workflow now directly publishes to PyPI after downloading CI artifacts
- No more duplicate builds - reuses artifacts from CI
- build-and-publish.yml renamed to 'CI - Build Multi-Platform Packages'
- Publishing in CI workflow only for emergency manual triggers
- Updated RELEASE.md to reflect the new streamlined process
This fixes the issue where releases would trigger redundant builds.
- Manual Release workflow creates tags but build-and-publish.yml only published on 'release' events
- Now build-and-publish.yml will also publish when v* tags are pushed
- This fixes the issue where manual releases didn't trigger PyPI uploads
- Add continue-on-error to TestPyPI step
- Check if TEST_PYPI_API_TOKEN exists before attempting upload
- Add graceful failure handling with clear messages
- Update docs to explain TestPyPI token configuration
- Clarify that TestPyPI testing is optional
Now the release won't fail if TestPyPI is not configured or upload fails
- Add 'actions: read' permission to access workflow runs
- Use workflow name instead of filename for gh run list
- Look for CI run on HEAD~1 (before version bump commit)
- Improve error messages for better debugging
Fixes HTTP 403 error when trying to find successful CI runs
- Download pre-built wheels from successful CI runs
- Avoids duplicate builds and ensures consistency
- CI artifacts are already tested across all platforms
- Faster release process (no build time)
- Updates release documentation to reflect new flow
This ensures the released packages are exactly what was tested in CI.
- Remove scripts/ from .gitignore
- Add build_and_test.sh for local testing
- Add bump_version.sh for version updates (used by CI)
- Add release.sh and upload_to_pypi.sh for publishing
- Fixes CI error: ./scripts/bump_version.sh: No such file or directory
- Add --python python flag to uv build commands
- This ensures wheels are built with the correct Python version (cp313 for Python 3.13, etc)
- Fixes issue where Python 3.13 CI was building cp311 wheels
- Also adds Python version verification before build
- Update actions/upload-artifact from v3 to v4 (v3 deprecated April 2024)
- Update actions/setup-python from v4 to v5 (latest version)
- Add Python 3.12 and 3.13 to CI test matrix
- Ensure compatibility with latest Python versions and GitHub Actions
- Add ci.yml for continuous integration on every commit
- Test builds on Ubuntu/macOS with Python 3.9/3.10/3.11
- Ensure code quality before any release
- Add release-manual.yml for controlled releases
- Manual trigger prevents accidental releases
- Version validation and tag creation
- Optional TestPyPI testing before production
- Only creates tag after validation passes
- Keep build-and-publish.yml for automated PyPI deployment
- Triggered by new tags (separation of concerns)
- Handles multi-platform wheel building
- Allows retry if PyPI upload fails
- Update RELEASE.md with clear prerequisites and workflow
This setup ensures:
1. Every commit is tested (CI)
2. Releases are deliberate (manual trigger)
3. Failed CI won't create broken tags
4. PyPI publish can be retried independently
- HNSW (Ubuntu): add libopenblas-dev for BLAS requirements
- DiskANN (Ubuntu): keep MKL, remove redundant pkg-config (HNSW already has it)
- DiskANN (macOS): add protobuf for build requirements
- Both: ensure patchelf for auditwheel on Linux
This avoids OpenBLAS/MKL conflicts by using them in separate jobs
* chore: shorter build time
* chore: update faiss
* fix: no longger do embedding server reuse
* fix: do not reuse emb_server and close it properly
* feat: cli tool
* feat: cli more args
* fix: same embedding logic
Andy Lee