TBNilles/LEANN
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

upgrade: switch from manylinux2014 to manylinux_2_35

Browse Source 
- Use manylinux_2_35 (GLIBC 2.35) instead of manylinux2014 (GLIBC 2.17)
- Still compatible with Google Colab (requires ≤2.35)
- Benefits: newer toolchain, better performance, modern C++ features
- Switch from yum to dnf package manager
- Remove pyzmq version cap as manylinux_2_35 has newer ZeroMQ
- Update documentation to reflect the change
This commit is contained in:
Andy Lee
2025-07-25 13:40:07 -07:00
parent 02672c040d
commit 971653fa1a
4 changed files with 17 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files

26
.github/workflows/build-cibuildwheel.yml vendored
View File

@@ -46,15 +46,14 @@ jobs:
CIBW_BUILD: cp39-* cp310-* cp311-* cp312-* cp313-* CIBW_BUILD: cp39-* cp310-* cp311-* cp312-* cp313-*
CIBW_SKIP: "*-win32 *-manylinux_i686 pp* *musllinux*" CIBW_SKIP: "*-win32 *-manylinux_i686 pp* *musllinux*"
# 使用manylinux2014生成兼容性更好的wheels # Use manylinux_2_35 for Colab compatibility with modern features
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014 CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_35
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014 CIBW_MANYLINUX_AARCH64_IMAGE: manylinux_2_35
# Linux dependencies - 使用yum因为manylinux2014基于CentOS 7 # Linux dependencies - using dnf for manylinux_2_35 (based on AlmaLinux 9)
CIBW_BEFORE_ALL_LINUX: | CIBW_BEFORE_ALL_LINUX: |
yum install -y epel-release dnf install -y epel-release
yum install -y gcc-c++ boost-devel zeromq-devel openblas-devel cmake3 dnf install -y gcc-c++ boost-devel zeromq-devel openblas-devel cmake python3-devel
ln -sf /usr/bin/cmake3 /usr/bin/cmake
# Install numpy before building # Install numpy before building
CIBW_BEFORE_BUILD: | CIBW_BEFORE_BUILD: |
@@ -70,7 +69,7 @@ jobs:
# Pre-install test dependencies to avoid compilation # Pre-install test dependencies to avoid compilation
CIBW_BEFORE_TEST: | CIBW_BEFORE_TEST: |
pip install --only-binary :all: "pyzmq>=23.0.0,<27" pip install --only-binary :all: "pyzmq>=23.0.0"
# Test command to verify the wheel works # Test command to verify the wheel works
CIBW_TEST_COMMAND: | CIBW_TEST_COMMAND: |
@@ -109,13 +108,12 @@ jobs:
CIBW_BUILD: cp39-* cp310-* cp311-* cp312-* cp313-* CIBW_BUILD: cp39-* cp310-* cp311-* cp312-* cp313-*
CIBW_SKIP: "*-win32 *-manylinux_i686 pp* *musllinux*" CIBW_SKIP: "*-win32 *-manylinux_i686 pp* *musllinux*"
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014 CIBW_MANYLINUX_X86_64_IMAGE: manylinux_2_35
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014 CIBW_MANYLINUX_AARCH64_IMAGE: manylinux_2_35
CIBW_BEFORE_ALL_LINUX: | CIBW_BEFORE_ALL_LINUX: |
yum install -y epel-release dnf install -y epel-release
yum install -y gcc-c++ boost-devel zeromq-devel openblas-devel cmake3 dnf install -y gcc-c++ boost-devel zeromq-devel openblas-devel cmake python3-devel
ln -sf /usr/bin/cmake3 /usr/bin/cmake
# Install numpy before building # Install numpy before building
CIBW_BEFORE_BUILD: | CIBW_BEFORE_BUILD: |
@@ -131,7 +129,7 @@ jobs:
# Pre-install test dependencies to avoid compilation # Pre-install test dependencies to avoid compilation
CIBW_BEFORE_TEST: | CIBW_BEFORE_TEST: |
pip install --only-binary :all: "pyzmq>=23.0.0,<27" pip install --only-binary :all: "pyzmq>=23.0.0"
# Test command to verify the wheel works # Test command to verify the wheel works
CIBW_TEST_COMMAND: | CIBW_TEST_COMMAND: |

2
MANYLINUX_BUILD_STRATEGY.md
View File

@@ -4,7 +4,7 @@
Google Colab requires wheels compatible with `manylinux_2_35_x86_64` or earlier. Our previous builds were producing `manylinux_2_39_x86_64` wheels, which are incompatible. Google Colab requires wheels compatible with `manylinux_2_35_x86_64` or earlier. Our previous builds were producing `manylinux_2_39_x86_64` wheels, which are incompatible.
## Solution ## Solution
We're using `cibuildwheel` with `manylinux2014` images to build wheels that are compatible with a wide range of Linux distributions, including Google Colab. We're using `cibuildwheel` with `manylinux_2_35` images to build wheels that are compatible with Google Colab while maintaining modern toolchain features.
### Key Changes ### Key Changes

2
packages/leann-backend-hnsw/pyproject.toml
View File

@@ -11,7 +11,7 @@ description = "Custom-built HNSW (Faiss) backend for the Leann toolkit."
dependencies = [ dependencies = [
"leann-core==0.1.8", "leann-core==0.1.8",
"numpy", "numpy",
"pyzmq>=23.0.0,<27", # Cap at 26.x for manylinux2014 compatibility "pyzmq>=23.0.0",
"msgpack>=1.0.0", "msgpack>=1.0.0",
] ]

6
pyproject.toml
View File

@@ -65,9 +65,9 @@ leann-backend-hnsw = { path = "packages/leann-backend-hnsw", editable = true }
# Skip 32-bit and PyPy builds # Skip 32-bit and PyPy builds
skip = "*-win32 *-manylinux_i686 pp* *musllinux*" skip = "*-win32 *-manylinux_i686 pp* *musllinux*"
# Use manylinux2014 for maximum compatibility (supports GLIBC 2.17) # Use manylinux_2_35 for Colab compatibility while keeping modern features
manylinux-x86_64-image = "manylinux2014" manylinux-x86_64-image = "manylinux_2_35"
manylinux-aarch64-image = "manylinux2014" manylinux-aarch64-image = "manylinux_2_35"
# Linux system dependencies # Linux system dependencies
[tool.cibuildwheel.linux] [tool.cibuildwheel.linux]