refactor: consolidate release and publish into single workflow

- Manual Release workflow now directly publishes to PyPI after downloading CI artifacts
- No more duplicate builds - reuses artifacts from CI
- build-and-publish.yml renamed to 'CI - Build Multi-Platform Packages'
- Publishing in CI workflow only for emergency manual triggers
- Updated RELEASE.md to reflect the new streamlined process

This fixes the issue where releases would trigger redundant builds.
This commit is contained in:
Andy Lee
2025-07-24 17:04:47 -07:00
parent 47a4c153eb
commit 95cf2f16e2
3 changed files with 68 additions and 16 deletions

View File

@@ -1,15 +1,14 @@
name: Build and Publish to PyPI name: CI - Build Multi-Platform Packages
on: on:
release:
types: [published]
push: push:
tags: branches: [ main ]
- 'v*' pull_request:
branches: [ main ]
workflow_dispatch: workflow_dispatch:
inputs: inputs:
publish: publish:
description: 'Publish to PyPI' description: 'Publish to PyPI (only use for emergency fixes)'
required: true required: true
default: 'false' default: 'false'
type: choice type: choice
@@ -221,12 +220,12 @@ jobs:
name: leann-meta-dist name: leann-meta-dist
path: packages/leann/dist/ path: packages/leann/dist/
# Publish to PyPI # Publish to PyPI (only for emergency fixes or manual triggers)
publish: publish:
name: Publish to PyPI name: Publish to PyPI (Emergency)
needs: [build-core, build-hnsw, build-diskann, build-meta] needs: [build-core, build-hnsw, build-diskann, build-meta]
runs-on: ubuntu-latest runs-on: ubuntu-latest
if: github.event_name == 'release' || (github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) if: github.event_name == 'workflow_dispatch' && github.event.inputs.publish != 'false'
steps: steps:
- name: Download all artifacts - name: Download all artifacts
@@ -245,7 +244,17 @@ jobs:
echo "📦 Packages to be published:" echo "📦 Packages to be published:"
ls -la all_wheels/ ls -la all_wheels/
- name: Publish to Test PyPI
if: github.event.inputs.publish == 'test'
uses: pypa/gh-action-pypi-publish@release/v1
with:
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
repository-url: https://test.pypi.org/legacy/
packages-dir: all_wheels/
skip-existing: true
- name: Publish to PyPI - name: Publish to PyPI
if: github.event.inputs.publish == 'prod'
uses: pypa/gh-action-pypi-publish@release/v1 uses: pypa/gh-action-pypi-publish@release/v1
with: with:
password: ${{ secrets.PYPI_API_TOKEN }} password: ${{ secrets.PYPI_API_TOKEN }}

View File

@@ -155,6 +155,34 @@ jobs:
echo "To test installation:" echo "To test installation:"
echo "pip install -i https://test.pypi.org/simple/ leann" echo "pip install -i https://test.pypi.org/simple/ leann"
- name: Publish to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
run: |
if [ -z "$TWINE_PASSWORD" ]; then
echo "❌ PYPI_API_TOKEN not configured!"
echo " Please add PYPI_API_TOKEN to repository secrets"
exit 1
fi
pip install twine
echo "📦 Publishing to PyPI..."
# Collect all wheels in one place
mkdir -p all_wheels
find packages/*/dist -name "*.whl" -exec cp {} all_wheels/ \;
find packages/*/dist -name "*.tar.gz" -exec cp {} all_wheels/ \;
echo "📋 Packages to publish:"
ls -la all_wheels/
# Upload to PyPI
twine upload all_wheels/* --skip-existing --verbose
echo "✅ Published to PyPI!"
echo "🎉 Check packages at: https://pypi.org/project/leann/"
- name: Create and push tag - name: Create and push tag
run: | run: |
git tag "v${{ inputs.version }}" git tag "v${{ inputs.version }}"
@@ -187,8 +215,16 @@ jobs:
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Trigger PyPI publish - name: Summary
run: | run: |
echo "🚀 Triggering PyPI publish workflow..." echo "✅ Release v${{ inputs.version }} completed!"
# The existing build-and-publish.yml will be triggered by the tag push echo ""
echo "✅ Release process completed! The publish workflow will run automatically." echo "📊 What was done:"
echo "1. ✅ Downloaded build artifacts from CI"
echo "2. ✅ Updated version numbers to ${{ inputs.version }}"
echo "3. ✅ Published packages to PyPI"
echo "4. ✅ Created git tag v${{ inputs.version }}"
echo "5. ✅ Created GitHub Release"
echo ""
echo "🎉 Installation:"
echo "pip install leann==${{ inputs.version }}"

View File

@@ -7,6 +7,14 @@ Before releasing, ensure:
2. ✅ CI has passed on the latest commit (check [Actions](https://github.com/yichuan-w/LEANN/actions/workflows/ci.yml)) 2. ✅ CI has passed on the latest commit (check [Actions](https://github.com/yichuan-w/LEANN/actions/workflows/ci.yml))
3. ✅ You have determined the new version number 3. ✅ You have determined the new version number
### Required: PyPI Configuration
To enable PyPI publishing:
1. Get a PyPI API token from https://pypi.org/manage/account/token/
2. Add it to repository secrets: Settings → Secrets → Actions → New repository secret
- Name: `PYPI_API_TOKEN`
- Value: Your PyPI token (starts with `pypi-`)
### Optional: TestPyPI Configuration ### Optional: TestPyPI Configuration
To enable TestPyPI testing (recommended but not required): To enable TestPyPI testing (recommended but not required):
@@ -29,12 +37,11 @@ To enable TestPyPI testing (recommended but not required):
6. Click "Run workflow" 6. Click "Run workflow"
**What happens:** **What happens:**
- ✅ Validates version format
- ✅ Downloads pre-built packages from CI (no rebuild needed!) - ✅ Downloads pre-built packages from CI (no rebuild needed!)
- ✅ Updates all package versions - ✅ Updates all package versions
- ✅ Optionally tests on TestPyPI - ✅ Optionally tests on TestPyPI
-**Publishes directly to PyPI**
- ✅ Creates tag and GitHub release - ✅ Creates tag and GitHub release
- ✅ Automatically triggers PyPI publish
### Via Command Line ### Via Command Line
@@ -54,7 +61,7 @@ This script will:
1. Update all package versions 1. Update all package versions
2. Commit and push changes 2. Commit and push changes
3. Create GitHub release 3. Create GitHub release
4. CI automatically builds and publishes to PyPI 4. **Manual Release workflow will automatically publish to PyPI**
⚠️ **Note**: If CI fails, you'll need to manually fix and re-tag ⚠️ **Note**: If CI fails, you'll need to manually fix and re-tag