Andy Lee
9a5c197acd
fix: auto-detect normalized embeddings and use cosine distance
...
- Add automatic detection for normalized embedding models (OpenAI, Voyage AI, Cohere)
- Automatically set distance_metric='cosine' for normalized embeddings
- Add warnings when using non-optimal distance metrics
- Implement manual L2 normalization in HNSW backend (custom Faiss build lacks normalize_L2)
- Fix DiskANN zmq_port compatibility with lazy loading strategy
- Add documentation for normalized embeddings feature
This fixes the low accuracy issue when using OpenAI text-embedding-3-small model with default MIPS metric.
2025-07-27 20:21:05 -07:00
yichuan520030910320
af1790395a
fix ruff errors and formatting
2025-07-27 02:22:54 -07:00
GitHub Actions
5d09586853
chore: release v0.1.14
2025-07-27 08:50:56 +00:00
Andy Lee
a7cba078dd
chore: consolidate essential fixes and add pre-commit hooks
...
- Add pre-commit configuration with ruff and black
- Fix lint CI job to use uv tool install instead of sync
- Add essential LlamaIndex dependencies to leann-core
Co-Authored-By: Yichuan Wang <73766326+yichuan-w@users.noreply.github.com >
2025-07-27 01:24:24 -07:00
Andy Lee
b3e9ee96fa
fix: resolve all ruff linting errors and add lint CI check
...
- Fix ambiguous fullwidth characters (commas, parentheses) in strings and comments
- Replace Chinese comments with English equivalents
- Fix unused imports with proper noqa annotations for intentional imports
- Fix bare except clauses with specific exception types
- Fix redefined variables and undefined names
- Add ruff noqa annotations for generated protobuf files
- Add lint and format check to GitHub Actions CI pipeline
2025-07-26 22:38:13 -07:00
GitHub Actions
8375f601ba
chore: release v0.1.13
2025-07-27 01:08:17 +00:00
yichuan520030910320
c87c0fe662
update colab install & fix colab path
2025-07-26 18:07:31 -07:00
GitHub Actions
802020cb41
chore: release v0.1.12
2025-07-26 23:35:28 +00:00
yichuan520030910320
cdb92f7cf4
update pytoml version && fix colab env && fix pdf extract in pip
2025-07-26 16:33:13 -07:00
yichuan520030910320
dc69bdec00
Merge branch 'main' of https://github.com/yichuan-w/LEANN
2025-07-25 17:54:43 -07:00
yichuan520030910320
98073e9868
update missing pkg
2025-07-25 17:54:21 -07:00
GitHub Actions
cf2ef48967
chore: release v0.1.11
2025-07-26 00:12:37 +00:00
GitHub Actions
52584a171f
chore: release v0.1.10
2025-07-25 23:12:16 +00:00
GitHub Actions
75ddcd6158
chore: release v0.1.9
2025-07-25 20:04:42 +00:00
Andy Lee
2a4df11f5c
fix: absolute path for passages
2025-07-25 11:59:30 -07:00
yichuan520030910320
cd8b970eff
Merge branch 'main' of https://github.com/yichuan-w/LEANN
2025-07-25 01:45:57 -07:00
GitHub Actions
e1ae087207
chore: release v0.1.8
2025-07-25 08:24:40 +00:00
Andy Lee
48c5e12ac1
fix: use absolute path for passages_file to prevent FileNotFoundError
...
When embedding server is launched as a subprocess, it may run in a different
working directory. Using absolute paths ensures the server can always find
the metadata file regardless of where it's launched from.
2025-07-25 01:23:47 -07:00
yichuan520030910320
800d4cf111
Merge branch 'main' of https://github.com/yichuan-w/LEANN
2025-07-25 00:12:47 -07:00
yichuan520030910320
b6d43f5fd9
add gif
2025-07-25 00:12:35 -07:00
GitHub Actions
e64b599276
chore: release v0.1.7
2025-07-25 04:47:57 +00:00
GitHub Actions
166986d5e6
chore: release v0.1.6
2025-07-25 04:30:07 +00:00
GitHub Actions
ed27a127d5
chore: release v0.1.5
2025-07-25 04:00:54 +00:00
GitHub Actions
9000a7083d
chore: release v0.1.4
2025-07-25 02:23:36 +00:00
GitHub Actions
20f2aece08
chore: release v0.1.3
2025-07-25 02:05:11 +00:00
GitHub Actions
cea1f6f87c
chore: release v0.1.2
2025-07-25 01:53:29 +00:00
GitHub Actions
faf5ae3533
chore: release v0.1.1
2025-07-24 23:36:23 +00:00
yichuan520030910320
de252fef31
[chat] update 30s example
2025-07-24 14:40:33 -07:00
Andy Lee
7add391b2c
chore: build and package
2025-07-24 00:47:46 -07:00
yichuan520030910320
efd6373b32
[chat] update huggingface chat and make qwen no thinking
2025-07-24 00:11:42 -07:00
yichuan520030910320
42de27e16a
Merge branch 'main' of https://github.com/yichuan-w/LEANN
2025-07-23 18:17:19 -07:00
yichuan520030910320
c083bda5b7
fix several bug
2025-07-23 18:17:11 -07:00
Andy Lee
e86da38726
fix: ollama hint for similar models
2025-07-23 15:45:10 -07:00
yichuan520030910320
851f0f04c3
fix some para
2025-07-23 01:46:34 -07:00
Andy Lee
43155d2811
fix: supress resources leak logs
2025-07-22 19:53:45 -07:00
Andy Lee
d3f85678ec
perf: much faster loading and embedding serving
2025-07-22 19:38:22 -07:00
Andy Lee
8513471573
feat: make diskann runnable
2025-07-22 14:26:03 -07:00
Andy Lee
71e5f1774c
docs: cli
2025-07-21 23:48:40 -07:00
Andy Lee
ab72a2ab9d
fix: more logs
2025-07-21 23:08:53 -07:00
yichuan520030910320
046d457d22
Merge branch 'main' of https://github.com/yichuan-w/LEANN
2025-07-21 23:04:00 -07:00
yichuan520030910320
7fd0a30fee
upd log
2025-07-21 23:03:52 -07:00
Andy Lee
c2f35c8e73
fix: logs
2025-07-21 23:02:13 -07:00
Andy Lee
573313f0b6
refactor: logs
2025-07-21 22:45:24 -07:00
Andy Lee
c112956d2d
fix: mlx
2025-07-21 21:29:15 -07:00
Andy Lee
b3970793cf
fix: cache the loaded model
2025-07-21 21:20:53 -07:00
yichuan520030910320
727724990e
add todo
2025-07-21 20:59:09 -07:00
yichuan520030910320
530f6e4af5
add progress bar in build
2025-07-21 20:55:18 -07:00
Andy Lee
2f224f5793
fix: use server to emb query only when recompute
2025-07-21 20:40:21 -07:00
Andy Lee
1b6272ce0e
Building, CLI tool & Embedding Server Fixed ( #5 )
...
* chore: shorter build time
* chore: update faiss
* fix: no longger do embedding server reuse
* fix: do not reuse emb_server and close it properly
* feat: cli tool
* feat: cli more args
* fix: same embedding logic
2025-07-21 20:17:25 -07:00
yichuan520030910320
83b7ea5a59
change wecaht app split logic& merge
2025-07-19 19:44:33 -07:00
