Commit Graph

23 Commits

Author SHA1 Message Date
Andy Lee
b3e9ee96fa fix: resolve all ruff linting errors and add lint CI check
- Fix ambiguous fullwidth characters (commas, parentheses) in strings and comments
- Replace Chinese comments with English equivalents
- Fix unused imports with proper noqa annotations for intentional imports
- Fix bare except clauses with specific exception types
- Fix redefined variables and undefined names
- Add ruff noqa annotations for generated protobuf files
- Add lint and format check to GitHub Actions CI pipeline
2025-07-26 22:38:13 -07:00
Andy Lee
2baaa4549b fix: handle relative paths in HNSW embedding server metadata
- Convert relative paths to absolute paths based on metadata file location
- Fixes FileNotFoundError when starting embedding server
- Resolves issue with passages file not found in different working directories
2025-07-25 16:09:53 -07:00
Andy Lee
43155d2811 fix: supress resources leak logs 2025-07-22 19:53:45 -07:00
Andy Lee
c2f35c8e73 fix: logs 2025-07-21 23:02:13 -07:00
Andy Lee
573313f0b6 refactor: logs 2025-07-21 22:45:24 -07:00
yichuan520030910320
587ce65cf6 Merge branch 'main' of https://github.com/yichuan-w/LEANN 2025-07-21 21:54:27 -07:00
yichuan520030910320
ccf6c8bfd7 fix flush print 2025-07-21 21:54:20 -07:00
Andy Lee
c112956d2d fix: mlx 2025-07-21 21:29:15 -07:00
Andy Lee
b3970793cf fix: cache the loaded model 2025-07-21 21:20:53 -07:00
Andy Lee
1b6272ce0e Building, CLI tool & Embedding Server Fixed (#5)
* chore: shorter build time

* chore: update faiss

* fix: no longger do embedding server reuse

* fix: do not reuse emb_server and close it properly

* feat: cli tool

* feat: cli more args

* fix: same embedding logic
2025-07-21 20:17:25 -07:00
yichuan520030910320
0796a52df1 change wecaht app split logic 2025-07-19 19:43:30 -07:00
Andy Lee
1c5fec5565 perf: make embedder loading faster by 6x, and embed queries through the server 2025-07-17 20:08:06 -07:00
Andy Lee
a13c527e39 feat: openai embeddings 2025-07-17 17:02:47 -07:00
Andy Lee
6a1dc895fb feat: disable warmup by default 2025-07-15 22:16:02 -07:00
yichuan520030910320
e5a9ca8787 fix mem compare 2025-07-14 22:55:10 -07:00
Andy Lee
baa60b40d1 fix: smaller warmup id 2025-07-14 15:20:45 -07:00
Andy Lee
3da5b44d7f fix: mlx when searching, added to embedding_server 2025-07-14 01:11:21 -07:00
Andy Lee
8b4654921b fix: run faiss in subprocess to prevent kmp 2025-07-14 00:29:21 -07:00
Andy Lee
3b5a185e60 refactor: check if current emb_server has correct passages/embedder 2025-07-13 22:43:51 -07:00
Andy Lee
eb6f504789 Datastore reproduce (#3)
* fix: diskann zmq port and passages

* feat: auto discovery of packages and fix passage gen for diskann

* docs: embedding pruning

* refactor: passage structure

* feat: reproducible research datas, rpj_wiki & dpr

* refactor: chat and base searcher

* feat: chat on mps
2025-07-11 23:37:23 -07:00
Andy Lee
910927a405 feat: support more embedders 2025-07-06 00:35:07 +00:00
Andy Lee
0aa84e147b feat: hnsw embedding server and csr format 2025-07-05 23:04:41 +00:00
yichuan520030910320
46f6cc100b Initial commit 2025-06-30 09:05:05 +00:00