Andy Lee
85b7ba0168
feat: allow build from existed embeddings
2025-07-19 01:27:37 -07:00
yichuan520030910320
e117743d24
Merge branch 'main' of github.com:yichuan520030910320/LEANN-RAG
2025-07-17 22:29:39 -07:00
yichuan520030910320
aec2291f04
add embedding api
2025-07-17 22:29:31 -07:00
Andy Lee
71c7de9c84
fix: build with direct embedding
2025-07-17 21:49:36 -07:00
Andy Lee
1c5fec5565
perf: make embedder loading faster by 6x, and embed queries through the server
2025-07-17 20:08:06 -07:00
Andy Lee
a13c527e39
feat: openai embeddings
2025-07-17 17:02:47 -07:00
yichuan520030910320
51255bdffa
update readme and add timer
2025-07-16 17:15:51 -07:00
Andy Lee
f77c4e38cb
perf: reuse embedding server for query embed
2025-07-16 16:12:15 -07:00
Andy Lee
2a1a152073
refactor: nits
2025-07-16 15:39:58 -07:00
Andy Lee
7b9406a3ea
feat: different search_args and docstrings
2025-07-16 15:25:58 -07:00
Andy Lee
e595bbb5fb
feat: hint for users about wrong model name
2025-07-15 22:40:40 -07:00
Andy Lee
6a1dc895fb
feat: disable warmup by default
2025-07-15 22:16:02 -07:00
yichuan520030910320
e5a9ca8787
fix mem compare
2025-07-14 22:55:10 -07:00
Andy Lee
baa60b40d1
fix: smaller warmup id
2025-07-14 15:20:45 -07:00
Andy Lee
3da5b44d7f
fix: mlx when searching, added to embedding_server
2025-07-14 01:11:21 -07:00
Andy Lee
8b4654921b
fix: run faiss in subprocess to prevent kmp
2025-07-14 00:29:21 -07:00
Andy Lee
3b5a185e60
refactor: check if current emb_server has correct passages/embedder
2025-07-13 22:43:51 -07:00
Andy Lee
fddc61df5e
chore: reset to latest version
2025-07-13 17:06:48 -07:00
yichuan520030910320
c69afb56e4
Resolve submodule conflict - update to af2a264
2025-07-13 17:03:42 -07:00
yichuan520030910320
0fa8a9191f
add wechat history extract app
2025-07-13 16:52:08 -07:00
Andy Lee
48dda1cb5b
feat: mlx
2025-07-13 02:13:04 -07:00
Andy Lee
71ef4b7d4c
fix: reproducible dpr on mac
2025-07-12 18:13:22 -07:00
Fangzhou66
88ca09440d
fix some hf problem
2025-07-12 16:13:15 -07:00
Andy Lee
8e0ab4a28d
chore: update deps
2025-07-12 22:48:13 +00:00
yichuan520030910320
9b8c5041dc
update readme
2025-07-12 13:01:11 -07:00
yichuan520030910320
74ffd7ec64
add email test code
2025-07-11 23:59:47 -07:00
Andy Lee
eb6f504789
Datastore reproduce ( #3 )
...
* fix: diskann zmq port and passages
* feat: auto discovery of packages and fix passage gen for diskann
* docs: embedding pruning
* refactor: passage structure
* feat: reproducible research datas, rpj_wiki & dpr
* refactor: chat and base searcher
* feat: chat on mps
2025-07-11 23:37:23 -07:00
Andy Lee
27b3a26e75
fix(deps): Update DiskANN with cleaned up CMake configuration
2025-07-08 23:27:05 +00:00
Andy Lee
41d872504e
feat(deps): Update DiskANN to use system-installed Boost and Protobuf
2025-07-08 23:13:36 +00:00
Andy Lee
963cd05273
chore: diskann modules
2025-07-08 21:57:38 +00:00
Andy Lee
09b6e67baf
chore: diskann upg boost
2025-07-08 21:44:44 +00:00
yichuan520030910320
dafb2aacab
update macos env
2025-07-08 14:37:41 -07:00
Andy Lee
a6c400cd4f
chroe: linux boost and protobuf
2025-07-08 21:25:43 +00:00
Andy Lee
c013e5ccce
chore: linux deps
2025-07-08 13:55:39 -07:00
Andy Lee
f25a1a3840
chore: macos compatible
2025-07-08 13:32:00 -07:00
yichuan520030910320
6497e17671
add gpu chunk embedd and add complexity in hnsw
2025-07-08 18:40:52 +00:00
yichuan520030910320
44369a8138
update diskann module
2025-07-07 18:27:07 -07:00
yichuan520030910320
637dab379e
add workaround code
2025-07-07 23:13:47 +00:00
yichuan520030910320
6fc57eb48e
add reuse code
2025-07-07 21:07:00 +00:00
yichuan520030910320
95a653993a
rm useless
2025-07-06 06:47:20 +00:00
yichuan520030910320
af0959818d
rm useless
2025-07-06 05:21:05 +00:00
Andy Lee
cf17c85607
Make DiskANN and HNSW work on main example ( #2 )
...
* fix: diskann zmq port and passages
* feat: auto discovery of packages and fix passage gen for diskann
2025-07-05 22:18:12 -07:00
Andy Lee
a38bc0a3fc
refactor: embedding server manager
2025-07-06 01:54:46 +00:00
yichuan520030910320
df63526503
merge main
2025-07-06 00:50:58 +00:00
yichuan520030910320
e92deee1e8
fix larger file read and add faq
2025-07-06 00:48:57 +00:00
Andy Lee
910927a405
feat: support more embedders
2025-07-06 00:35:07 +00:00
Andy Lee
0aa84e147b
feat: hnsw embedding server and csr format
2025-07-05 23:04:41 +00:00
yichuan520030910320
368474d036
fix larger file read and add faq
2025-07-03 23:25:36 +00:00
yichuan520030910320
a627abe794
fix file path bug still compatiable bug in hnsw search
2025-07-03 02:02:42 +00:00
yichuan520030910320
44815ee7fd
add configuable funcname
2025-07-02 05:18:00 +00:00
