Andy Lee
43155d2811
fix: supress resources leak logs
2025-07-22 19:53:45 -07:00
Andy Lee
d3f85678ec
perf: much faster loading and embedding serving
2025-07-22 19:38:22 -07:00
Andy Lee
8513471573
feat: make diskann runnable
2025-07-22 14:26:03 -07:00
Andy Lee
71e5f1774c
docs: cli
2025-07-21 23:48:40 -07:00
Andy Lee
ab72a2ab9d
fix: more logs
2025-07-21 23:08:53 -07:00
Andy Lee
c2f35c8e73
fix: logs
2025-07-21 23:02:13 -07:00
Andy Lee
573313f0b6
refactor: logs
2025-07-21 22:45:24 -07:00
Andy Lee
c112956d2d
fix: mlx
2025-07-21 21:29:15 -07:00
Andy Lee
b3970793cf
fix: cache the loaded model
2025-07-21 21:20:53 -07:00
Andy Lee
2f224f5793
fix: use server to emb query only when recompute
2025-07-21 20:40:21 -07:00
Andy Lee
1b6272ce0e
Building, CLI tool & Embedding Server Fixed ( #5 )
...
* chore: shorter build time
* chore: update faiss
* fix: no longger do embedding server reuse
* fix: do not reuse emb_server and close it properly
* feat: cli tool
* feat: cli more args
* fix: same embedding logic
2025-07-21 20:17:25 -07:00
Andy Lee
34c71c072d
chore: parallel compile fix
2025-07-19 22:51:47 -07:00
Andy Lee
6d2149c503
chore: parallel compile fix
2025-07-19 22:46:24 -07:00
Andy Lee
043b0bf69d
chore: parallel compile fix
2025-07-19 22:34:19 -07:00
Andy Lee
9b07e392c6
chore: parallel compile
2025-07-19 22:32:13 -07:00
Andy Lee
e60fad8c73
chore: mark diskann as optional
2025-07-19 22:24:44 -07:00
Andy Lee
19c1b182c3
docs: effects figure
2025-07-19 22:07:04 -07:00
Andy Lee
49edea780c
docs: figure
2025-07-19 21:59:58 -07:00
Andy Lee
12ef5a1900
docs: effects
2025-07-19 21:57:12 -07:00
Andy Lee
d21a134b2a
docs: polish
2025-07-19 21:53:41 -07:00
Andy Lee and Yichuan Wang
1cd809aa41
[Docs] README polished version ( #4 )
...
* docs: polish
* docs: logo
* docs: logo
* docs: logo with text
* docs: readme effects
* docs: polish
* docs: highlight applications
* docs: polish
* docs: how it works earlier
* docs: polish
* docs: polish
* docs: follow yichuan's suggestion
* docs: follow yichuan's suggestion
---------
Co-authored-by: Yichuan Wang <73766326+yichuan-w@users.noreply.github.com >
2025-07-19 21:47:25 -07:00
Andy Lee
85b7ba0168
feat: allow build from existed embeddings
2025-07-19 01:27:37 -07:00
Andy Lee
71c7de9c84
fix: build with direct embedding
2025-07-17 21:49:36 -07:00
Andy Lee
1c5fec5565
perf: make embedder loading faster by 6x, and embed queries through the server
2025-07-17 20:08:06 -07:00
Andy Lee
a13c527e39
feat: openai embeddings
2025-07-17 17:02:47 -07:00
Andy Lee
f77c4e38cb
perf: reuse embedding server for query embed
2025-07-16 16:12:15 -07:00
Andy Lee
2a1a152073
refactor: nits
2025-07-16 15:39:58 -07:00
Andy Lee
7b9406a3ea
feat: different search_args and docstrings
2025-07-16 15:25:58 -07:00
Andy Lee
c3fb949693
docs: ollama
2025-07-16 15:12:37 -07:00
Andy Lee
e595bbb5fb
feat: hint for users about wrong model name
2025-07-15 22:40:40 -07:00
Andy Lee
6a1dc895fb
feat: disable warmup by default
2025-07-15 22:16:02 -07:00
Andy Lee
125c1f6f25
fix: model name
2025-07-15 21:48:45 -07:00
Andy Lee
d94a5176dc
docs: storage reduction data
2025-07-15 15:37:17 -07:00
Andy Lee
f2feccdbd0
fix: mem compare
2025-07-14 16:35:08 -07:00
Andy Lee
b89e56e9c2
fix: file name
2025-07-14 15:34:56 -07:00
Andy Lee
baa60b40d1
fix: smaller warmup id
2025-07-14 15:20:45 -07:00
Andy Lee
ef01d6997a
fix: faiss only
2025-07-14 13:15:55 -07:00
Andy Lee
3da5b44d7f
fix: mlx when searching, added to embedding_server
2025-07-14 01:11:21 -07:00
Andy Lee
8b4654921b
fix: run faiss in subprocess to prevent kmp
2025-07-14 00:29:21 -07:00
Andy Lee
711fb4a775
feat: compare faiss
2025-07-13 22:44:16 -07:00
Andy Lee
3b5a185e60
refactor: check if current emb_server has correct passages/embedder
2025-07-13 22:43:51 -07:00
Andy Lee
fddc61df5e
chore: reset to latest version
2025-07-13 17:06:48 -07:00
Andy Lee
53c58fa755
perf: switch to tranditional pdf reader
2025-07-13 17:04:23 -07:00
Andy Lee
48dda1cb5b
feat: mlx
2025-07-13 02:13:04 -07:00
Andy Lee
71ef4b7d4c
fix: reproducible dpr on mac
2025-07-12 18:13:22 -07:00
Andy Lee
ecab43e307
feat: dataset for evaluation
2025-07-12 23:43:10 +00:00
Andy Lee
8e0ab4a28d
chore: update deps
2025-07-12 22:48:13 +00:00
Andy Lee
eb6f504789
Datastore reproduce ( #3 )
...
* fix: diskann zmq port and passages
* feat: auto discovery of packages and fix passage gen for diskann
* docs: embedding pruning
* refactor: passage structure
* feat: reproducible research datas, rpj_wiki & dpr
* refactor: chat and base searcher
* feat: chat on mps
2025-07-11 23:37:23 -07:00
Andy Lee
b744faa7e6
chore: all deps
2025-07-08 23:37:40 +00:00
Andy Lee
27b3a26e75
fix(deps): Update DiskANN with cleaned up CMake configuration
2025-07-08 23:27:05 +00:00
Andy Lee
41d872504e
feat(deps): Update DiskANN to use system-installed Boost and Protobuf
2025-07-08 23:13:36 +00:00
Andy Lee
963cd05273
chore: diskann modules
2025-07-08 21:57:38 +00:00
Andy Lee
09b6e67baf
chore: diskann upg boost
2025-07-08 21:44:44 +00:00
Andy Lee
a6c400cd4f
chroe: linux boost and protobuf
2025-07-08 21:25:43 +00:00
Andy Lee
c013e5ccce
chore: linux deps
2025-07-08 13:55:39 -07:00
Andy Lee
f25a1a3840
chore: macos compatible
2025-07-08 13:32:00 -07:00
Andy Lee
cf17c85607
Make DiskANN and HNSW work on main example ( #2 )
...
* fix: diskann zmq port and passages
* feat: auto discovery of packages and fix passage gen for diskann
2025-07-05 22:18:12 -07:00
Andy Lee
a38bc0a3fc
refactor: embedding server manager
2025-07-06 01:54:46 +00:00
Andy Lee
910927a405
feat: support more embedders
2025-07-06 00:35:07 +00:00
Andy Lee
0aa84e147b
feat: hnsw embedding server and csr format
2025-07-05 23:04:41 +00:00
Andy Lee
30898814ae
chore: docling deps
2025-06-30 10:52:10 +00:00
Andy Lee