TBNilles/LEANN
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
yichuan520030910320
2025-06-30 09:05:05 +00:00
commit 46f6cc100b
1231 changed files with 278432 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
research/paper_plot/data/main_latency.csv Normal file
View File

@@ -0,0 +1,25 @@
Dataset,Hardware,Recall_target,HNSW,IVF,DiskANN,IVF-Disk,IVF-Recompute,Our,BM25,LLM_Gen_Time_1B,LLM_Gen_Time_3B,LLM_Gen_Time_7B
NQ,A10,85%,0.046,1.656,0.017,2.996,482.53,3.323,0.021,0.085,0.217,0.472
NQ,A10,90%,0.051,2.552,0.028,3.437,769.04,4.616,0,0.085,0.217,0.472
NQ,A10,95%,0.055,5.163,0.070,5.602,1436.26,19.494,0,0.085,0.217,0.472
NQ,MAC,85%,0,0,0.152,2.199,1535.10,7.971,0.033,0.316,0.717,1.468
NQ,MAC,90%,0,0,0.37,2.936,2446.60,13.843,0,0.316,0.717,1.468
NQ,MAC,95%,0,0,1.207,4.191,4569.29,44.363,0,0.316,0.717,1.468
TriviaQA,A10,85%,0.042,1.772,0.032,2.464,560.5,3.752,0.033,0.139,0.156,0.315
TriviaQA,A10,90%,0.043,3.541,0.057,3.651,997.81,5.777,0,0.139,0.156,0.315
TriviaQA,A10,95%,0.053,7.168,0.090,5.458,2005.33,20.944,0,0.139,0.156,0.315
TriviaQA,MAC,85%,0,0,0.481,1.875,1783.14787,8.889,0.036,0.325,0.692,1.415
TriviaQA,MAC,90%,0,0,0.984,2.639,3174.410301,17.145,0,0.325,0.692,1.415
TriviaQA,MAC,95%,0,0,1.578,3.884,6379.712245,47.909,0,0.325,0.692,1.415
GPQA,A10,85%,0.041,0.134,0.024,0.048,40.16,1.897,0.137,0.443,0.396,0.651
GPQA,A10,90%,0.042,0.174,0.034,0.06,54.71,1.733,0,0.443,0.396,0.651
GPQA,A10,95%,0.045,0.292,0.051,0.11,97.67,4.033,0,0.443,0.396,0.651
GPQA,MAC,85%,0,0,0.144,0.087,127.7707505,4.762,0.100,0.37,0.813,1.676
GPQA,MAC,90%,0,0,0.288,0.108,174.0647409,5.223,0,0.37,0.813,1.676
GPQA,MAC,95%,0,0,0.497,0.132,310.7380142,9.715,0,0.37,0.813,1.676
HotpotQA,A10,85%,0.044,2.519,0.054,4.048,724.26,10.358,0.70,0.144,0.196,0.420
HotpotQA,A10,90%,0.049,3.867,0.109,5.045,1173.67,15.515,0,0.144,0.196,0.420
HotpotQA,A10,95%,0.07,10.928,0.412,8.659,3079.57,61.757,0,0.144,0.196,0.420
HotpotQA,MAC,85%,0,0,0.974,2.844,2304.125187,23.636,0.052,0.144,0.196,0.420
HotpotQA,MAC,90%,0,0,1.913,3.542,3415.736201,44.803,0,0.144,0.196,0.420
HotpotQA,MAC,95%,0,0,5.783,6.764,9797.244043,140.62,0,0.144,0.196,0.420
1 Dataset Hardware Recall_target HNSW IVF DiskANN IVF-Disk IVF-Recompute Our BM25 LLM_Gen_Time_1B LLM_Gen_Time_3B LLM_Gen_Time_7B
2 NQ A10 85% 0.046 1.656 0.017 2.996 482.53 3.323 0.021 0.085 0.217 0.472
3 NQ A10 90% 0.051 2.552 0.028 3.437 769.04 4.616 0 0.085 0.217 0.472
4 NQ A10 95% 0.055 5.163 0.070 5.602 1436.26 19.494 0 0.085 0.217 0.472
5 NQ MAC 85% 0 0 0.152 2.199 1535.10 7.971 0.033 0.316 0.717 1.468
6 NQ MAC 90% 0 0 0.37 2.936 2446.60 13.843 0 0.316 0.717 1.468
7 NQ MAC 95% 0 0 1.207 4.191 4569.29 44.363 0 0.316 0.717 1.468
8 TriviaQA A10 85% 0.042 1.772 0.032 2.464 560.5 3.752 0.033 0.139 0.156 0.315
9 TriviaQA A10 90% 0.043 3.541 0.057 3.651 997.81 5.777 0 0.139 0.156 0.315
10 TriviaQA A10 95% 0.053 7.168 0.090 5.458 2005.33 20.944 0 0.139 0.156 0.315
11 TriviaQA MAC 85% 0 0 0.481 1.875 1783.14787 8.889 0.036 0.325 0.692 1.415
12 TriviaQA MAC 90% 0 0 0.984 2.639 3174.410301 17.145 0 0.325 0.692 1.415
13 TriviaQA MAC 95% 0 0 1.578 3.884 6379.712245 47.909 0 0.325 0.692 1.415
14 GPQA A10 85% 0.041 0.134 0.024 0.048 40.16 1.897 0.137 0.443 0.396 0.651
15 GPQA A10 90% 0.042 0.174 0.034 0.06 54.71 1.733 0 0.443 0.396 0.651
16 GPQA A10 95% 0.045 0.292 0.051 0.11 97.67 4.033 0 0.443 0.396 0.651
17 GPQA MAC 85% 0 0 0.144 0.087 127.7707505 4.762 0.100 0.37 0.813 1.676
18 GPQA MAC 90% 0 0 0.288 0.108 174.0647409 5.223 0 0.37 0.813 1.676
19 GPQA MAC 95% 0 0 0.497 0.132 310.7380142 9.715 0 0.37 0.813 1.676
20 HotpotQA A10 85% 0.044 2.519 0.054 4.048 724.26 10.358 0.70 0.144 0.196 0.420
21 HotpotQA A10 90% 0.049 3.867 0.109 5.045 1173.67 15.515 0 0.144 0.196 0.420
22 HotpotQA A10 95% 0.07 10.928 0.412 8.659 3079.57 61.757 0 0.144 0.196 0.420
23 HotpotQA MAC 85% 0 0 0.974 2.844 2304.125187 23.636 0.052 0.144 0.196 0.420
24 HotpotQA MAC 90% 0 0 1.913 3.542 3415.736201 44.803 0 0.144 0.196 0.420
25 HotpotQA MAC 95% 0 0 5.783 6.764 9797.244043 140.62 0 0.144 0.196 0.420