Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
docs=151 chars=1,450,048 chunks=2713

Loading weights:   0%|          | 0/103 [00:00<?, ?it/s]
Loading weights: 100%|██████████| 103/103 [00:00<00:00, 3734.30it/s]
model loaded in 0.7s
embedded 2713 chunks in 84.9s
heading queries: 773
identifier queries: 305
paraphrase queries: 30

Pre-computing BM25 and vector rankings for all query sets...

 alpha | heading MRR | ident MRR | paraphrase hit@1  hit@5    MRR
   0.0 |       0.491 |     0.279 |            0.667  0.933  0.768
   0.1 |       0.544 |     0.394 |            0.667  0.967  0.787
   0.2 |       0.581 |     0.471 |            0.667  1.000  0.794
   0.3 |       0.610 |     0.515 |            0.633  0.967  0.789
   0.4 |       0.642 |     0.569 |            0.667  0.967  0.806
   0.5 |       0.679 |     0.696 |            0.567  0.967  0.751
   0.6 |       0.710 |     0.735 |            0.533  0.967  0.731
   0.7 |       0.744 |     0.740 |            0.533  0.967  0.731
   0.8 |       0.782 |     0.753 |            0.533  0.933  0.722
   0.9 |       0.813 |     0.768 |            0.500  0.900  0.678
   1.0 |       0.859 |     0.772 |            0.367  0.867  0.566

 alpha | heading hit@1  hit@5    MRR | ident hit@1  hit@5    MRR | para hit@1  hit@5    MRR
   0.0 |         0.382  0.657  0.491 |       0.207  0.397  0.279 |      0.667  0.933  0.768
   0.1 |         0.437  0.700  0.544 |       0.338  0.489  0.394 |      0.667  0.967  0.787
   0.2 |         0.473  0.732  0.581 |       0.420  0.554  0.471 |      0.667  1.000  0.794
   0.3 |         0.505  0.765  0.610 |       0.459  0.603  0.515 |      0.633  0.967  0.789
   0.4 |         0.545  0.779  0.642 |       0.515  0.656  0.569 |      0.667  0.967  0.806
   0.5 |         0.586  0.812  0.679 |       0.607  0.830  0.696 |      0.567  0.967  0.751
   0.6 |         0.625  0.840  0.710 |       0.652  0.856  0.735 |      0.533  0.967  0.731
   0.7 |         0.652  0.887  0.744 |       0.652  0.869  0.740 |      0.533  0.967  0.731
   0.8 |         0.692  0.931  0.782 |       0.662  0.889  0.753 |      0.533  0.933  0.722
   0.9 |         0.721  0.957  0.813 |       0.685  0.889  0.768 |      0.500  0.900  0.678
   1.0 |         0.785  0.966  0.859 |       0.692  0.889  0.772 |      0.367  0.867  0.566

best alpha for heading MRR: 1.0 (0.859)
best alpha for identifier MRR: 1.0 (0.772)
best alpha for paraphrase MRR: 0.4 (0.806)
best alpha for sum of all three MRR: 0.9 (heading 0.813 + identifier 0.768 + paraphrase 0.678)

median single-query latency over 2713 chunks: BM25=1.82ms, vector(encode+search)=23.99ms