# BM25

Retrieving documents using BM25. We must always encode documents before queries when 
using BM25 retriever in order to fit the vectorizer, otherwise the system will raise an error.
BM25 takes as input two distinct parameters, `b` and `k1`. b is a float value that determines
the impact of document length normalization. The default value is `0.75`. The higher the value, the
more penalized longer documents will be. k1 is a float value that determines how quickly the impact
of term frequency saturates. The default value is `1.5`. The higher the value, the more influential
term frequency will be.

```python
from neural_cherche import retrieve
from lenlp import sparse

documents = [
    {"id": "doc1", "title": "Paris", "text": "Paris is the capital of France."},
    {"id": "doc2", "title": "Montreal", "text": "Montreal is the largest city in Quebec."},
    {"id": "doc3", "title": "Bordeaux", "text": "Bordeaux in Southwestern France."},
]

retriever = retrieve.BM25(
    key="id",
    on=["title", "text"],
    count_vectorizer=sparse.CountVectorizer(normalize=True, ngram_range=(3, 5), analyzer="char_wb"),
    k1=1.5,
    b=0.75,
    epsilon=0.,
)

documents_embeddings = retriever.encode_documents(
    documents=documents,
)

retriever.add(
    documents_embeddings=documents_embeddings,
)
```

Once we have created our index, we can use the retriever to retrieve the candidates.

```python
queries = [
    "What is the capital of France?",
    "What is the largest city in Quebec?",
    "Where is Bordeaux?",
]

queries_embeddings = retriever.encode_queries(
    queries=queries,
)

scores = retriever(
    queries_embeddings=queries_embeddings,
    k=100,
)

scores
```

```python
[[{'id': 'doc1', 'similarity': 88.86143220961094},
  {'id': 'doc2', 'similarity': 8.409232541918755},
  {'id': 'doc3', 'similarity': 7.134543210268021}],
 [{'id': 'doc2', 'similarity': 107.05374336242676},
  {'id': 'doc1', 'similarity': 9.28911879658699},
  {'id': 'doc3', 'similarity': 1.9025448560714722}],
 [{'id': 'doc3', 'similarity': 18.506150543689728},
  {'id': 'doc1', 'similarity': 0.7961864173412323},
  {'id': 'doc2', 'similarity': 0.7676786482334137}]]
```