Build Hybrid Search for RAG in Python

build hybrid search for rag in python

What You Will Build

In this tutorial, you will build hybrid search for RAG in Python by combining BM25 keyword search with FAISS semantic search. Instead of relying on a single retrieval method, your Retrieval-Augmented Generation (RAG) system will use both exact keyword matching and vector similarity to retrieve more relevant documents.

Hybrid search is widely used in production AI applications because keyword search and semantic search complement each other. BM25 excels at finding exact terms, product names, error codes, and technical identifiers, while FAISS retrieves documents based on semantic meaning even when different words are used.

By combining these two approaches, you can significantly improve retrieval quality and reduce the chance of missing important documents. The result is a more accurate Retrieval-Augmented Generation (RAG) pipeline capable of handling both precise technical queries and natural language questions.

What You Will Learn

  • Why semantic search alone is not always sufficient
  • How BM25 keyword search works
  • How to combine BM25 with FAISS semantic search
  • How to merge and rank hybrid search results
  • How to build hybrid search for RAG in Python using a production-oriented retrieval pipeline

Prerequisites

Before you build hybrid search for RAG in Python, make sure you have completed the previous tutorials in this series. This guide extends the same Retrieval-Augmented Generation (RAG) pipeline by combining keyword search with semantic search to improve document retrieval.

Your project should already include:

  • Python 3.10 or later
  • An OpenAI API key
  • A FAISS vector index
  • Sentence Transformers for generating embeddings
  • A collection of indexed document chunks
  • Metadata filtering (recommended)

Install the required libraries:

pip install openai sentence-transformers faiss-cpu rank-bm25 numpy

The rank-bm25 library provides a lightweight implementation of the BM25 ranking algorithm, making it easy to combine keyword search with semantic retrieval.

The official OpenAI Developer Documentation provides API references, SDK examples, and authentication guides for integrating large language models into Retrieval-Augmented Generation (RAG) applications.

OpenAI API Documentation

To successfully build hybrid search for RAG in Python, your application should already be able to retrieve documents using FAISS. In this tutorial, you’ll add BM25 keyword search, merge the results from both retrieval methods, and create a hybrid search pipeline that delivers more accurate and reliable answers than either approach alone.

Step 1 — Why Semantic Search Is Not Enough

The first step to build hybrid search for RAG in Python is understanding the limitations of semantic search. While vector search is excellent at finding documents with similar meaning, it is not always the best choice for every type of query.

Semantic search compares vector embeddings rather than exact words. This allows a Retrieval-Augmented Generation (RAG) system to retrieve relevant documents even when the user’s question uses different terminology. However, semantic similarity can sometimes overlook documents containing critical keywords, product names, model numbers, error codes, or technical identifiers.

Consider the following query:

How do I configure error E102 on PLC-X200?

A semantic search engine understands the general meaning of the question, but it may rank documents discussing PLC configuration higher than the document that specifically contains the exact error code E102.

Keyword search behaves differently. Instead of comparing meanings, it searches for exact terms:

Query:
E102 PLC-X200

↓

Documents containing:

"E102"

"PLC-X200"

"Error E102"

In this situation, BM25 immediately identifies documents containing the exact identifiers, even if the surrounding text is not semantically similar.

Now consider a different query:

How can I improve semantic retrieval in a RAG system?

Here, semantic search performs much better because relevant documents may use phrases such as vector search, embedding similarity, or retrieval optimization without repeating the exact words from the user’s question.

This is why developers build hybrid search for RAG in Python. BM25 and FAISS solve different retrieval problems. BM25 excels at exact keyword matching, while FAISS excels at semantic similarity. By combining both methods, a Retrieval-Augmented Generation (RAG) system can retrieve documents that are both semantically relevant and contain the precise technical terms the user is looking for.

In the next step, you’ll implement BM25 keyword search and use it alongside your existing FAISS vector search engine.

Step 2 — Implement BM25 Keyword Search

Now it’s time to build hybrid search for RAG in Python by adding BM25 keyword search to your retrieval pipeline. Unlike FAISS, which compares vector embeddings, BM25 ranks documents according to the exact terms that appear in both the user’s query and the document text.

BM25 has been one of the most widely used ranking algorithms in information retrieval for many years. Modern search engines still rely on BM25 because it performs exceptionally well when users search for exact phrases, product names, error codes, file names, or technical terminology.

Begin by importing the BM25 implementation:

from rank_bm25 import BM25Okapi

Next, tokenize every document in your knowledge base:

tokenized_documents = [
    document["text"].lower().split()
    for document in documents
]

Create the BM25 index:

bm25 = BM25Okapi(tokenized_documents)

When a user submits a query, tokenize it using the same approach:

query = "How do I configure error E102?"

tokenized_query = query.lower().split()

Retrieve the BM25 relevance scores:

scores = bm25.get_scores(tokenized_query)

Select the highest-ranked documents:

import numpy as np

top_indices = np.argsort(scores)[::-1][:5]

bm25_results = [
    documents[i]
    for i in top_indices
]

Unlike semantic search, BM25 does not require embeddings or vector databases. It simply analyzes the distribution of query terms across the document collection and assigns higher scores to documents containing important keywords.

This is one of the key reasons developers build hybrid search for RAG in Python. BM25 retrieves documents that contain exact technical terms, while FAISS retrieves documents with similar meaning. Individually, each approach has limitations. Together, they provide much more reliable retrieval across a wide range of user queries.

In the next step, you’ll combine BM25 keyword search with FAISS semantic search and merge their results into a single hybrid retrieval pipeline.

Step 3 — Combine BM25 with FAISS

Now it’s time to build hybrid search for RAG in Python by combining the strengths of BM25 keyword search and FAISS semantic search. Instead of choosing one retrieval method over the other, your Retrieval-Augmented Generation (RAG) pipeline will execute both searches and merge their results.

The hybrid retrieval workflow looks like this:

User Question
        │
        ▼
Generate Query
        │
        ├─────────────┐
        ▼             ▼
    BM25 Search   FAISS Search
        │             │
        └──────┬──────┘
               ▼
        Merge Results
               │
               ▼
        Remove Duplicates
               │
               ▼
         Rank Documents
               │
               ▼
         Send to LLM

Run BM25 and FAISS independently:

bm25_results = bm25_search(
    query,
    top_k=5
)

faiss_results = faiss_search(
    query,
    top_k=5
)

Merge the retrieved documents:

combined_results = (
    bm25_results +
    faiss_results
)

Remove duplicate documents:

unique_results = {
    document["id"]: document
    for document in combined_results
}

results = list(
    unique_results.values()
)

At this stage, every retrieved document comes from either BM25, FAISS, or both. Documents appearing in both result sets are often strong candidates because they match both exact keywords and semantic meaning.

This is the core idea behind build hybrid search for RAG in Python. BM25 contributes lexical relevance, while FAISS contributes semantic relevance. By merging both retrieval methods, the Retrieval-Augmented Generation (RAG) system gains a much broader understanding of the user’s intent than either search engine can provide individually.

The merged list is not yet ordered optimally because BM25 scores and FAISS similarity scores use different scales. In the next step, you’ll learn how to normalize these scores and rank the combined results to produce a single, high-quality retrieval list.

Step 4 — Rank Hybrid Results

After combining BM25 and FAISS results, the next step to build hybrid search for RAG in Python is ranking the merged documents. Simply concatenating the two result lists is not enough because BM25 and FAISS use completely different scoring systems.

BM25 produces relevance scores based on keyword frequency, while FAISS returns similarity scores based on vector distances or cosine similarity. Since these scores are measured on different scales, they cannot be compared directly.

A common solution is to normalize both scores before combining them.

For example, each retrieval method can assign a normalized score between 0 and 1:

{
    "id": 15,
    "bm25_score": 0.82,
    "faiss_score": 0.91
}

Then calculate a weighted hybrid score:

hybrid_score = (
    0.4 * bm25_score +
    0.6 * faiss_score
)

You can implement this directly:

for document in results:

    document["hybrid_score"] = (
        0.4 * document["bm25_score"] +
        0.6 * document["faiss_score"]
    )

results.sort(
    key=lambda x: x["hybrid_score"],
    reverse=True
)

The weighting depends on your application. Technical documentation often benefits from a higher BM25 weight because exact keywords such as product names, error codes, or API methods are extremely important. General question-answering systems usually give more weight to semantic similarity.

For example:

  • Technical documentation: BM25 = 0.6, FAISS = 0.4
  • General knowledge: BM25 = 0.3, FAISS = 0.7
  • Balanced retrieval: BM25 = 0.5, FAISS = 0.5

This flexibility is one of the main reasons developers build hybrid search for RAG in Python. Instead of relying entirely on keyword matching or semantic similarity, the retrieval pipeline can be tuned for different domains while still using the same architecture.

After ranking the merged results, the Retrieval-Augmented Generation (RAG) pipeline now has a single ordered list of the most relevant document chunks. In the next step, you’ll integrate this hybrid retrieval process into a complete RAG assistant that generates answers using both keyword relevance and semantic similarity.

Step 5 — Build a Hybrid RAG Assistant

Now it’s time to bring everything together and build hybrid search for RAG in Python using a complete production-style retrieval pipeline. Instead of relying on a single search method, your assistant will combine metadata filtering, BM25 keyword search, FAISS semantic search, and a Large Language Model to generate accurate answers.

The complete Retrieval-Augmented Generation (RAG) workflow now looks like this:

User Question
      │
      ▼
Metadata Filtering
      │
      ▼
      ├─────────────┐
      ▼             ▼
 BM25 Search   FAISS Search
      │             │
      └──────┬──────┘
             ▼
     Merge & Rank Results
             │
             ▼
      Build Context
             │
             ▼
        OpenAI API
             │
             ▼
     Generate Answer

A simplified implementation might look like this:

def ask_rag(question):

    filtered_docs = filter_documents(
        documents,
        metadata_filter
    )

    bm25_results = bm25_search(
        filtered_docs,
        question,
        top_k=5
    )

    faiss_results = faiss_search(
        filtered_docs,
        question,
        top_k=5
    )

    ranked_results = hybrid_rank(
        bm25_results,
        faiss_results
    )

    context = "\n\n".join(
        doc["text"]
        for doc in ranked_results[:5]
    )

    return generate_answer(
        question,
        context
    )

Notice that the language model never searches the knowledge base directly. Its only responsibility is generating an answer from the retrieved context. The retrieval pipeline performs all document selection before the prompt reaches the LLM.

This separation of responsibilities is one of the defining characteristics of modern Retrieval-Augmented Generation (RAG) systems. Metadata filtering reduces the search space, BM25 retrieves documents containing important keywords, FAISS contributes semantic understanding, and the language model focuses entirely on answer generation.

When you build hybrid search for RAG in Python, this layered retrieval architecture provides several important advantages:

  • Higher retrieval accuracy
  • Better handling of technical terminology
  • Improved semantic understanding
  • Lower hallucination rates
  • More reliable answers from the language model

This architecture is widely used in enterprise AI assistants, document search systems, customer support platforms, and internal knowledge bases because it combines the strengths of both lexical and semantic retrieval while remaining scalable to very large document collections.

In the next section, you’ll compare hybrid search with pure semantic search using several real-world examples to see where hybrid retrieval delivers the greatest improvements.

Test Hybrid Search

After you build hybrid search for RAG in Python, it’s important to compare its performance with standalone semantic search and standalone keyword search. Testing different query types clearly demonstrates why hybrid retrieval has become the preferred approach for many production Retrieval-Augmented Generation (RAG) systems.

Consider the following technical query:

How do I fix error E102 on PLC-X200?

Using only FAISS semantic search may retrieve documents discussing PLC configuration or troubleshooting in general, but it could miss the document containing the exact error code E102.

Using only BM25 keyword search retrieves documents containing E102 and PLC-X200, but it may overlook semantically related troubleshooting guides that use different wording.

Hybrid search combines both approaches:

User Question
        │
        ▼
Metadata Filtering
        │
        ▼
        ├─────────────┐
        ▼             ▼
    BM25 Search   FAISS Search
        │             │
        └──────┬──────┘
               ▼
      Merge & Rank Results
               │
               ▼
      Top Relevant Documents

Now consider a more natural-language question:

How can I improve document retrieval in my RAG system?

In this case, FAISS retrieves documents discussing semantic search, vector similarity, retrieval optimization, and embeddings, even when those exact words do not appear in the query. BM25 contributes documents containing precise terms such as retrieval, RAG, and documents. Together, they produce a more complete and balanced result set.

These examples illustrate why developers build hybrid search for RAG in Python. Exact keyword matching captures critical technical terms, while semantic search understands user intent. Combining both methods produces retrieval results that are consistently more accurate than relying on either approach alone.

As your knowledge base grows, the advantages become even more apparent. Hybrid retrieval reduces the chance of missing important documents, improves answer quality, and provides a stronger foundation for enterprise AI assistants, document search platforms, and Retrieval-Augmented Generation (RAG) applications operating at scale.

In the next section, you’ll explore where to go next and learn how reranking can further improve retrieval quality after hybrid search has selected the most relevant candidate documents.

Where to Go Next

Congratulations! You have successfully learned how to build hybrid search for RAG in Python by combining BM25 keyword search with FAISS semantic search. Your Retrieval-Augmented Generation (RAG) system can now retrieve documents using both exact keyword matching and semantic similarity, producing more accurate and reliable results.

Hybrid search is one of the core components of modern production RAG systems. However, the retrieved documents are not always ordered perfectly. Even after combining BM25 and FAISS, some highly relevant documents may still appear lower in the ranking than they should.

Continue expanding your RAG pipeline with these tutorials:

By combining metadata filtering, hybrid retrieval, reranking, conversation memory, and semantic search, you’ll create a Retrieval-Augmented Generation (RAG) system capable of handling complex enterprise knowledge bases with high retrieval accuracy and low hallucination rates.

In the next tutorial, you’ll learn how to build Reranking for RAG in Python, allowing your retrieval pipeline to reorder candidate documents and deliver even more relevant context to the language model.

Frequently Asked Questions

Why should I build hybrid search for RAG in Python?

Hybrid search combines BM25 keyword search with FAISS semantic search, allowing a Retrieval-Augmented Generation (RAG) system to retrieve documents using both exact keyword matching and semantic similarity. This typically produces more accurate results than using either retrieval method alone.

What is the difference between BM25 and FAISS?

BM25 ranks documents based on exact keyword matches and term frequency, making it ideal for technical identifiers, product names, and error codes. FAISS performs semantic search using vector embeddings, allowing it to retrieve documents with similar meaning even when different words are used.

Should I always use hybrid search instead of semantic search?

Not necessarily. Small knowledge bases containing general text often perform well with semantic search alone. However, if your documents contain technical terminology, product codes, API names, or other exact identifiers, hybrid retrieval usually provides significantly better results.

Can hybrid search work together with metadata filtering?

Yes. In production Retrieval-Augmented Generation (RAG) systems, metadata filtering is typically applied first to reduce the search space. BM25 and FAISS then retrieve documents only from the filtered subset, improving both retrieval speed and answer quality.

What is the best way to build hybrid search for RAG in Python?

A production-ready implementation combines metadata filtering, BM25 keyword search, FAISS semantic search, score normalization, and result ranking before sending the final context to the language model. This layered retrieval pipeline delivers accurate, scalable, and reliable document retrieval for modern AI applications.

Conclusion

In this tutorial, you learned how to build hybrid search for RAG in Python by combining BM25 keyword search with FAISS semantic search. Instead of relying on a single retrieval method, your Retrieval-Augmented Generation (RAG) system now benefits from both exact keyword matching and semantic understanding.

You implemented the complete hybrid retrieval pipeline, including BM25 indexing, FAISS vector search, result merging, score ranking, and integration with a production-style RAG assistant. Together, these components provide higher retrieval accuracy while reducing the risk of missing important documents.

Hybrid retrieval has become a standard architecture for modern AI applications because it performs well across a wide variety of queries. Exact keyword search captures technical terms, product names, and error codes, while semantic search understands user intent and retrieves conceptually related information. Combining both methods produces more reliable context for the language model and leads to higher-quality answers.

As your Retrieval-Augmented Generation (RAG) system continues to evolve, the next improvement is reranking. A reranking model examines the documents retrieved by hybrid search and reorders them according to their relevance to the user’s question. This final ranking step often provides another significant improvement in retrieval quality before the information is sent to the language model.

In the next tutorial, you’ll learn how to build Reranking for RAG in Python and complete another essential component of a production-ready retrieval pipeline.

Scroll to Top