Integration: fastRAG

fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines

Intel Labs

fastRAG is a research framework, that extends Haystack, with abilities to build efficient and optimized retrieval augmented generative pipelines (with emphasis on Intel hardware), incorporating state-of-the-art LLMs and Information Retrieval modules.

Key Features

  • Optimized RAG: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
  • Optimized for Intel Hardware: Leverage Intel extensions for PyTorch (IPEX), 🤗 Optimum Intel and 🤗 Optimum-Habana for running as optimal as possible on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
  • Customizable: fastRAG is built using Haystack and HuggingFace. All of fastRAG’s components are 100% Haystack compatible.


For a brief overview of the various unique components in fastRAG refer to the Components Overview page.

LLM Backends
Intel Gaudi Accelerators Running LLMs on Gaudi 2
ONNX Runtime Running LLMs with optimized ONNX-runtime
Llama-CPP Running RAG Pipelines with LLMs on a Llama CPP backend
Optimized Components
Embedders Optimized int8 bi-encoders
Rankers Optimized/sparse cross-encoders
RAG-efficient Components
ColBERT Token-based late interaction
Fusion-in-Decoder (FiD) Generative multi-document encoder-decoder
REPLUG Improved multi-document decoder
PLAID Incredibly efficient indexing engine


Preliminary requirements:

  • Python 3.8 or higher.
  • PyTorch 2.0 or higher.

To set up the software, clone the project and run the following, preferably in a newly created virtual environment:

git clone
cd fastrag

There are several dependencies to consider, depending on your specific usage:

Basic installation:

pip install .

fastRAG with Intel-optimized backend:

pip install .[intel]

Other installation options can be found here.