Build a QA System Without Elasticsearch

Open In Colab

Haystack provides alternatives to Elasticsearch for developing quick prototypes.

You can use an InMemoryDocumentStore or a SQLDocumentStore(with SQLite) as the document store.

If you are interested in more feature-rich Elasticsearch, then please refer to the Tutorial 1.

Prepare environment

Colab: Enable the GPU runtime

Make sure you enable the GPU runtime to experience decent speed in this tutorial. Runtime -> Change Runtime type -> Hardware accelerator -> GPU

# Make sure you have a GPU running
# Install the latest release of Haystack in your own environment 
#! pip install farm-haystack

# Install the latest master of Haystack
!pip install git+
!pip install urllib3==1.25.4
!pip install torch==1.6.0+cu101 torchvision==0.6.1+cu101 -f
from haystack import Finder
from import clean_wiki_text
from haystack.preprocessor.utils import convert_files_to_dicts, fetch_archive_from_http
from import FARMReader
from haystack.reader.transformers import TransformersReader
from haystack.utils import print_answers

Document Store

# In-Memory Document Store
from haystack.document_store.memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
# SQLite Document Store
# from haystack.document_store.sql import SQLDocumentStore
# document_store = SQLDocumentStore(url="sqlite:///qa.db")

Preprocessing of documents

Haystack provides a customizable pipeline for:

  • converting files into texts
  • cleaning texts
  • splitting texts
  • writing them to a Document Store

In this tutorial, we download Wikipedia articles on Game of Thrones, apply a basic cleaning function, and index them in Elasticsearch.

# Let's first get some documents that we want to query
# Here: 517 Wikipedia articles for Game of Thrones
doc_dir = "data/article_txt_got"
s3_url = ""
fetch_archive_from_http(url=s3_url, output_dir=doc_dir)

# convert files to dicts containing documents that can be indexed to our datastore
# You can optionally supply a cleaning function that is applied to each doc (e.g. to remove footers)
# It must take a str as input, and return a str.
dicts = convert_files_to_dicts(dir_path=doc_dir, clean_func=clean_wiki_text, split_paragraphs=True)

# We now have a list of dictionaries that we can write to our document store.
# If your texts come from a different source (e.g. a DB), you can of course skip convert_files_to_dicts() and create the dictionaries yourself.
# The default format here is: {"name": "<some-document-name>, "text": "<the-actual-text>"}

# Let's have a look at the first 3 entries:
# Now, let's write the docs to our DB.

Initalize Retriever, Reader, & Finder


Retrievers help narrowing down the scope for the Reader to smaller units of text where a given question could be answered.

With InMemoryDocumentStore or SQLDocumentStore, you can use the TfidfRetriever. For more retrievers, please refer to the tutorial-1.

# An in-memory TfidfRetriever based on Pandas dataframes

from haystack.retriever.sparse import TfidfRetriever
retriever = TfidfRetriever(document_store=document_store)


A Reader scans the texts returned by retrievers in detail and extracts the k best answers. They are based on powerful, but slower deep learning models.

Haystack currently supports Readers based on the frameworks FARM and Transformers. With both you can either load a local model or one from Hugging Face's model hub (

Here: a medium sized RoBERTa QA model using a Reader based on FARM (

Alternatives (Reader): TransformersReader (leveraging the pipeline of the Transformers package)

Alternatives (Models): e.g. "distilbert-base-uncased-distilled-squad" (fast) or "deepset/bert-large-uncased-whole-word-masking-squad2" (good accuracy)

Hint: You can adjust the model to return "no answer possible" with the noansboost. Higher values mean the model prefers "no answer possible"


# Load a  local model or any of the QA models on
# Hugging Face's model hub (

reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2", use_gpu=True)


# Alternative:
# reader = TransformersReader(model_name_or_path="distilbert-base-uncased-distilled-squad", tokenizer="distilbert-base-uncased", use_gpu=-1)


The Finder sticks together reader and retriever in a pipeline to answer our actual questions.

finder = Finder(reader, retriever)

Voilà! Ask a question!

# You can configure how many candidates the reader and retriever shall return
# The higher top_k_retriever, the better (but also the slower) your answers. 
prediction = finder.get_answers(question="Who is the father of Arya Stark?", top_k_retriever=10, top_k_reader=5)
# prediction = finder.get_answers(question="Who created the Dothraki vocabulary?", top_k_reader=5)
# prediction = finder.get_answers(question="Who is the sister of Sansa?", top_k_reader=5)
print_answers(prediction, details="minimal")
© 2020 - 2021 deepset. All rights reserved.Imprint